YouSaid · the spoken record

Dario Amodei

lines on the record
181
first
2024-11-11
most recent
2024-11-11
sittings or episodes
1
sources
podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

  1. “Yeah, yeah, it's something both the human and the AI system can read. So it has this nice kind of translatability or symmetry. You know, in practice, we both use a model constitution and we use RLHF and we use some of these other methods. So it's turned into one tool in a toolkit that both reduces the need for RLHF and increases the value we get from using each data point of RLHF. It also interacts in interesting ways with kind of future reasoning type RL methods. So it's one tool in the toolkit, but I think it is a very important tool.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  2. “So, two ideas. One is could the AI system itself decide which response is better, right? Could you show the AI system these two responses and ask which response is better? And then second, well, what criterion should the AI use? And so then there's this idea because you have a single document, a constitution, if you will, that says these are the principles the model should be using to respond. And the AI system reads those principles as well as reading the environment and the response. And it says, well, how good did the AI model do? It's basically a form of self-play. You're kind of training the model against itself. And so the AI gives the response and then you feed that back into what's called the preference model, which in turn feeds the model to make it better. So you have this triangle of like the AI, the preference model, and the improvement of the AI itself.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  3. “Yes, so this was from two years ago. The basic idea is, so we describe what RLHF is. You have a model and it spits out like you just sample from it twice. It spits out two possible responses and you're like human, which responds you like better or another variant of it is rate this response on a scale of one to seven. So that's hard because you need to scale up human interaction. And it's very implicit, right? I don't have a sense of what I want the model to do. I just have a sense of like what this”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  4. “I don't think you can scale up humans enough to get high quality, any kind of method that relies on humans and uses a large amount of compute, it's going to have to rely on some scaled supervision method, like debate or iterated amplification or something like that”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  5. “At the present moment, it is still the case that pre-training is the majority of the cost. I don't know what to expect in the future, but I could certainly anticipate a future where post-training is the majority of the cost.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  6. “It also increases what was this word in Leopold's essay, Unhobbling, where basically the models are hobbled and then you do various trainings to them to unhobble them. I like that word because it's like a rare word, but so I think RLHF unhobbles the models in some ways. And then there are other ways where a model hasn't yet been unhobbled and needs to unhobble.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  7. “Don't think it makes the model smarter. I don't think it just makes the model appear smarter. It's like RLHF bridges the gap between the human and the model, right? I could have something really smart that like can't communicate at all, right? We all know people like this, people who are really smart, but you can't understand what they're saying. So I think RLHF just bridges that gap. I think it's not the only kind of RL we do. It's not the only kind of RL that will happen in the future. I think RL has the potential to make models smarter, to make them reason better, to make them operate better, to make them develop new skills even. And perhaps that could be done even in some cases with human feedback, but the kind of RLHF we do today mostly doesn't do that yet, although we're very quickly starting to be able to.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  8. “Trained model being halfway to anywhere. So once you have the pre trained model, you have all the representations you need to get the model where you want it to go.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  9. “If I go back to like the scaling hypothesis, one of the ways to skate the scaling hypothesis is if you train for X and you throw enough compute at it, then you get X. And so RLIGF is good at doing what humans want the model to do, or at least to state it more precisely, doing what humans who look at the model for a brief period of time and consider different possible responses, what they prefer as the response, which is not perfect from both a safety and capabilities perspective in that humans are often not able to perfectly identify what the model wants and what humans want in the moment may not be what they want in the long term. So there's a lot of subtlety there, but the models are good at producing what the humans in some shallow sense want. And it actually turns out that you don't even have to throw that much compute at it because of another thing, which is this thing about a strong pre-”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  10. “Usually, it isn't, oh my God, we have this secret magic method that others don't have, right? Usually, it's like, well, you know, we got better at the infrastructure, so we could run it for longer, or, you know, we were able to get higher quality data, or we were able to filter our data better, or we were able to combine these methods in practice. It's usually some boring matter of kind of practice and tradecraft. So, you know, when I think about how to do something special in terms of how we train these models, both pre-training, but even more so post-training, you know, I really think of it a little more, again, as like designing airplanes or cars. Like, you know, it's not just like, oh, man, I have the blueprint. Like maybe that makes you make the next airplane, but like, there's some cultural trade craft of how we think about the design process that I think is more important than any particular gizmo we're able to invent.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  11. “Yeah, I mean, so first of all, we're not perfectly able to measure that ourselves. When you see some great character ability, sometimes it's hard to tell whether it came from pre-training or post-training. We've developed ways to try and distinguish between those two, but they're not perfect. The second thing I would say is, you know, when there isn't advantage, and I think we've been pretty good in general at RL, perhaps the best, although I don't know because I don't see what goes on inside other companies.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  12. “Long long horizon learning and long horizon tasks where there's a lot to be done. I think evaluations are still, we're still very early in our ability to study evaluations, particularly for dynamic systems acting in the world. I think there's some stuff around multi-agent. Skate where the puck is going is my advice. And you don't have to be brilliant to think of it. All the things that are going to be exciting in five years, like people even mention them as like, you know, conventional wisdom, but like it's just somehow there's this barrier that people don't double down as much as they could or they're afraid to do something that's not the popular thing. I don't know why it happens, but like getting over that barrier, that's my number one piece of advice.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  13. “I think my number one piece of advice is to just start playing with the models. Actually, I worry a little, this seems like obvious advice now. I think three years ago, it wasn't obvious and people started by, oh, let me read the latest reinforcement learning paper. Let me kind of, no, I mean, that was really the, and I mean, you should do that as well. But now, you know, with wider availability of models and APIs, people are doing this more. But I think just experiential knowledge. These models are new artifacts that no one really understands. And so getting experience playing with them. I would also say, again, in line with the like, do something new, think in some new direction. Like there are all these things that haven't been explored. Like, for example, mechanist.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  14. “Another example of this like some of the early work in mechanistic interpretability so simple. It's just no one thought to care about this question before.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  15. “Digit number of people have driven forward the whole field by realizing this. And it's often like that if you look back at the discovery, you know, the discoveries in history, they're often like that. And so this open-mindedness and this willingness to see with new eyes that often comes from being newer to the field, often experience is a disadvantage for this. That is the most important thing. It's very hard to look for and test for, but I think it's the most important thing because when you find something, some really new way of thinking about things, when you have the initiative to do that, it's absolutely transformative.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  16. “Oh, you know, we don't have the right algorithms yet. We haven't come up with the right way to do things. And I was just like, oh, I don't know. This neural net has like 30 million parameters. Like what if we gave it 50 million instead? Like, let's plot some graphs. Like, that basic scientific mindset of like, oh man, like, I just like, you know, I see some variable that I could change. Like what happens when it changes? Like, let's try these different things and like create a graph. For even this, this was like the simplest thing in the world, right? Change the number of, you know, this wasn't like PhD level experimental design. This was like, this was like simple and stupid. Like anyone could have done this if you just told them that it was important. It's also not hard to understand. You didn't need to be brilliant to come up with this, but you put the two things together. And, you know, some tiny number of people, some single.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  17. “Yeah. I think the number one quality, especially on the research side, but really both, is open-mindedness. Sounds easy to be open-minded, right? You're just like, oh, I'm open to anything. But if I think about my own early history in the scaling hypothesis, I was seeing the same data others were seen. I don't think I was like a better programmer or better at coming up with research ideas than any of the hundreds of people that I worked with. In some ways, I was worse. You know, like I've never precise programming of like, you know, finding the bug, writing the GPU kernels, like I could point you to 100 people here who are better, who are better at that than I am. But the thing that I think I did have that was different was that I was just willing to look at something with new eyes, right?”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  18. “Different fiefdoms that all want to do their own thing, that are all optimizing for their own thing. It's very hard to get anything done. But if everyone sees the broader purpose of the company, if there's trust and there's dedication to doing the right thing, that is a superpower. That in itself, I think, can overcome almost every other disadvantage.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  19. “And now we've slowed down. We're at like, you know, last three months, we went from 800 to 900, 950, something like that. Don't quote me on the exact numbers, but I think there's an inflection point around 1,000, and we want to be much more careful how we grow Early on and now as well, you know, we've hired a lot of physicists, theoretical physicists can learn things really fast. Even more recently, as we've continued to hire that, we really had a high bar for on both the research side and the software engineering side have hired a lot of senior people, including folks who used to be at other companies in this space. And we've just continued to be very selective. It's very easy to go from 100 to 1,000 and 1,000 to 10,000 without paying attention to making sure everyone has a unified purpose. It's so powerful. If your company consists of a lot of”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  20. “Talented and super dedicated. That sets the tone for everything, right? That sets the tone for everyone is super inspired to work at the same place. Everyone trusts everyone else. If you have a thousand or ten thousand people and things have really regressed, right? You are not able to do selection and you're choosing random people. What happens is then you need to put a lot of processes and a lot of guardrails in place just because people don't fully trust each other or you have to adjudicate political battles like there are so many things that slow down the org's ability to operate. And so we're nearly a thousand people. And, you know, we've tried to make it so that as large a fraction of those thousand people as possible are like super talented, super skilled. It's one of the reasons we've slowed down hiring a lot in the last few months. We grew from 300 to 800, I believe, I think, in the first seven, eight months of the year.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  21. “This is one of these statements that's more true every month, every month. I see this statement as more true than I did the month before. So if I were to do a thought experiment, let's say you have a team of 100 people that are super smart, motivated and aligned with the mission, and that's your company. Or you can have a team of 1,000 people where 200 people are super smart, super aligned with the mission, and then like And then like 800 people are let's just say you pick 800 like random random big tech employees. Which would you rather have right? The talent mass is greater in the group of in the group of a thousand people, right? You have even a larger number of incredibly talented, incredibly aligned, incredibly smart people. But the issue is just that if every time someone super talented looks around, they see someone else super talented.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  22. “Also, be successful, and some will be more successful than others. That's less important than, again, that we align the incentives of the industry. And that happens partly through the race to the top, partly through things like RSP, partly through, again, selected surgical regulation.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  23. “Look, I'm sure we've made plenty of mistakes along the way. The perfect organization doesn't exist. It has to deal with the imperfection of a thousand employees. It has to deal with the imperfection of our leaders, including me. It has to deal with the imperfection of the people we've put to oversee the imperfection of the leaders, like the board and the long-term benefit trust. It's all a set of imperfect people trying to aim imperfectly at some ideal that will never perfectly be achieved. That's what you sign up for. That's what it will always be. But imperfect doesn't mean you just give up. There's better and there's worse. And hopefully, hopefully, we can begin to do well enough that we can begin to build some practices that the whole industry engages in. And then, you know, my guess is that multiple of these companies will be successful. Andropic will be successful. These other companies, like once I've been at the past, will.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  24. “Dynamic is what we should be pointing at. And I think it abstracts away the question of which company's winning, who trusts who I think all these questions of drama are profoundly uninteresting. And the thing that matters is the ecosystem that we all operate in and how to make that ecosystem better because that constrains all the players.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  25. “In good practices. Then at the end of the day, it doesn't matter who ends up winning. It doesn't even matter who started the race to the top. The point isn't to be virtuous. The point is to get the system into a better equilibrium than it was before. And individual companies can play some role in doing this. Individual companies can help to start it, can help to accelerate it. And frankly, I think individuals at other companies have done this as well, right? The individuals that when we put out an RSP react by pushing harder to get something similar done at other companies. Sometimes other companies do something that's like, we're like, oh, it's a good practice. We think that's good. We should adopt it too. The only difference is, you know, I think we try to be more forward-leaning. We try and adopt more of these practices first and adopt them more quickly when others invent them. But I think this.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  26. “That, you know, people find genuinely appealing. And I want it to be in substance, not just in appearance. And, you know, I think researchers are sophisticated and they look at substance. And then other companies start copying that practice and they win because they copied that practice. That's great. That's success. That's like the race to the top. It doesn't matter who wins in the end as long as everyone is copying everyone else's good practices, right? One way I think of it is like the thing we're all afraid of is the race to the bottom, right? And the race to the bottom doesn't matter who wins because we all lose, right? Like, you know, in the most extreme world, we make this autonomous AI that, you know, the robots enslave us or whatever, right? I mean, that's half joking, but that is the most extreme thing that could happen. Then it doesn't matter which company was ahead. If instead you create a race to the top where people are competing to engage in good.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  27. “Their behavior in a much more compelling way than if they're your boss and you're arguing with them. I just, I don't know how to be any more specific about it than that, but I think it's generally very unproductive to try and get someone else's vision to look like your vision. It's much more productive to go off and do a clean experiment and say, this is our vision. This is how we're going to do things. Your choice is you can ignore us. You can reject what we're doing, or you can start to become more like us and imitation is the sincerest form of flattery. And that plays out in the behavior of customers. That pays out in the behavior of the public. That plays out in the behavior of where people choose to work. And again, again, at the end, it's not about one company winning or another company winning if we or another company are engaging in some practice.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  28. “If you have a vision for how to do it, you should go off and you should do that vision. It is incredibly unproductive to try and argue with someone else's vision. You might think they're not doing it the right way. You might think they're dishonest. Who knows? Maybe you're right. Maybe you're not. But what you should do is you should take some people you trust and you should go off together and you should make your vision happen. And if your vision is compelling, if you can make it appeal to people, some combination of ethically in the market, you know, if you can, if you can make a company that's a place people want to join, that engages in practices that people think are reasonable while managing to maintain its position in the ecosystem at the same time. If you do that, people will copy it. And the fact that you are doing it, especially the fact that you're doing it better than they are, causes them to change.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  29. “Left because we didn't like the deal with Microsoft false, although it was like a lot of discussion, a lot of questions about exactly how we do the deal with Microsoft. We left because we didn't like commercialization. That's not true. We built GPD3, which was the model that was commercialized. I was involved in commercialization. It's more, again, about how do you do it? Civilization is going down this path to very powerful AI. What's the way to do it that is cautious, straightforward, honest that builds trust in the organization and an individuals? How do we get from here to there? And how do we have a real vision for how to get it right? How can safety not just be something we say because it helps with recruiting? And, you know, I think at the end of the day, if you have a vision for that, forget about anyone else's vision. I don't want to talk about anyone else's vision.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  30. “Yeah. So, look, I'm going to put things this way. And I think it ties to the race to the top, right? Which is in my time at OpenAI, what I come to see is I'd come to appreciate the scaling hypothesis and as I'd come to appreciate kind of the importance of safety along with the scaling hypothesis. The first one, I think, you know, open AI was getting on board with. The second one in a way had always been part of OpenAI's messaging. But over many years of the time that I spent there, I think I had a particular vision of how we should handle these things, how we should be brought out in the world, the kind of principles that the organization should have. And look, I mean, there were like many, many discussions about like, you know, should the org do, should the company do this? Should the company do that? Like, there's a bunch of misinformation out there. People say, like, we.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  31. “Get out of their way. Yeah. Don't impose your own ideas about how they should learn. And, you know, this was the same thing as Rich Sutton put out in the bitter lesson or Gurn put out in the scaling hypothesis. You know, I think generally the dynamic was, you know, I got this kind of inspiration from Ily and from others, folks like Alec Radford who did the original GPT-1. And then ran really hard with it, me and my collaborators on GPT-2, GPT-3, RL from Human Feedback, which was an attempt to kind of deal with the early safety and durability, things like debate and amplification, heavy on interpretability. So again, the combination of safety plus scaling, probably 2018. Many of whom became co founders of Anthropic kind of really had a vision and drove the direction.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  32. “Yeah, so I was at OpenAI for roughly five years for the last, I think it was a couple years. I was vice president of research there. Probably myself and Ilya Sutzgiver were the ones who really kind of set the research direction around 2016 or 2017. I first started to really believe in or at least confirm my belief in the scaling hypothesis when Ilya famously said to me, the thing you need to understand about these models is they just want to learn. The models just want to learn. And again, sometimes there are these one sentence, there are these one sentences, these zen cones that you hear them and you're like, ah, that explains everything. That explains like a thousand things that I've seen. And then ever after I had this visualization in my head of like, you optimize the models in the right way. You point the models in the right way. They just want to learn. They just want to solve the problem regardless of what the problem is.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  33. “Yeah, yeah, yeah, exactly. And we need to get away from this intense pro-safety versus intense anti-regulatory rhetoric, right? It's turned into these flame wars on Twitter, and nothing good's going to come of that.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  34. “The industry or hampering innovation any more necessary than it needs to. And I think for whatever reason that things got too polarized and those two groups didn't get to sit down in the way that they should. And I feel urgency. I really think we need to do something in 2025. If we get to the end of 2025 and we've still done nothing about this, then I'm going to be worried. I'm not worried yet because, again, the risks aren't here yet. But I think time is running short.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  35. “I would love it if some of the most reasonable opponents and some of the most reasonable proponents would sit down together and the different AI companies, anthropic was the only AI company that felt positively in a very detailed way. I think Elon tweeted briefly something positive. But some of the big ones like Google, OpenAI, Meta, Microsoft were pretty staunchly against. So I would really like is if, you know, some of the key stakeholders, some of the most thoughtful proponents and some of the most thoughtful opponents would sit down and say, how do we solve this problem in a way that the proponents feel brings a real reduction in risk and that the opponents feel that it is not hampering”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  36. “Seen how regulations play out in practice. And the people who have seen that understand to be very careful. If this was some lesser issue, I might be against regulation at all. But what I want the opponents to understand is that the underlying issues are actually serious. They're not something that I or the other companies are just making up because of regulatory capture. They're not sci-fi fantasies. They're not any of these things. Every time we have a new model, every few months, we measure the behavior of these models and they're getting better and better at these concerning tasks just as they are getting better and better at good, valuable, economically useful tasks. And so I would just love it if some of the former, I think SB 1047 was very polarizing.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  37. “Could is if we get something in place that is poorly targeted, that wastes a bunch of people's time, what's going to happen is people are going to say see these safety risks there, you know, this is nonsense. I just, you know, I just had to hire 10 lawyers to, you know, to fill out all these forms. I had to run all these tests for something that was clearly not dangerous. And after six months of that, there will be a groundswell and we'll end up with a durable consensus against regulation. And so I think the worst enemy of those who want real accountability is badly designed regulation. We need to actually get it right. And this is, if there's one thing I could say to the advocates, it would be that I want them to understand this dynamic better. And we need to be really careful and we need to talk to people who actually have, who actually have experience.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  38. “Bill doesn't apply if you're headquartered in California. Bill only applies if you do business in California, or that it would damage the open source ecosystem, or that it would cause all of these things. I think those were mostly nonsense, but there are better arguments against regulation. There's one guy, Dean Ball, who's really, you know, I think a very scholarly analyst who looks at what happens when a regulation is put in place and ways that they can kind of get a life of their own or how they can be poorly designed. And so our interest has always been we do think there should be regulation in this space, but we want to be an actor who makes sure that that regulation is something that's surgical, that's targeted at the serious risks, and is something people can actually comply with. Because something I think the advocates of regulation don't understand as well as they.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  39. “That those are unusual and they warrant an unusually strong response. And so I think it's very important. Again, we need something that everyone can get behind. I think one of the issues with SB 1047, especially the original version of it, was it had a bunch of the structure of RSPs, but it also had a bunch of stuff that was either clunky or that just would have created a bunch of burdens, a bunch of hassle, and might even have missed the target in terms of addressing the risks. You don't really hear about it on Twitter. You just hear about kind of, you know, people are cheering for any regulation. And then the folks who are against makeup these often quite intellectually dishonest arguments about how, you know, it'll make us move away from California built.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  40. “They definitely will do. Right, some people, you know, I think there's a class of people who are against regulation on principle. I understand where that comes from. If you go to Europe and you see something like GDPR, you see some of the other stuff that they've done, some of it's good, but some of it is really unnecessarily burdensome. And I think it's fair to say really has slowed innovation. And so I understand where people are coming from on priors. I understand why people come from start from that position. But again,”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  41. “Our RSP is checked by our long-term benefit trust. So we do everything we can to adhere to our own RSP. But you hear lots of things about various companies saying, oh, they said they said they would give this much compute, and they didn't. They said they would do this thing and they didn't. I don't think it makes sense to, you know, to litigate particular things that companies have done, but I think this broad principle that if there's nothing watching over them, there's nothing watching over us as an industry, there's no guarantee that we'll do the right thing and the stakes are very high. And so I think it's important to have a uniform standard that everyone follows and to make sure that simply that the industry does what a majority of the industry has already said is important and has already said that.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  42. “One, there are still some companies that don't have RSP-like mechanisms, like OpenAI, Google did adopt these mechanisms a couple months after Anthropic did. But there are other companies out there that don't have these mechanisms at all. And so if some companies adopt these mechanisms and others don't, it's really going to create a situation where some of these dangers have the property that it doesn't matter if three out of five of the companies are being safe. If the other two are being unsafe, it creates this negative externality. And I think the lack of uniformity is not fair to those of us who have put a lot of effort into being very thoughtful about these procedures. The second thing is I don't think you can trust these companies to adhere to these voluntary plans in their own, right? I like to think that anthropic will, we do everything we can that we will.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  43. “To really make them a central part of work and anthropic and to make sure that all the thousand people, and it's almost a thousand people now at Anthropic, understand that this is one of the highest priorities of the company, if not the highest priority.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  44. “We ended up making some suggestions to the bill, and then some of those were adopted. And we felt, I think, quite positively about the bill by the end of that. It did still have some downsides. And, you know, of course, of course, it got vetoed. I think at a high level, I think some of the key ideas behind the bill are, you know, I would say similar to ideas behind our RSPs. And I think it's very important that some jurisdiction, whether it's California or the federal government and or other countries and other states passes some regulation like this. And I can talk through why I think that's so important. So I feel good about our RSP. It's not perfect. It needs to be iterated on a lot, but it's been a good forcing function for getting the company to take these risks seriously, to put them into product planning.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  45. “I think it's probably not the right approach. I think the right approach, instead of having something unaligned that you're trying to prevent it from escaping, I think it's better to just design the model the right way or have a loop where you look inside the model and you're able to verify properties and that gives you an opportunity to iterate and actually get it right. I think containing bad models is much worse solution than having good models.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  46. “Kind of break out of any box. And so there we need to think about mechanistic interpretability, about if we're going to have a sandbox, it would need to be a mathematically provable sandbox. That's a whole different world than what we're dealing with with the models today.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  47. “Yeah, we sandboxed during training. So, for example, during training, we didn't expose the model to the internet. I think that's probably a bad idea during training because the model can be changing its policy. It can be changing what it's doing and it's having an effect in the real world. In terms of actually deploying the model, right, it kind of depends on the application. Like, you know, sometimes you want the model to do something in the real world. But of course, you can always put guardrails on the outside, right? You can say, okay, well, you know, this model's not going to move data from my, you know, model's not going to move any files from my computer or my web server to anywhere else. Now, when you talk about sandboxing, again, when we get to ASL4, none of these precautions are going to make sense there, right? Where when you talk about ASL4, you're then the model is being kind of, you know, there's a theoretical worry the model could be smart enough to break it to.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  48. “It's just there are a lot of, like I said, like there are a lot of petty criminals in the world. And, you know, it's like every new technology is like a new way for petty criminals to do something stupid and malicious.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  49. “Yeah, I mean, we follow a lot about things like spam, capcha, you know, mass camp. There's all, you know, every like if one secret I'll tell you, if you've invented a new technology, not necessarily the biggest misuse, but the first misuse you'll see, scams, just petty scams. It's like a thing as people scamming each other. It's this thing as old as time and it's just every time you got to deal with it.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  50. “Yeah, yeah. No, and we've been very aware of that. Look, my view actually is computer use isn't a fundamentally new capability like the CBRN or autonomy capabilities are. It's more like it kind of opens the aperture for the model to use and apply its existing abilities. And so the way we think about it, going back to our RSP, is nothing that this model is doing Inherently increased. To do something at the ASL 3 and ASL 4 level, this may be the thing that kind of unbounds it from doing so. So going forward, certainly this modality of interaction is something we have tested for and that we will continue to test for in our RSP going forward. I think it's probably better to have to learn and explore this capability before the model is super capable.”

    2024-11-11 · Lex Fridman Podcast · #452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · IDENTIFIED FROM THE TRANSCRIPT · source