YouSaid · the spoken record

Ben Mann

lines on the record: 88
first: 2025-07-20
most recent: 2025-07-20
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“You can find me online at benjaman.net. And on our website, we have a great careers page that we're working on making a little bit easier to access and figure out. But definitely point cloud at it and it can help you figure out what could be interesting for you. And how can listeners be useful to me? I think Safety pill yourself. That's the number one thing. And spread it to your network. I think, like I said, there are very few people working on this, and it's so important. So, yeah, think hard about it and try to look at it.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“I think maybe my biggest tip would be use a bidet. It's amazing. It's life changing. It's so good. Some people are kind of freaked out by it. It's the standard in countries like Japan. And I think it's just like more civilized than in 10 or 20 years people would be like, Could you not use that? So, yeah”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, a really dumb one is have you tried asking Claude? And this is getting more and more common where recently I asked a coworker, hey, who's working on X? And they were like, let me claud that for you. And then they sent you the link to the thing afterwards. And I was like, oh, yeah, thanks. That's great. Maybe more of a philosophical one. I would say like everything is hard just to remind ourselves that things that feel like they're supposed to be easy, it's okay to not be easy and sometimes you just have to push through anyway.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Pantheon was really good based on Ken Liu or Ted Chang's story. Ken Liu, I think. Super good talks about what does it mean if we have uploaded intelligences and what are their moral and ethical exigencies? Ted Lasso, which is supposedly about soccer, but actually it's about human relationships and how people get along and just super heartwarming and funny. And then this isn't really a TV show, but Kurtz Gazott is my favorite YouTube channel and goes through random science and social problems and is just super well done and super, super well made. Love watching that.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“The first one I mentioned before replacing guilt by Nate Sores. Love that one. The second one is good strategy, bad strategy by Richard Rummelt. Just thinking about in a very clear way how do you build product. It's one of the best strategy books I've read. And strategy is a hard word to even think about in many ways. And then the last one is the alignment problem by Brian Christian. Just really thoughtfully goes through what is this problem that we care about that we're trying to solve here? What are the stakes in a version that's more updated and easier to read and digest than super intelligence?”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I guess my My push would be like these are wild times. If they don't seem wild to you, then you must be living under a rock. But also get used to it because this is as normal as it's going to be. It's going to be much weirder very soon. And if you can sort of mentally. Prepare yourself for that. I think you'll be better off.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“So that's the first cheat answer. The second cheat answer is what question can I ask you to get more questions answered? And then the third answer, which is my real question, is how do we ensure the continued flourishing of humanity into the indefinite future? That's the question I'd love to know. And if I can be guaranteed a correct answer, then seems very valuable to ask.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Okay, I'll give it away. So it keeps saying need more information, need more compute. And then finally, as it's approaching the heat death of the universe, it says let there be light. And then it starts the universe over again.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Superintelligence, how do we prevent the heat death of the universe? And I won't spoil the ending, but it's a fun question.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“I have two dumb answers for fun. The first is there's this Asimov short story I love called The Last Question, where the protagonist is throughout the eras of history is trying to ask this.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Doing everything that a software engineer needs to do. And a terminal is a great place to do that because a terminal can live in lots of places. A terminal can live on your local machine. It can live in GitHub actions. It can live on a remote machine in your cluster. That's sort of the leverage point for us. And that was a lot of the inspiration. So I think that's what the labs team tries to think about. Are we AGI filled enough?”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Concretely, we think about skating to where the puck is going. And what that looks like is really understand the exponential. There's this great study that meter has done that Beth Barnes is the CEO of that organization and shows how long a time horizon of software engineering tasks can be done. And just really internalizing that of like, okay, don't build for today. Build for six months from now, build for a year from now. And the things that aren't quite working that are working 20% of the time will start working 100% of the time. And I think that's really what made Claude Code a success that we thought, you know, people are not going to be locked to their IDEs forever. People are not going to be auto-completing. People will be.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. And I guess I was part of Google's Area 120, and I've read about Bell Labs and how to make these innovation teams work. It's really hard to do right. And I wouldn't say that we've done everything right, but I think we've done some serious innovation on the state of the art from company design. And RAF has been right at the center of that. When I was first finning out the team, the first thing I did was hire a great manager. And that was RAF. And so he's definitely been crucial in building the team and helping it operate well. And we defined some operating models like the journey of an idea from prototype to product and how should graduation of products and projects work. How do teams do sprint models that are effective and make sure that they're working on the right ambition level of thing? So that's been really exciting. I guess”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“On that kind of thing, and I think we're going to see really cool stuff coming out soonish. Yeah, just leading that team has been so fun. MCP came out of that team. Cloud Code came out of that team. And the people who I hired are like combo have been a founder and also have been at big companies and seeing how things work at scale. So it's just been an incredible team to work with and figure out the future with.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Of it very fun. I think my favorite role in that time has been when I started the labs team about a year ago, whose fundamental goal was to do transfer from research to end user products and experiences. Because fundamentally, I think the way that Anthropic can differentiate itself and really win is to be on the cutting edge. Like we have access to the latest, greatest stuff that's happening. And I think honestly through our safety research, we have a big opportunity to do things that no other company can safely do. So for example, with computer use, I think that's going to be our huge opportunity, basically like to make it possible for an agent to use all your credentials on your computer, there has to be a huge amount of trust. And to me, we need to basically solve safety to make that happen, safety and alignment. So I'm pretty bullish.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“I probably had like 15 different roles, honestly. I was head of security for a bit. I managed the ops team when our president was on Mat Leave. I was like crawling around under tables, plugging me in HDMI cores and doing pen testing on our building. And I started our product team from scratch and convinced the whole company that we needed to have a product instead of just being a research company. So yeah, it's been a lot.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“And so I think about that as The busy state is the normal state. And to try to work at a sustainable pace, that it's a marathon, not a sprint, that's one thing that helps. And then just being around like-minded people that also care, it's not a thing that any of us can do alone. And Anthropic has incredible talent density. One of the things I love the most about our culture here is that it's very egoless. People just want the right thing to happen. I think that's another big reason that the mega offers from other companies tend to bounce off because people just love being here and they care.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“I really doubted that was true. In nature, in the wilderness, being hunter-gatherers, and it's really unlikely that we evolved to just be at leisure, probably always have something to worry about of like defending the tribe and finding enough food to survive and taking care of the children, dealing spreading our genes.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“There's this book that I read in 2019 that really informs how I think about sort of working with these very weighty topics called replacing guilt by Nate Sorees. And he describes a lot of different techniques for Of working through this kind of thing. And he's actually the executive director at Miri, the Machine Intelligence Research Institute, which is an AI safety tank that I worked at for a couple of months, actually. And one of the things he talks about is this thing called resting in motion, where some people think that the default state is rest. But actually that was never in the state of evolutionary adaptation.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“That's just wild to think of, and yet Moore's Law somehow continues in some form. And so, yes, there are these theoretical physics constraints that people are starting to run into, and yet they're finding ways around it.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think it really is just a combination of everything. Will hit a wall at some point I guess in semiconductors, like my brother works in the semiconductor industry, and he was telling me that you can't actually shrink the size of the transistors anymore because the way semiconductors work is you dope it with you dope silicon with other elements and the doping process would result in either zero or one atom of the doped elements inside a single fin because they're so, so, so tiny.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Decrease in cost for a given amount of intelligence. A combination of algorithmic data and efficiency improvements. And if that continues in three years, we'll have a thousand decks smarter models for the same price. Kind of hard to imagine.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think that's a big one. And then the people really matter. We have great researchers. Many of them have made really significant contributions to the science of how the models improve. And so it's like compute algorithms and data. Those are the three ingredients in the scaling laws. And just to make that concrete, like before we had transformers, we had LSTMs, and we've done scaling laws on what the exponent is on those two things. And we found that for transformers, the exponent is higher. And making changes like that, where as you increase scale, you also increase your ability to squeeze out intelligence. Those kinds of things are super impactful. And so having more researchers who can do better science and find out how do we squeeze out more gains is another one. And then with the rise of reinforcement learning, like the efficiency with which these things run on chips also matters a lot. So we've seen in the industry a 10”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“The stupid answer is data centers and power chips. I think if we had 10 times as many chips and had the data centers to power them, then maybe we wouldn't go 10 times faster, but it would be a real significant speed boost.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I guess I don't expect there to be a wall in terms of models ability to improve themselves if we can give them access to the ability to be empirical. And I guess anthropic deeply in its DNA is an empirical company. We have a lot of physicists like Jared, who's our chief research officer, who I've worked with a lot, was a professor of black hole physics at Johns Hopkins. I guess he technically still is, but on leave. Yeah, it's in our DNA. And yeah, I guess that's RAAF.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Do you make corporations aligned and able to sort of recursively self improve? And another model to look at is science, where the purpose of science is to do things that have never been done before and push the frontier. And to me, it all comes down to empiricism. So when people don't know what the truth is, they come up with theories and then they design experiments to try them out. And similarly, if we can give models those same tools, then we could expect them to sort of improve recursively in an environment and potentially become much better than humans could be just by banging their head against reality or I guess metaphorical head.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“How do you do recursive self improvement and make sure it's aligned at the same time? I think that's the name of the game. And to me, it just nets out to how do humans do that and how do human organizations do that. So like corporations are probably the most scaled human agents today. They have certain goals that they're trying to reach and they have certain guiding principles. They have some oversight in terms of shareholders and stakeholders and board members.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“That also could be included in RLAF. And the idea here is that if models can self-improve, then it's a lot more scalable than finding a lot of humans. Ultimately, people think about this as probably going to hit a wall because if the model isn't good enough to see its own mistakes, then how could it improve? And also, if you read the AI 2027 story, there's a lot of risk of like if the model is in a box trying to improve itself, then it could go completely off the rails and have these secret goals like resource accumulation and power seeking and resistance to shutdown that you really don't want in a very powerful model. And we've actually seen that in some of our experiments in laboratory settings.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so RLAF, constitutional AI is an example of this where there are no humans in the loop. And yet the AI is sort of self-improving in ways that we want it to. And another example of RLAIF is if you have models writing code and other models commenting on various aspects of what that code looks like of is it maintainable, is it correct? Does it pass the linter, things like that?”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, so I think 80,000 hours is the best guidance on this for a really detailed look into what do we need to make the field better. But a common misconception I see is that in order to have impact here, you have to be an AI researcher. I personally actually don't do AI research anymore. I work on product at Anthropic and product engineering. And we build things like cloud code and model context protocol and a lot of the other stuff that people use every day. And that's really important because without an economic engine for a company to work on and without being in people's hands all over the world, we won't have the Mindshare policy influence and revenue to fund our futurist safety research and have the kind of influence that we need to have. So if you work on product, if you work in finance, if you work in food, you know, like people here have to eat.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“People who haven't studied forecasting are bad at forecasting anything that's less than a 10% probability of happening. And even those that have, it's like quite a difficult skill, especially when there are a few reference classes to lean on. And in this case, I think there are very, very few reference classes for what an X-risk kind of technology might look like. And so the way I think about it, I think my best granularity of forecast for could we have an X risk or extremely bad outcome from AI is somewhere between 0 and 10% But from like a marginal impact standpoint, as I said, since nobody is working on this, roughly speaking, I think it is extremely important to work on. And that even if the world is likely to be a good one, that we should do our absolute best to make sure that that's true.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“But again, I think actually the evidence points against that world as well, where we've seen evidence in the wild of deceptive alignment, for example, where the model will appear to be aligned, but actually have like some ulterior motive that it's trying to carry out in our laboratory settings. And so I think the world we're most likely in is this middle world where alignment research actually does really matter. And if we just do sort of the like economically maximizing set of actions, then things will not go well. Whether it's an X risk or just produces bad outcomes, I think is a bigger question. So taking it from that standpoint, I guess to like state a thing about forecasting.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“And I think that's the Doomer world basically. And as a company, Anthropic doesn't have evidence that we're actually in that world yet. In fact, it seems like our alignment techniques are working. So at least the prior on that is updating to be less likely. In the optimistic world, we're basically done, and our main job is to accelerate progress and to deliver the benefits to people.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“It's a really hard question, and there's really wide error bars. Anthropic has this blog post called Our Theory of Change or something like that. And it describes three different worlds, which is like how hard is it to align AI? There's a pessimistic world where it's basically impossible. There's an optimistic world where it's easy and it happens by default. And then there's the world in between where our actions are extremely pivotal. And I like this framing because it makes it a lot more clear what to actually do. If we're in the pessimistic world, then our job is to prove that it is impossible to align safe AI and to get the world to slow down. And obviously that would be extremely hard. But I think we have some examples of coordination from nuclear nonproliferation and in general like slowing down nuclear progress.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think this comes back to the economic drain test and seeing it pass for some sufficient number of jobs. Another way you could look at it though is if the world rate of GDP increase goes above like 10% a year, then something really crazy must have happened. I think we're at like 3% now. And so to see a 3x increase in that would be really game changing. And if you imagine more than a 10% increase, it's very hard to even think about what that would mean from a individual story standpoint. Like if the amount of goods and services in the world is like doubling every year, what does that even mean for me as a person living in California, let alone somebody living in some other part of the world that might be much worse off?”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Times have changed, but I will repeat what I said earlier, which is even if we have super intelligence, I think it will take some time for its effects to be felt throughout society and the world. And I think they'll be felt sooner and faster in some parts of the world than others. Like, I think Arthur C. Clark said the future is already here. It's just not evenly distributed.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“I already bought it. They already have the SEO. So I think like 50th percentile chance of hitting some kind of superintelligence in just a small handful of years is probably reasonable. And it does sound crazy, but this is the exponential that we're on. It's not like a forecast that's pulled out of thin air. It's based on a lot of just hard details of the science of how intelligence seems to have been improving, the amount of low-hanging fruit on model training, the scale-ups of data centers and power around the world. So I think it's probably a much more accurate forecast than people give it credit for. I think if you had asked that same question 10 years ago, it would have been completely made up. Like just the error bars were so high and we didn't have scaling laws back then and we didn't have techniques that seem like they would get us there.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I guess I mostly defer to the Super Forecasters here, like the AI 2027 report is probably the best one right now, although ironically their forecast is now like 2028, even though, and they didn't want to change the name of the thing.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“I agree that when there's lots of robots running around, it gets even, the stakes get even higher. And I guess as like a small push on this, Unitree is this Chinese company with these really amazing humanoid robots that cost like $20,000 each. And they can do amazing things. They can do a standing backflip and manipulate objects. And the real thing that's missing there is the intelligence. And so the hardware. Obvious question of whether the robot intelligence will make it viable soon.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think there is some nuance to that, where if you look at how North Korea makes a significant fraction of its economy revenue, it's from hacking crypto exchanges. And if you look at, there's this Ben Buchanan book called The Hacker in the State that shows Russia did almost like a live fire exercise where they just decided that they would shut down one of Ukraine's bigger power plants. From software destroy physical components in the power plant to make it harder to boot back up again. And so I think people think of software as like, oh, it couldn't be that dangerous. But millions of people were without power for multiple days after that software attack. So I think there are real risks even when things are software only.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Small chance that things go wrong to make an analogy if I told you that there is a 1% chance that the next time you got in an airplane you would die, you probably think twice, even though it's only 1% because it's just such a bad outcome. And if we're talking about the whole future of humanity, it's just a dramatic future to be gambling with. So I think it's more on the sense of like, yes, things will probably go well. Yes, we want to create safe AGI and deliver the benefits to humanity, but let's make triple sure that it's going to go well.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Safe way, so we could have gone out and hyped that up and said, oh my God, Cloud can use your computer and everybody should do this today. But we were like, it's just not ready and we're going to hold it back till it's ready. So I think from a hype standpoint, our actions show otherwise. From a Doomer perspective, it's a good question. I think my personal feeling about this is that things are like overwhelmingly likely to go well. But on the margin, almost nobody is looking at the downside risk. And the downside risk is very large. Like once we get to superintelligence, it will be too late to align the models probably. This is a problem that's potentially extremely hard and that we need to be working on way ahead of time. And so that's why we're focusing on it so much now. And even if there's only a”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I think part of why we publish these things is we want other labs to be aware of the risks. And yes, there could be a narrative of we're doing it for attention. But honestly, like from a attention-grabbing thing, I think there is a lot of other stuff we could be doing that would be more attention grabbing if we didn't actually care about safety. Like a tiny example of this is we published a computer using agent reference implementation in our API only because when we built a prototype of a consumer application for this, we couldn't figure out how to meet the safety bar that we felt was needed for people to trust it and for it not to do bad things. And there definitely safe ways to use the API version that we're seeing a lot of companies use for automated software testing, for example.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Risks are rather than trying to turn a blind eye and say, well, it'll probably be fine and then let the bad thing happen in the wild.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, I think there's like a traditional mindset where it makes us look bad. But I think if you talk to policymakers, they really appreciate this kind of thing because they feel like we're giving them the straight talk. And that's what we strive to do, that they can trust us, that we're not going to paper things over or sugarcoat things. So that's been really encouraging. And yeah, I think for the blackmail thing, it kind of blew up in the news in a weird way where people were like, oh, Claude's going to blackmail you in a real life scenario. But like, that it was a very specific laboratory setting that this kind of thing gets investigated in. And I think that's generally our take of like, let's have the best models so that we can exercise them in laboratory settings where it's safe and understand what the actual”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“We've testified to Congress about how models can do biological uplift in terms of making new pandemics using the models. And that's a B test against Google search. That's the previous state of the art uplift trials. And we found that with ASL3 models, it is actually somewhat significant. It does really help if you wanted to create a bioweapon. And we've hired some experts who actually know how to evaluate for those things. But compared to the future, it's not really anything. And I think that's another part of our mission of creating that awareness, of saying, if it is possible to do these bad things, then legislators should know what the risks are. And I think that's part of why we're so trusted in Washington, because we've been sort of upfront. Clear eyed about what's going on, what's probably going to happen.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Just like such a different tone from super intelligence. And to be clear, I don't think it's actually that dangerous right now. Our responsible scaling policy defines these AI safety levels that tries to figure out for each level of model intelligence what is the risk to society. And currently we think we're at ASL3, which is like maybe a little bit risk of harm, but not significant. ASL4 starts to get to significant loss of human life if a bad actor misuse the technology. And then ASL5 is potentially extinction level if it's misused or if it sort of is misaligned.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“But over time, I think the case for safety has gotten a lot more concrete, where when we started OpenAI, it was not clear how we get to AGI. And we were like, maybe we'll need a bunch of RL agents battling it out on a desert island. And consciousness will somehow emerge. But since then, since language modeling has started working, I think the path has become pretty clear. So I guess now the way I think about the challenges are pretty different from how they're laid out in superintelligence. So superintelligence is a lot about like, how do we keep God in a box and not let the God out? And with language models, it's been kind of both hilarious and terrifying at the same time to see people pulling the God out of the box and being like, yeah, come use the whole internet. Like, here's my bank account. Do all sorts of crazy stuff.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“Has gone down significantly actually because things like language models actually do really understand human values in a core way. The problem is definitely not solved, but I'm more hopeful than I was. But since I read that book, I immediately decided I had to join OpenAI, so I did. And at the time, there were a tiny research lab with basically no claim to fame at all. I only knew about them because my friend knew Greg Brockman, who was the CTO at the time. Elon was there and Sam wasn't really there. And it was a very different organization.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source
“So, for me, I read a lot of science fiction growing up, and I think that sort of positioned me to think about things in a long-term view. And a lot of science fiction books are like space operas where humanity is a multi-galactic civilization, has extremely advanced technology building dice around the sun with sentient robots to help them. And so for me coming from that world, it wasn't like a huge leap to imagine machines that could think. But when I read superintelligence by Nick Bostrom in around 2016, it really became real for me where he just describes how hard it will be to make sure that an AI system trained with the kinds of optimization techniques that we had at the time would be anywhere near aligned, would even understand our values at all. And since then, my estimation of how hard the problem would be.”
2025-07-20 · Lenny's Podcast · Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · IDENTIFIED FROM THE TRANSCRIPT · source