YouSaid · the spoken record

Leopold Aschenbrenner

lines on the record: 332
first: 2024-06-04
most recent: 2024-06-04
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Somewhat separately, but then you have the sort of second tier of coalitions, which is the sort of Adams for Peace thing, where you go to a bunch of countries, including like the UAE, and you're like, look, we're going to basically, there's a deal similar to the NPT stuff where it's like, you're not allowed to do the crazy military stuff, but we're going to share the civilian applications. We're in fact going to help you and share the benefits and sort of kind of like this new sort of post-superintelligence world order. All right, US checks and balances, right? So obviously Congress is going to have to be involved, right? Appropriate trillions of dollars. I think probably ideally you have Congress needs to kind of like confirm whoever's running this. So you have Congress, you have different factions of the government, you have the courts. I expect the First Amendment to continue being really important. And maybe I think that sounds kind of crazy to people, but I actually think, again, I think these are like institutions that have withheld the tests of time in a really sort of powerful way. Eventually, this is why, honestly, alignment is important is like, you know, the AIs, you program the AIs to follow the Constitution. And it's like,”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, the government actually has decades of experience and actually really cares about the stuff. I mean, it's like they deal, they deal with nukes, they deal with really powerful technology. And this is the stuff that the National Security State cares about. Again, to the go, let's talk about the government checks and balances a little bit. So what are checks and balances in the government world? First of all, I think it's actually quite important that you have some amount of international coalition. And I talked about these sort of two tiers before. Basically, I think the inner tier is sort of modeled on the Quebec agreement, right? This is like Churchill and Roosevelt. They kind of agreed secretly. We're going to like pull our efforts on nukes, but we're not going to use them against each other, and we're not going to use them against anyone else with their consent. And I think basically, look, bring in the UK. They have DeepMind, bring in the kind of like Southeast Asian states who have the chip supply chain, bring in some more of like NATO close democratic allies for talent and industrial resources. And you have this sort of, you know, so you have those checks and balances in terms of more international countries at the table.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“But yeah, why has it held? Because the government has the biggest guns and has never before has a single CEO or a random nonprofit board had the ability to launch nukes. And so again, it's like, you know, what is the track record of the government checks and balances versus the track record of the private company checks and balances? Well, the i lab, you know, like first stress test went really badly. That didn't really work. I mean, even worse in the sort of private company world, so it's both like, it is not just the two, it is like the two private companies and the CCP. And they just like instantly have all the shit. And then they probably won't have good enough internal control. So it's like, it's not just the random CEO, but it's like, you know, rogue employees that can kind of like use these superintelligences to do whatever they want.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Private project. Talking about the government project a little bit and checks and balances. In some sense, I think my argument is a sort of Berkeley argument, which is like American checks and balances have held for over 200 years and through crazy technological revolutions. The US military could kill every civilian in the United States.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“One of them could be a head, so it's not obvious that it'll be multiple. I think it's, again, if there's like a six month lead, maybe there's two or three. I agree. Sorry, but if there's two or three, then what you have is just like a crazy race between these two or three companies. You know, it's like, you know, whatever Demis and Sam, they're just like, I don't want to let the other one win. And they're both developing their nuclear arsenals in the robot. It's just also like, come on, the government is not going to let these people, you know, are they going to let, you know, Dario going to be the one developing the kind of like, you know, super hacking Stuxnet and like deploying against the Chinese data center? The other issue, though, is it won't just, if it's two or three, it won't just be two or three. It'll be two or three and it'll be China and Russia and North Korea because the private in the private lab world, there is no way they'll have security that is good enough.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Sorry, the other important thing is so I talked about the sort of maybe it's like it's a company with all this power and I think it is like I think it is unprecedented because it's like the industrial fertilizer guy cannot overthrow the US government. I think it was quite plausible that like the AI company was super intelligent, can't overthrow the U.S.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Where you need the kind of government project. If you can make it through that, then you kind of go to now we can now, you know, the situation has been stabilized. We don't face this imminent national security threat. It's like, yes, they were kind of WMDs that came along the way, but either we've managed to kind of have a sort of stable offense defense balance, right? Like, I think bioweapons initially are a huge issue, right? An attacker can just create a thousand different synthetic viruses and spread them. And it's going to be really hard for you to kind of make a defense against each. But maybe at some point you figure out the kind of universal defense against every possible virus. And then you're in a stable situation again on the offense defense balance. Or you do the thing, you do with planes where it's there's like, you know, there's certain capabilities that the private sector isn't allowed to have. And you've figured out what's going on, restrict those, and then you can kind of let the sort of civilian uses.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“That is an incredibly scary period, and it is incredibly scary because you're going through just this sort of enormous array of destructive technology and the sort of like enormous amount of like basically military advance. I mean, you would have gone from kind of like bayonets and horses to kind of like tank armies and fighter jets in like a couple years and then from then to like, you know, nukes and ICBMs and stuff, you know, it's just like in a matter of years. And so. It is sort of that speed that creates, I think, basically the way I think about it is there's going to be this initial just incredibly volatile, incredibly dangerous period. And somehow we have to make it through that. And that's going to be incredibly challenging.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“But Look, I mean, what do we do with nukes, right? The way we keep the sort of nukes in check is not like a sort of beefed up Second Amendment where like each state has their own little nuclear arsenal and like, you know, Dario and Sam have their own little nuclear arsenal. No, no, it's like it's institutions, it's constitutions, it's laws, it's courts. So anyway, I don't actually, I'm not sure that this, you know, I'm not sure that the sort of balance of power analogy holds. In fact, the government having the biggest guns was sort of like an enormous civilizational achievement, right? Like Lanfrieden in the sort of Holy Roman Empire, right? You know, if somebody from the town over kind of committed a crime on you, you didn't kind of start a sort of a big battle between the two towns. No, you take it to a court of the Holy Roman Empire and they would decide. And it's a big achievement. Now the thing about the Industrial Fertilizer, I think the key difference is kind of speed and offense defense balance issues, right? So it's like 20th century and 10 years and a few years.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“World And so look, a few things. So, first of all, You talk about the sort of like enormous power that sort of superintelligence will have and that the government will have. Think it's pretty plausible that the alternative world is that like one.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“First of all, okay, so a lot of people right now talk about open source. And I think there's this sort of misconception that AGI development is going to be like, oh, it's going to be some beautiful decentralized thing. And some giddy community of coders who gets to collaborate on it. That's not how it's going to look like, right? $100 billion, trillion dollar cluster. It's not going to be that many people that have it. The algorithms, it's like right now open source is kind of good because people just use the stuff that was published. And so they basically, you know, the algorithms were published. Or as Mistral, they just kind of like leave deep mind and take all the secrets with them and they just kind of replicate it. But that's not going to continue to be the case. And so, you know, the sort of like open source alternative, I mean, also people say stuff like, you know, 10, 26 flops. It'll be in my phone. No, it won't. It's like Moore's law is really slow. I mean, AI chips are getting better, but like, you know, the $100 billion computer will not cost $1,000 within your lifetime or whatever, aside from A. So any, so it's going to be like two or three big players on the prep.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“And some random startup are now more effective organizations because they have a bunch of AGI workers. To win Privatization, that's the way we get to this classic liberal market based system we want for the ESIs. Yep. All right. So a lot of talk about here. I think, yeah, maybe I'll start a bit about like actually looking at what the private world would look like. And I think this is part of where the sort of there's no alternative comes from. And then let's look at what the government project looks like, what checks and balances look like, and so on. All right, private world.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Know There is a history of just dual use technologies, right? And so I think AI in some sense is going to be dual use in the same way. And so there's going to be lots of civilian uses of it. Like nuclear energy, it's like itself, right? There's the government project developed the military angle of it. And then it was like, you know, then the government worked with private companies. There's a sort of like real flourishing of nuclear energy until the environmentalists stopped it. planes, right? Like Boeing, right? Actually, the Manhattan Project wasn't the biggest defense R&D project during World War II. It was the B-29 bomber, right? Because they needed the bomber that had long enough range to reach Japan to destroy their cities. And then Boeing made some Boeing made that Boeing made the B-47, made the B-52, you know, the plane the U.S. military uses today. And then they used that technology later on to build the 707 and sort of the...”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“The power of muddling, imagine you go through like the 20th century in like, you know, a decade. It's just great technological actually.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“You weren't actually like, you hit the most, you know, it's like the sort of destructive potential, the sort of military potential. It's not because of the project. It is because of the technology. And that will unfold regardless.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Obviously, the super did they get their participation because of the project or because of the technology itself. I think people will regret it, but I think it's about the nature of the technology and it's not about the project.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Scope for Government involvement. But again, as we were talking about earlier, I think that would have been sort of this very unique moment in history. And I think basically almost all other moments in history, there would have been the sort of great power competitor.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, I mean, two responses to this. One is, I mean, you still have like Russia, you still have these other countries, you've got to have Russia-proof security, right? It's like you can't just have Russia steal all your stuff. And like, maybe their clusters aren't going to be as big, but like they're still going to be able to make the crazy bioweapons and the mosquito-sized drone, you know, and so on. So, I mean, I think the security component is just actually a pretty large component of the project in the sense of like, I currently do not see another way where we don't kind of like instantly proliferate this to everybody. And so, yeah, so I think it's sort of like you still have to deal with Russia, you know, Iran, North Korea. And, you know, like, you know, Saudi and Iran are going to be trying to get it because they want to screw each other. And, you know, Pakistan and India, because they want to screw each other. There's like this enormous destabilization still. That said, look, I agree with you. Somehow things are shaking out differently and like, you know, AGI would have been in 2005, sort of like unparalleled American hegemony, I think there would have been more.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“It's sort of like we all have literal, like, you know, when we have literal superintelligence on our cluster, right? And it's like you have 100 billion, sorry, you have a billion superintelligence scientists that they can hack everything. They can the Chinese data centers, you know, they're starting to build the robo armies. You really think that'll be like a private company and the government would be like, oh my God, what is going on?”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, look, look on the project, there's sort of descriptive and prescriptive claims or sort of normative, positive claims. I think the main thing I'm trying to say is, you know, look, we're at these SF parties or whatever. And I think people talk about AGI, and they're always just talking about the private AI labs. And I think I just really want to challenge that assumption. It just seems like pretty likely to me, you know, as we've talked about for reasons we've talked about, that look, like the national security state is going to get involved. And I think there's a lot of ways this could look like, right? Is it like nationalization? Is it a public-private partnership? Is it a kind of defense contractor like relationship? Is it a sort of government project that soaks up all the people? And so there's a spectrum there. But I think people are just vastly underrating the chances of this more or less looking like a government project. And look, I mean, look, if”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“And so then it doesn't mean they get the chips, but it just means the machines are deactivated. But look, I mean, imagine if during the Cold War, you know, all of the world's uranium deposits had been in Berlin. And Berlin was already, I mean, almost multiple times and was caused nuclear war. So God help us all.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Okay, well, I mean, sort of in the sort of Taiwan Watcher circles, people often talk about like the late 2020s as like maximum period of risk for Taiwan because sort of like, you know, military modernization cycles and basically extreme fiscal tightening on the military budget in the United States over the last decade or two has meant that sort of we're in this kind of like trough in the late 20s of like basically overall naval capacity. And that's sort of when China is saying they want to be ready. So it's already kind of like it's kind of pitching there's some sort of like you know parallel timeline there yeah look it looks appealing to invade Taiwan I mean maybe not because they you know basically remote cutoff of the chips”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, look, I mean, we talked about the data center one, and then, you know, you probably have to threaten nuclear retaliation to protect that. They might also just blow it up. Ways they can do it without sort of attribution, right? Like you pay Yeah, I mean, this is part of it. We'll talk about this later, but I think we need to be working on the Stuxnet for the Chinese project. In Taiwan, I mean, Taiwan, the Taiwan thing, I talk about AGI by 27 or whatever. Do you know about the terrible 20s?”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“But it has to be the thing we need. The only way this is possible is if we lock it down. If we don't lock it down, we are in this fever struggle. Greatest peril mankind will have ever seen.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“It'll be interesting. I mean, yeah, I think there's a, I mean, there's two ways to buy the galaxies. One is at some point, you know, it's like post superintelligence, you know, there's some crazy.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“So then we offer them a deal. I think there's an incentive to come to the table. I think there's a sort of more stable arrangement you can do. It's sort of an Adams for peace arrangement. And we're like, look, we're going to respect you. We're not going to use superintelligence against you. You can do what you want. You're going to get your slice of the galaxy. We're going to benefit share with you. We're going to have some compute agreement where it's like there's some ratio of compute that you're allowed to have and that's enforced with or like opposing AIs or whatever. We're just not going to do we're just not going to do this kind of like volatile sort of WMD arms race to the death. And sort of it's like a new world order that's US led, that's sort of democratic led, but that respects China, lets them do what they want.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“I think basically we want to get in a position where it is clear that the United States that a sort of coalition of democratic allies will win. It is clear the United States, it is clear to China, that will require having locked down the secrets. That will require having built the 100 gigawatt cluster in the United States and having done the natural gas and doing what's necessary. Then when it is clear that the Democratic Coalition was well ahead, then you go to China and then you offer them a deal. And China will know they're going to win. This is going to be Very scared of what's going to happen. We're going to know we're going to win, but we're also very scared of what's going to happen because we really want to avoid this kind of like breakneck race right at the end and where things could really go awry.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Think I have some uncertainty basically on how easy that decisive advantage will be. I'm pretty confident that if you have superintelligence, you have two years, you have the robots, you're able to get that 30 year lead. Look, then you're in this like Go for one situation. You have your millions or billions of mosquito-sized drones that can just take it out. I think there's even a possibility you can kind of get a decisive advantage earlier. So, you know, there's these stories about these as well, about, you know, like colonization Here's what I think we should do. I really don't want this volatile period. And so I deal with China would be nice. It's going to be really tough if you're in this unstable equilibrium.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“The data centers in Nuclear Robinson. Nuclear deterrence for data centers. I mean, this is Berlin, you know, in the late 50s, early 60s, both Eisenhower and Kennedy multiple times kind of made the threat of full-on nuclear war against the Soviets if they tried to encroach on West Berlin. Sort of insane. It's kind of insane that that went well. But basically, I think that's going to be the only option for the data centers. It's a terrible option. This whole scheme is terrible, right? Being in this neck and neck race sort of at this point is terrible. And, you know, it's also, you know.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Something From Whatever you That gets to the sort of, I mean, that gets to the sort of insane vulnerability of this period post superintelligence, right? Because basically, I think, so you have the intelligence explosion, you have these like vastly superhuman things on your cluster, but you haven't done the industrial explosion yet. You don't have your robots yet. You haven't kind of, you haven't covered the desert in robot factories yet. And that is the sort of crazy moment where say the United States is ahead, the CCP is somewhat behind. There's actually an enormous incentive for first strike, right? Because if they can take out your data center, they know you're about to have just this command, decisive lead. They know if we can just take out this data center, then we can stop it. And they might get desperate. And, you know, so I think basically we're going to get into a position. It's actually, I think it's going to be pretty hard to defend early on. I think we're basically going to be in a position where protecting data centers with like the threat of nuclear retaliation. It's like maybe sounds kind of crazy, though, you know.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Explosion If you're able to go from kind of AGI to superintelligence, if that superintelligence is decisive, like either, you know, like a year after because you developed some crazy WMD or because you have some super hacking ability that lets you kind of completely deactivate the sort of enemy arsenal. That means suppose you're trying to like put in a break we both we're both gonna like cooperate and we're gonna go slower on the cusp of AGI or whatever there is gonna be such an enormous incentive to kind of race ahead to break out We're just gonna do the intelligence explosion if we can get three months ahead we win I think that makes it basically I think any sort of arms control agreement that comes as a situation where there's close very unstable”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“We saw way more nukes than you are. I mean, it's still not ideal for destabilization, but it'd be very different if the arms control agreement had been zero nukes, right? Because if it had been zero nukes, then it's just like one rogue stake makes one nuke. The whole thing is destabilized. Breakout is very easy. Your adversary state starts making nukes. And so basically when you're going to sort of like very low levels of arms or when you're going to kind of in your very dynamic technological situation, arms control is really tough because breakout is easy. There's some other sort of stories about this in sort of like 1920s, 1930s. It's like, you know, all the European states have done disarmament and Germany did this like crash program to build the Luftwaffe. And that was able to massively destabilize things because not that they were the first, they were able to pretty easily build kind of a modern Air Force because the others didn't really have one. And that really destabilized things. And so I think the issue with EGI and superintelligence is the explosiveness of it, right? So if you have an intelligence...”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Competitive perspective. I mean, look, I mean, one thing I want to say is a lot of the stuff I talk about in the series is sort of primarily descriptive, right? And so I think that on the China stuff, it's like, you know, in some ideal world, it's just all merry-go-round and cooperation. But again, it's sort of, I think, I think people wake up to AGI. I think the issue, particularly on sort of like, can we make a deal? Can we make an international treaty? I think it really relates to sort of what is the stability of sort of international arms control universe, right? And so we did very successful arms control on nuclear weapons in the 80s, right? And the reason it was successful is because the sort of new equilibrium was stable, right? So you take go down from whatever 60,000 nukes to 10,000 nukes. When you have 10,000 nukes, basically break out breakout doesn't matter that much, right? Suppose the other guy now try to make 20,000 nukes. Well, it's like, who cares, right? You know, like it's still mutually short destruction. Suppose a rogue state kind of went from zero nukes to one nukes. It's like, who cares?”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Bad people are in charge of the Chinese government Jupiter brain descendants won't know who like the China will be something like distant memory.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“I think basically if they actually really face the full force of state level espionage, I don't really think this is a thing private companies can do. Both, I mean, empirically, right? Like, you know, Microsoft recently had executives' emails hacked by Russian hackers and government emails they posted hacked by government actors, but also it's basically there's just a lot of stuff that only kind of people behind the security cleaners know and only they deal with. And so I think to actually kind of resist the sort of full force of espionage, you're going to need the government. Anyway, so I think basically we could do it by always being ahead of the curve. I think we're just going to always be behind the curve. And I think, you know, maybe unless we get the sort of government project.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Some point we're going to face kind of the full force of the Ministry of State Security. And again, you're talking about smart people underrating espionage and the sort of insane capabilities of states. I mean, this stuff is wild, right? You know, they can get, you know, there's papers about you can find out the location of where you are in a video game map just from sounds, right? Like states can do a lot with electromagnetic emanations. At some point, you have to be working from a SCF. Your cluster needs to be air-gapped and basically be a military base. It's like, you know, you need to have intense kind of security clearance procedures for employees. They have to be like, you know, all this shit is monitored. They basically have security guards. You can't use any kind of other dependencies. It's all got to be intensely vetted and all your hardware has to be intensely vetted.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, anyway, and so all the alpha could be gone. But in fact, their alpha persists, right? And often for many years and decades, and so this doesn't seem to happen. And so I think there's like, you know, I think there's a lot you could go if you went from kind of current startup security. You just got to look through the window and you can look at the slides to kind of good private sector security hedge funds, the way Google treats customer data or whatever. Be good right now. The issue is, you know, basically the CCP will also get more AGI build.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“So, like, right now, you've got to be resistant to kind of like normal economic espionage. They're not, right? I mean, I probably wouldn't be talking about the stuff that the labs were, right? Because they wouldn't want to wake them up more, the CCP, but they're not. This stuff is really trivial for them to do right now. I mean, it's also, anyway, so they're not resistant to that. I think it would be possible for a private company to be resistant to it, right? So both of us have friends in the kind of like quantitative trading world, right? And I think actually those secrets are shaped kind of similarly where it's like, you know, they've said, you know, yeah, if I got on a call for an hour with somebody from a competitive firm, I could most of our alpha would be gone. And that's sort of like, that's the like list of details of really how to make worry about.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“What do they do? So, again, I think basically it's, you know, I think people, there's kind of like two reactions there, which is like, it's, you know, we're already secure, you know, not. And there's, you know, fatalism, it's impossible. I think the thing you need to do is you kind of got to stay ahead of the curve of basically how EGI pill is the CCP.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Giving away sort of secret information was death. And so to graduate from the Soviets Fyc Academy, you had to condemn a countryman to death. States do this stuff.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“There's a good book along the lines of the Gulag or Bellovo Inside the Aquarium, which is by a Soviet GRU defector. GRU was like military intelligence. Ilya recommended this book to me. I think reading that, I was just kind of like shocked that I would attend sort of state-level espionages. The whole book was about they go to these European countries and they try to get all the technology and recruit all these people to get the technology. I mean, yeah, maybe one anecdote, you know, so when, so the spot, this eventual defector, so he's being trained, he goes to the kind of GRU spy academy. And so then to graduate from the spy academy, sort of before you're sent abroad, you kind of had to pass a test to show that you can do this. And the test was, you know, you had to, in Moscow, recruit a Soviet scientist and recruit them to give you information, sort of like you would do in the foreign country. But of course, for whomever you recruited, the penalty for”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, yeah. Intelligence agencies have just stockpiles of zero days. When things get really hot, I don't know. I'm able to send special forces to go to the data center or something that's, you know, or I mean, China does this. They threaten people's families, right? And they're like, look, if you don't cooperate, if you don't give us the intel.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Know why do people underrate it? I mean, I think there's a sort of way in which being kind of in the trenches actually kind of, I think, gives you a less clear picture of the trend lines. You actually have to zoom out that much, only like a few years, right? But you're in the trenches, you're like trying to get the next model to work. There's always something that's hard. For example, you might underrate algorithmic progress because you're like, ah, things are hard right now or data wall or whatever. But you zoom out just a few years and you actually try to count up how much algorithmic progress made in the last few years. And it's enormous. But I also just don't think people think about this stuff. I think smart people really underrate espionage, right? And I think part of the security issue is I think people don't realize how intense state level espionage can be, right? Like, you know, the Israeli company had software that could just zero click hack any iPhone, right? They just put in your number and then it's just like straight download of everything, right? Like the United States infiltrated and air gapped atomic.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“We're talking about national security. I made this mistake with COVID, right? So February of 2020 and I thought just it was going to sweep the world and all the hospitals would collapse and it would be crazy. And then, you know, and then it'd be over. And a lot of people thought this kind of the beginning of COVID. They shut down their offices a month or whatever. I think the thing I just really didn't price in was societal reaction, right? And within weeks, Congress spent over 10% of GDP on like COVID measures. The entire country was shut down. That was crazy. And so I don't know. I didn't price it in with COVID sufficiently.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“You know, my girlfriend sometimes complains that I, you know, when I'm off doing work or whatever, she's like, I'm not spending time with her. She's threatens to replace me with, you know, GBD6 or whatever. And I'm like, you know, GP6 will also be too busy doing.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“This is the quiet period. You want to go on vacation, you know, you want to, yeah, you want to have, you know, maybe like now is the last time you can have some kids.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I think in some sense, one of the most important inputs to whether we will kind of destroy ourselves or whether we will get through this just incredibly crazy period is whether we have that buffer.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Of maybe they'd get a decisive advantage. They were building clusters like Crazy, they're willing to throw all caution to the wind. We have to keep up. There's some crazy new WMDs popping up. And then we're going to be in the situation where it's like, you know, crazy new military technology, crazy new WMDs, like deterrence and mutually disturbed instruction keeps changing every few weeks. And it's like completely unstable, volatile situation. That is incredibly dangerous. So I think both from just the technologies are dangerous from the alignment point of view, I think it might be really important during the intelligence explosion to have this sort of six months.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“Of decades worth of technological progress. And might, you know, again, it's like Goal 4 1, right? 20, 30 years of technological lead totally decisive. I think it really matters. The other reason it really matters is Suppose they steal the weight. Suppose they steal the algorithms and they're close on our tails. Suppose we still pull out a head, right? We just kind of we're a little bit faster. We're three months ahead. I think the sort of world in which we're really neck and neck, you know, we only have a three month lead are incredibly dangerous, right? And we're in this feverish struggle where like if they get ahead, they get to dominate.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I actually think if we lock down the labs, we have much better scientists. We're way ahead. It would be two years. I think whether you, I think, yeah, I think even six months a year would make a huge difference. And this gets back to the sort of intelligence explosion dynamics. Like a year might be the difference between a system that sort of like human level and a system that is vastly superhuman. Might be like five ooms. Even on the current pace, right? We went from, I think, on the math benchmark recently, right? Like, you know, three years ago on the math benchmark, this is sort of really difficult high school competition math problems. We were at a few percent, couldn't solve anything. Now it's solved. And now it's sort of at the normal pace of AI progress. You didn't have sort of a billion superintelligent resources. So like a year is a huge difference. And then particularly after superintelligence, right, once this is applied to sort of lots of elements of R&D, once you get the sort of like industrial explosion with robots and so on, I think a year, a couple years might be kind of.”
2024-06-04 · Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history · IDENTIFIED FROM THE TRANSCRIPT · source