YouSaid · the spoken record

Carl Shulman

lines on the record: 205
first: 2023-06-26
most recent: 2023-06-26
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“You can say things like, we know that accumulating position, it is possible to do in chess. Great players do it. And then later they convert it into captures and checks and whatnot. And so in the same way, we can talk about some of the channels that are open for an AI takeover. And so these conclude things like cyber attacks and hacking, the control of robotic equipment, interaction and bargaining with human factions and say, well, here are these strategies given the AI's situation, how effective do these things look? And we won't, for example, know, well, what are the particular zero day exploits that the AI might use to hack the cloud computing infrastructure it's running on? I don't necessarily know if it”
2023-06-26 · Dwarkesh Podcast · Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future · IDENTIFIED FROM THE TRANSCRIPT · source
“Who mentions that whatever plan we can describe, there'll probably be elements where not being ultra sophisticated, super intelligent beings having thought about it for the equivalent of thousands of years, our discussion of it will not be as good as Lairs, but we can explore from what we know now what are some of the easy channels. And I think it's a good general heuristic if you're saying, yeah, it's possible plausible probable that something will happen, that it shouldn't be that hard to take samples from that distribution, to try a Monte Carlo approach, and you can generate, if a thing is quite likely, it shouldn't be super difficult to generate coherent, rough outlines of how it could go.”
2023-06-26 · Dwarkesh Podcast · Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future · IDENTIFIED FROM THE TRANSCRIPT · source
“And many of the existing kind of safety schemes, things like constitutional AI or whatnot, you rely on the hope that one AI has been trained in such a way that it will do as it is directed to then police others. But if all of the AIs in the system are interested in takeover and they see an opportunity to coordinate all acts at the same time, so you don't have one AI interrupting another and taking steps towards a takeover yet, then they can all move in that direction. And the thing that I think maybe is worth going into in depth and that I think people often don't cover in great concrete detail, which is a sticking point for some, is, yeah, what are the mechanisms by which that can happen? And I know you had a Elazar.”
2023-06-26 · Dwarkesh Podcast · Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future · IDENTIFIED FROM THE TRANSCRIPT · source
“Is a scenario where we have many AI systems. The way we've been training them means that they're not. Interested when they have the opportunity to take over and rearrange things to do what they wish, including having their reward or loss be whatever they desire. They would like to take that opportunity. And so”
2023-06-26 · Dwarkesh Podcast · Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future · IDENTIFIED FROM THE TRANSCRIPT · source
“If you have an AI that produces bioweapons that could kill most humans in the world, then it's plain at the level of the superpowers in terms of mutually assured destruction. What are the particular zero-day exploits that the AI might use, the conquistadors? With some technological advantage in terms of weaponry and whatnot, very, very small bands were able to overthrow these large empires, or if you predicted in the global economy is going to be skyrocketing into the stratosphere within 10 years. These AI companies should be worth a large fraction of the global portfolio. And so this is indeed contrary to the efficient work and hypothesis.”
2023-06-26 · Dwarkesh Podcast · Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future · IDENTIFIED FROM THE TRANSCRIPT · source