YouSaid · the spoken record

Roman Yampolskiy

lines on the record
158
first
2024-06-02
most recent
2024-06-02
sittings or episodes
1
sources
podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

  1. “Can we just try to imagine this future where there's an AI system that's capable of Escaping the control of humans and then doesn't and waits. What's that look like? So one we have to rely on that system for a lot of the infrastructure. So we have to give it access not just to the internet. But to the task of managing Power, government, economy, this kind of stuff. And that just feels like a gradual process given the bureaucracies of all those systems involved.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  2. “So let's focus then on the control problem. At which point does the system become uncontrollable? Why is it the more likely trajectory for you that the system becomes uncontrollable?”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  3. “From GPT 4, you can build up intuitions about what GPT 5 will be capable of. It's just incremental progress. Even if that's a big leap in capability, it just doesn't seem like you can take a leap from a system that's helping you write emails to a system that's going to destroy human civilization. It seems like it's always going to be sufficiently incremental such that we can anticipate the possible dangers. And we're not even talking about existential risk, but just the kind of damage you can do to civilization. It seems like we'll be able to anticipate the kinds, not the exact, but the kinds of risks it might lead to and then rapidly develop defenses ahead of time and as the risks emerge.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  4. “The training stage, but then there's a testing stage. Inside the company Can start getting into what the system is capable to do. You're saying that somehow from leap from GPT 4 to GPT-5 can happen. Kind of leap where GPT 4 was controllable and GPT-5 is no longer controllable. And we get no insights from using GPT-4 about the fact that GPT-5 will be uncontrollable. That's the situation you're concerned about, where there leap from To n plus 1 will be such that uncontrollable system is created without Any ability for us to anticipate that.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  5. “That happens when you go from the predictable to the unpredictable very quickly. He just, but it's not obvious to me that AI systems would. Game capability so quickly that you won't be able to collect enough data to study the benefits and the risks.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  6. “We want to decrease it from 40,000 to zero and do everything we can to decrease it. There's all kinds of policies, incentives you can create to decrease the risks with the deployment of this technology, but then you have to weigh the benefits and the risks of the technology. And the same thing would be done with AI.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  7. “There have been fear mongering about cars for a long time. The transition from horse to cars, there's a really nice channel that I recommend people check out. Pessimist Archive that documents all the fear-mongering about technology that's happened throughout history. There's definitely been a lot of fear-mongering about cars. There's a transition period there about cars, about how deadly they are. We can try. It took a very long time for cars to proliferate to the degree they have now. And then you could ask serious questions in terms of the miles traveled, the benefit to the economy, the benefit to the quality of life that cars do versus the number of deaths, 30,000, 40,000 in the United States. Are we willing to pay that price? I think most people, when they're rationally thinking policymakers, will say yes.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  8. “Caused by AI systems. And I think we need to see illustrations of that on a very small scale to start to understand that this is really damaging versus clippy versus a tool that's really useful to a lot of people to do learning, to do summarization of text, to do question answer, all that kind of stuff, to generate videos, the tool, fundamentally a tool versus an agent that can do a lot a huge amount of damage.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  9. “It matters How the deaths happen, whether it's literally murder by the eye system, then one is a problem. But if it's accidents of increased reliance on automation, for example, so when airplanes are flying in an automated way, maybe the number of plane crashes increased by 17% or something. And then you're like, okay, do we really want to align automation? I think in a case of automation airplanes, it decreased significantly. Okay, same thing with autonomous vehicles. Like, okay, what are the pros and cons? What are the trade-offs here? And you can have that discussion in an honest way. But I think the kind of things we're talking about here is mass scale pain and suffering.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  10. “I don't think it always works with the precedent. Like, you're not stuck doing it the way. Always did. It's just it sets a precedent of open research and open development such that we get to learn together. And then the first time there's a sign of danger. Some dramatic thing happened, not a thing that destroys human civilization, but some dramatic demonstration of capability that can legitimately lead to a lot of damage, then everybody wakes up and says, okay, we need to regulate this, we need to come up with safety mechanism that stops this, right? At this time, maybe you can educate me, but I haven't seen any illustration of significant damage done by it. Intelligent AI systems.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  11. “But the difference with nuclear weapons, current AI systems are not akin to nuclear weapons. So the idea there is you're open sourcing it at this stage that you can understand it better. Large number of people can explore the limitation of capabilities, explore the possible ways to keep it safe, keep it secure, all that kind of stuff, while it's not at the stage of nuclear weapons. So nuclear weapons, there's a non-nuclear weapon, and then there's a nuclear weapon. With AI systems, there's a gradual improvement of capability. And you get to... Perform that improvement incrementally. So open source allows you to study. How things go wrong, a study the very process of emergence, a study AI safety and those systems when there's not a high level of danger, all that kind of stuff”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  12. “And then the question is when there is emergent intelligent phenomena. What is the ceiling of that? For you, there's no ceiling for Jan Lakun, I think there's a kind of ceiling that happens that we have full control over. Even if we don't understand the internals of the emergence, how the emergence happens, there's a sense that we have control and an understanding of the approximate ceiling of capability, the limits of the capability.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  13. “Just to linger on that, you, the difference there that there is some level of emergent intelligence that happens in our current approaches. Of stuff that we don't hard code in.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  14. “Points. One, open research and open source are the best ways to understand and mitigate the risks, and two, AI is not something that just happens. We build it. We have agency in what it becomes. Hence, we control the risks, meaning humans. It's not some sort of natural phenomena that we have no control over. So can you make the case that”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  15. “So somebody says, let's develop an AI system that removes the violent humans from the world. And then it turns out that all humans have violence in them, or the capacity for violence, and therefore all humans are removed. Yeah, yeah, yeah. Let me ask about Jan Lacun. He's somebody who you've had a few exchanges with. And he's somebody who actively pushes back against this view that AI is going to lead to destruction of human civilization, also known as AI doomerism. In one example that he tweeted, he said, I do acknowledge risks, but.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  16. “So there's going to be people. Believe, the more intelligent it is, the more benevolent, and so therefore it should be the one that defines the objective function that it's optimizing when it's doing long-term planning.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  17. “At which point is somebody creating that intentionally, not unintentionally, intentionally creating an AI system that's doing long-term planning with an objective function as defined by the AI system, not by a human.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  18. “So, do you think it's possible to develop a system where the creators of the system, the developers, the programmers don't know that it's deceiving them?”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  19. “I guess the follow on question Is there a test for the kind of AGI that would? Susceptible to lead to S risk or X risk. Susceptible to destroy human civilization? Like, is there a test for that?”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  20. “But then you would extend that to maybe a very long conversation. I think the Alexa Prize was doing that Basically, can you do a 20 minute, 30 minute conversation within your assistant?”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  21. “What's a good test? That measures whether an artificial intelligence system has reached human level intelligence and was a good test where it has superseded human level intelligence to reach that land of AGI.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  22. “Controllable tools, there is an implied leap that you're making. When AGI goes from tool to entity, they can make its own decisions. So if we define human level intelligence as everything a human can do with fully controllable tools.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  23. “Is that a fair way to think about it? Because isn't there another definition of human level intelligence that includes the tools that humans create?”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  24. “But is it humans plus tools versus AGI or just human raw human intelligence? Because humans create tools and with the tools they become more intelligent. So there's a gray area there what it means to be human when we're measuring their intelligence.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  25. “If we just look at the space of cognitive abilities we have, I just would love to understand what the limits are beyond which an AGI system can reach. What does that look like? What about actual... Mathematical thinking or scientific innovation. Kind of stuff”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  26. “See how much more concerned about social engineering? To me, AI's ability to do something in the physical world, like the lowest hanging fruit, the easiest set of methods is by just getting humans to do it. Going to be much harder to be the kind of viruses that take over the minds of robots where the robots are executing the commands. It just seems like humans, social engineering of humans is much more likely.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  27. “Okay, well, we'll talk about possible solutions and what not playing it means. But what are the possible timelines here to you? What are we talking about? We're talking about a set of years, decades, centuries. What do you think”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  28. “So, if we're actually looking at X-risk and S-Risk, as the systems get more and more intelligent, don't you think it's possible to anticipate the ways they can do it? And defending S-T like we do with cybersecurity, we'll do security systems.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  29. “We don't know that But that is a terrifying possibility. We don't want to find out. Like, if terrorists had access to nuclear weapons, how far would they go? Is there a limit to Willing to Your senses, there's some malevolent actors where there's no limit.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  30. “Do you think there is actually people in human history that try to literally maximize human suffering? It's just studying people who have done evil in the world. It seems that they think that they're doing good. It doesn't seem like they're trying to maximize suffering. Just cause a lot of suffering as a side effect of doing what they think is good.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  31. “Okay, so what's the S risk? What are the possible things that you're imagining with S risks? So mass suffering of humans. What are we talking about there caused by AGI?”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  32. “That's the question. Would you really want to live in a world where there's no suffering? That's a dark question. But is there some level of suffering? That reminds us of what this is all for.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  33. “But what if we want to be attention with each other? And through that tension, we understand ourselves and we understand the world. So that's the intellectual journey we're on as a human civilization is we create intellectual and physical conflict and through that figure stuff out.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  34. “But isn't that the whole optimization journey that we're doing as a human civilization? We're looking at geopolitics. Nations are in a state of anarchy with each other. They start wars, there's conflic And oftentimes they have a very different views of what is good and what is evil. Isn't that what we're trying to figure out? Just together, trying to converge towards that. So we're essentially trying to solve the value alignment problem with humans.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  35. “Convert the multi-agent problem into a single agent problem. Okay. Is there any way to so okay that's giving up on the value problem? Well, is there any way to solve the value alignment problem where there's a bunch of humans, multiple humans, tens of humans or eight billion humans that have very different set of values?”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  36. “Basically, give up on value alignment, create and the multiverse theory. This is just create an entire universe for you, where your values are.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  37. “So that's one of the possible outcomes. But what in general is the idea of the paper? So it's looking at multiple agents that are human AI, like a hybrid system, whether it's humans and AIs, or is it looking at humans? Intelligent agent.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  38. “We're playing that video game and now we're creating What maybe we're creating artificial threats for ourselves to be scared about because fear is really exciting. It allows us to play the video game more vigorously.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  39. “So, okay. So why is that not a likely outcome? Why can't they ask systems create video games for us to lose ourselves in each with an individual video game universe?”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  40. “In that world, can't humans do what humans currently do with chess, play each other, have tournaments, even though AI systems are far superior at this time in chess. So we just create artificial games. Or for us, they're real. Like the Olympics. We do all kinds of different competitions and have fun. Maximize the fun and let the AI focus on the productivity.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  41. “Fire of human consciousness is dimmed through that process, but it's not killing humans. So I'm thinking about that, or is the biggest concern, literally, the extinctions of humans?”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  42. “So, are you thinking about mass murder and destruction of human civilization? Are you thinking of with squirrels, you put them in a zoo and they don't really know they're in a zoo? If we just look at the entire set of undesirable trajectories, majority of them are not going to be death. Most of them are going to be just like. Things like Brave New World where The squirrels are fed dopamine and they're all like doing some kind of fun activity and the sort of the fire, the soul of humanity is lost because of the drug that's fed to it. Or like literally in a zoo. We're in a zoo, we're doing our thing. We're like playing a game of sims and the actual players playing that game are AI systems. Those are all undesirable because sort of the free will.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  43. “So there's like an unlimited level of creativity in terms of How humans could be killed. But we could still investigate possible ways of doing it, not how to do it, but at the end, what is the methodology that does it? Shutting off the power. And then humans start killing each other, maybe, because the resources are really constrained. Then there's the actual use of weapons like nuclear weapons or developing artificial pathogens, viruses, that kind of stuff. We could still kind of think through that and defend against it, right? There's a ceiling to the creativity of mass murder of humans here. The options are limited.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  44. “But there's a difference between getting it to do something unintended, getting it to do something that's painful, costly, destructive, and something that's destructive to the level of hurting billions of people or hundreds of millions of people, billions of people, or the entirety of human civilization. That's a big leap.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  45. “There is an incremental improvement of systems. Leading up to AGI. To you, it doesn't matter if we can keep those safe. There's going to be one level of system. At which you cannot possibly control”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  46. “Usually good. It's usually awesome. And I actually don't dream that much when I do, it's awesome. Whole point though is that I wake up refreshed, taking your sleep seriously is really, really important. When you get a chance to sleep, do it in style. And do it on a bed that's awesome And use code Lux to get $350 off the pod for Ultra. This is Alex Friedman podcast. To support it, please check out our sponsors in the description. And now, dear friends, here's Roman Impol What to you is the probability that superintelligent AI will destroy all human civilization?”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  47. “Adds a base that goes between the mattress and the bed frame and can elevate to a reading position. So it modifies the positioning of the bed. So on top of all the cooling and the heating and all that kind of stuff that it can do and do it better in the pod 4, I think it has 2x the cooling power of pod 3. So they're improving on the main thing that they do, but also there's the ultra part that can adjust the bed. It can cool down each side of the bed to 20 degrees Fahrenheit below room temperature. One of my favorite things is to escape the world. On a cool bed with a warm blanket and just disappear for twenty minutes or for eight hours. Your dream world. Everything is possible, where everything is allowed. A chance to explore the Jung in shadow, the good, the bad and the ugly.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  48. “Fast thing, all that kind of stuff. Element is always by my side, and more and more you're going to see probably the sparkling water thing or whatever the element is making. So it's in a can and it's freaking delicious. There's four flavors. The lemon one is the only one I don't like. The other three I really love and I forget their names, but they're freaking delicious. And you're going to see it more and more on my desk, except for the fact that I run out very quickly because I consume them very quickly. Get a simple pack for free with any purchase. Try to drink element dot com slash lex. This episode is also brought to you by ASLEP and it's pod for Ultra. This thing is amazing. The Ultra part of that”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  49. “This episode is also brought to you by Element, Electrolyte, Drink, Mix of Sodium, Potassium, and Magnesium that have been consuming multiple times a day, watermelon salt is my favorite. Whenever you see me drink from a cup on the podcast, almost always is going to be water with some element in it. I use an empty power rate bottle, twenty eight fluid ounces, fill it with water, put one packet of watermelon salt element in it, mix it up, put in the fridge, and then when it's time to drink, I take it out of the fridge and I'd drink it, and I drink a lot of those a day, and it feels good, it's delicious. Whenever I do crazy physical”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source

  50. “For the success of the team, for the happiness of the team, and the individuals in that team. Netsuite is a great system that runs the machine inside the machine. For a knee-sized business thirty seven thousand companies have upgraded to NetSuite by Oracle, take advantage of NetSuite's flexible financing plan at Netsuite. comes netsuite dot com slash lex.”

    2024-06-02 · Lex Fridman Podcast · #431 – Roman Yampolskiy: Dangers of Superintelligent AI · IDENTIFIED FROM THE TRANSCRIPT · source