YouSaid · the spoken record

Eliezer Yudkowsky

lines on the record: 301
first: 2023-04-06
most recent: 2023-04-06
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Like some of my intuition here is like I know how I would do this with dogs. And I think you could ask OpenAI to describe their theory of how to do it with dogs. And I would be like, oh wow, that sure going to get you killed. And that's kind of how I expect it to play out in practice.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't know, I feel like maybe I could do this given thousands of years to breed the dogs in a total absence of ethics, but it would actually be easier with the dogs, I think, than with gradient descent. Because I think it's, well, because the dogs are starting out with neural architecture very similar to human. And natural selection is just like a different idiom from gradient descent. Particular in terms of information bandwidth and like I'd be tearing to breathe the dogs into like very genuinely very nice human and like knowing the stuff that I know that you're typical dog breeder might not know when they set out to be embarked on this project. I would be like early on being like sort of prompting them into the weird stuff that I expected to get started later and trying to observe how they went during that.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it was just like rambling in my attempts to make predictions about these superdogs. You're like asking me to, I mean, I feel like in a world that had anything remotely like its priorities straight, this stuff is not me extemporizing on a blog post. There are 1,000 papers that were written by people who otherwise became philosophers writing about this stuff instead. But, you know, your world has not set its priorities that ways, and I'm concerned that it will not set them that way in the future, and I'm concerned that if it tries to set them that way, it will end up with garbage because the good stuff is hard to verify. But, you know, separate topics.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“The exact ice cream is like quite hard to predict, just like it would be very hard to look at, well, if you optimize something for inclusive genetic fitness, you'll get ice cream. That is a very hard call to make.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“It's hard well. I expect to blow up on you quite bad. I'm trying to think about whether I expect superdogs to be sufficiently in a human frame of reference in virtue of them also being mammals, that a superdog would like create a Human ice cream like you bred them to have preferences about humans and they invent something that is like ice cream to those preferences. Or does it just like go off someplace stranger?”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“If you keep on optimizing the dogs. Which is not the correct course of action. I think I mostly expect this to eventually blow up on you.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“When the dogs can manipulate you, if they get to that point, were the dogs can strategically present the particular appearances to fool you? Were the dogs are aware of the breeding process and possibly having opinions about where that should go in the long run? Where are the dogs are even if just by thinking and by adopting new rules of thought, modifying themselves in that small way? These are some of the points where, like I expect the weird shit to start to happen. And the weird shit will not necessarily show up while you're just reading the dogs.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“So I think that weird shit starts to happen at the point where the dogs get smart enough that they are like, what are these flaws in our thinking processes? How can we correct them? Over the CFAR threshold of dogs, although maybe that's CFAR has some strange baggage, over the Korsibski threshold of dogs after Alfred Korsibski. So I think that Know there's this whole domain where they're stupider than you and sort of like being shaped by their genes and not shaping themselves very much. And as long as that is true, you can probably go on breeding them. And issues start to arise when the dogs are smarter than you.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“As soon as they are past a certain level of intelligence, I object to us like humming and inbreeding them, they can no longer be owned. They are now sufficiently intelligent to not be owned anymore. But let us leave aside all morals. Carry on. In the thought experiment, not in real life. You can't leave out the morals in real life. Do you”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“I think I want to register for the record that the term breeding humans would cause me to look ask at any aliens who were proposed that as a policy action on their part. No, no, no. I said it. Move on”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Hypothetical motives, but anything like that, if you like, optimize it on its own terms without narrow down to where you want it to end up because it actually felt nice to you the way that you define niceness. Like it's all going to have somewhere else, somewhere that isn't as nice, something maybe where we'd be like sooner scour the surface of the planets clean with nuclear fire rather than let that AI come into existence though. I do think those are also probable because, you know, instead of hurting you know, there's like something more efficient for it to do that maxes out its utility function.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Maybe it's just as happy with bacteria because there's more of them. And that's equally old-fashioned. You create the specific spruce tree over there? Maybe from its perspective, a generic bacterium is just as good a form of life as generic spruce tree is of a spruce tree. And like, this is not specific to the example that you gave. It's me being like, well, suppose we took a criterion that sounds kind of like this and asked, how do we actually maximize it? What else satisfies it? Not just you're like trying to argue the AI into doing what you think is a good idea by giving the AI reasons why it should want to do the thing under like some set of like”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“And it's not a coincidence that I can zoom in and poke at this and ask questions like this and that you did not ask these questions of yourself. You are imagining nice ways you can get the thing, but reality is not necessarily imagining how to give you what you want. And the AI is not necessarily imagining how to give you what you want. And for everything you can be, like, oh, like hopeful thought, maybe I get all the stuff I want because the AI reasons like this. Because it's the optimism inside you that is generating this answer. And if the optimism is not in the AI, if the AI is not specifically being like, well, how do I pick a reason to do things that will give this person a nice outcome? You're not going to get the nice outcome. You're going to be reliving the last day of your life over and over. It's going to create or maybe create old-fashioned humans, ones from 50,000 years ago. Maybe that's more quaint.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“You see how the general trend I'm trying to point out to you here is you like have a rationalization for why they might do thing that is allegedly nice. And I'm saying like why exactly are they wanting to do thing? Well, if they want to do thing for this reason, maybe there is a way to do this thing that isn't as nice as you're imagining, and this is systematic. Your imagining reasons they might have to give you nice things that you want, but they are not you, not unless we get, you know, not unless we get this exactly right and they actually care about the part where you want some things and not others. You are not describing something, you are doing for the sake of the spruce trees. Do spruce trees have diseases in this world of yours? The disease has got to live? Do they get to live on spruce trees?”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Okay, but do you see how this perhaps leads to everybody's severed heads being kept alive in jars on its own premises, as opposed to humans getting the glorious transhumanous future? No, no, they have the globeist future. Those are not real spruce trees. You know, like you're talking about like plain old spruce trees you want to exist, right? Not the sparkling giant spruce trees with built-in rockets. You're talking about humans being kept as pets in their ancestral state forever, maybe being quite sad, maybe they still get cancer and die of old age, and they never get anything better than that. Does it keep us around as we are right now? Do we relive the same day over and over again? Maybe this is the day when that happens.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“If you still want to come the day, I don't think I myself would oppose it unless there'd be like distant aliens who are very, very sad about what we were doing to the mitochondria and then I don't want to ruin their day for no good reason.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“And people are like, oh, I'm going to optimize the test specifically. And they'll get higher scores than the carpenters and be worse at carpentry because they're like optimizing the test. And that's the story behind ice cream And you zoom in and look at the mechanics and not the grand scale view. Because the grand scale view just never gives you the right answer, basically. Like anytime you ask what would happen if you applied the grand scale view philosophy in the past, it's always just like, I don't see why this thing would change. Oh, it changed. How weird? Who could have possibly expected that?”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Generate additional options not blindly but according to the things that they want, and they invent ice cream, they you know, like not at random. It doesn't just like get coughed up at random. They are like searching the space of things that they want and generating new options for themselves that optimize these things more that weren't in the ancestral environment and Good Hart's law applies Goodheart's curse applies once you that like as you apply optimization pressure the correlations that were found naturally come apart and aren't present in the thing that gets optimized for like you know like just give some people some tests who've never gone to school the ones who high score high in the test will know the problem domain because they you know like you just like give gives a bunch of carpenters a carpentry test the ones who score high in the carpentry test will like know how to carpenter things then you're like yeah I'll like pay you for high scores in the carpentry test I'll give you this carpentry degree”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I think you look at the mechanics. You say, as people have gotten more options, they have gone further outside the ancestral distribution, and we zoom in and it's like there's all these different things that people want. And there's this narrow range of options that they had fifty thousand years ago. And the things that they want have maxima or optima fifty thousand years ago at stuff that coincides with reproductive fitness, and then as a result of the humans getting smarter, they start to accumulate culture, which produces changes on a time scale faster than natural selection runs, although it is still running contemporaneously the humans are just running faster than natural selection. It didn't actually halt.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, but plus, you wouldn't like really, I worry you wouldn't sell that transhumanism thing as well as it could be sold.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, man, I somewhat tempted to do that just for the sheer chaos and point out the drastic selection effects of, A, it's my Twitter followers. B, they read through a 4,000-character tweet. I feel like this is not likely to be truly very informative by my standards, but part of me is amused by the prospect for the chaos.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“I'd have to explain that poll pretty carefully because, you know, they haven't got the intelligence headbands yet, right?”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“But you would So, yeah, you just look down at your fellow humans. You have no confidence in their ability to tolerate weirdness, even if they have underlined?”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Like, look at the thing with ice cream. Look at the thing with condoms. Look at the thing with pornography. See where this is going.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Because dive under the surface, look at the things that have changed. Why did they change? Look at the processes that are generating those choices.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Look at all this stuff humans haven't changed yet. You say now, selecting the stuff we haven't changed yet, but if you go back twenty thousand years and be like, look at the stuff intelligence hasn't changed yet, you might very well select a bunch of stuff that was going to fall twenty thousand years later is the thing I'm trying to gesture at here.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Is the thing I'm saying there. You look around, it looks so normal, according to you. Who grew up here? If you'd grown up a millennium early or your argument for the persistence of normality might not seem as persuasive to you after you'd seen that much change.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“They're not evidence one way or the other because the basic prediction is like if you offer things enough options, they will go out People with language and being like, they haven't taken over the world yet. And like they have knock-on way out of distribution yet. And it's like they haven't had general intelligence for long enough to accumulate the things that would give them more options such that they could start trying to select the weirder options. The prediction is like when you give yourself more options, you start to select ones that look weird or relative to the ancestral distribution. As long as you don't have the weird options, you're not going to make the weird choices. And if you say like, we haven't yet observed your future, that's fine. But acknowledge that then evidence against that future is not being provided by the past.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“We might want to be polite to the sort of aliens who would be disturbed by it because they don't have quality and they just see like things don't want venom injected into them therefore they should not have venom. We might conserve some parts of nature, but again it's like firing an arrow and then drawing a circle around the target”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, the thing I'm trying to say is you're like, well, if you looked at the humans, would you not expect them to end up incompatible with the spruce trees? And I'm being like, sir, you a human have looked back and looked at how humans wanted the universe to be and been like, well, would you not have anticipated in retrospect that humans would like want the universe to be otherwise? And I agree that we might want to conserve a whole bunch of stuff. Maybe we don't want to conserve the parts of nature where things bite other things and inject venom into them, and the victims die in terrible pain, maybe even, you know, I think that many of them don't have qualia. This is disputed. Some people might be disturbed by it even if they didn't have qualia.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“And the answer is yes, as a very special case of us being the sort of things that would make some of us would maybe conclude that we specifically wanted spruce trees to go on existing at least on earth in the glorious transhuman future and their votes winning out against those of the mitochondrial liberation front.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“The long term, we sure aren't. I mean, like, maybe if we win, we'll have there be a space for spruce trees. Yeah, so you can have spruce trees as long as the mitochondrial liberation front does not object to that.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“It gets the super vast majority of predictions of utility functions are incompatible with human existing. I can make a mistake and I'll still be incompatible with human existing. I can just be like, I can just describe a randomly rolled utility function and end up with something incompatible with humans exist.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“No, no, what I'm saying is you're like, oh, well, prediction. Oh, no, no. I don't like my prediction. I want a different prediction.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, you definitely want to use the alien observation of 10,000 planets like this one prior for what you get after training on-like thingx.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“I expect things to continue on largely as they have, and, you know, and what distinguishes that from despair is that at the moment people were telling me, like, no, no, if you go outside the tech industry, people will actually listen. I'm like, all right, let's try that. Let's write the time article. Let's jump on that. Let's see if it works. It will lack dignity not to try. That's not the same as expecting as being like, oh yeah, oh, over 50%, they're totally going to do it that time article is totally going to take off. I'm not currently not over 50% on that. You said any one of these things could mean, and yet, like, even if this thing is technically feasible, that doesn't mean the world's going to do it. We are presently quite far from the world being on that trajectory. Or of doing the things that we needed to create time to pay the alignment tax to do it.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Becoming the Republican, not one eighth, what six, six stages of Dune. Therefore, he had less than 164th chance of becoming, I think, just a Republican candidate not winning. So, yeah, so you can't just break things down into stages and then say there for the probability is zero. You can break down anything at the stages. But even so, you're asking me, well, isn't over 1% that it's possible? I'm like, yeah, possibly even over 10%. That doesn't get me to... The reason why go ahead and tell people, yeah, don't put your hope in the future. You're probably dead is that The existence of this technical array of hope, if you do just the right things, is not the same as expecting that the world reshapes itself to permit that to be done without destroying the world in the meanwhile”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Go in at sufficiently the right angle to materialize the technical chances and not do it in the way that just ends up a suicide. Or if you're lucky, like gives you the clear warning signs and then people actually pay attention to those instead of just optimizing away the warning signs. And I don't want to make the sound like the multiple stagage file see of like, oh no, more than one thing has to happen, therefore the resulting thing can never happen, which super clear case in point of why you cannot prove anything will not happen this way of Nate Silver arguing that Trump needed to get through six stages to become the Republican presidential candidate, each of which was less than half probability.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Just shut it all down. They could get shut it all down and then not do the things that they would need to do to have an exit strategy. I feel like even if you told me that they went for shut it all down, I would be like, then next expect them to have no exit strategy until the world ended anyways. But perhaps I underestimate them. Maybe there's a will in humanity to do something else which is not that. And if there really were, yeah, I think I'm even over 10% that. Would be a technically feasible path if they looked in just the right direction. But I am not over fifty percent on them. Actually, doing the shut it all down. I am not if they do that, I am them not over fifty percent on there really truly being the will of something else that is not that to really have an exit strategy. Then from there you have to”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“I am not very high on us doing it. Maybe I will be wrong, maybe the time article I wrote saying shut it all down gets picked up and there are very serious conversations and the very serious conversations are actually effective in shutting down the headlong plunge and there is a narrow exception carved out for the kind of narrow application of trying to build an artificial general intelligence that applies its intelligence narrowly and to the problem of augmenting humans. And I think might be a harder sell to the world.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, because it's not just a question of the technical feasibility of can you build a thing that applies its general intelligence narrowly to The neuroscience of augmenting humans, it's a question of like the So, like, one, I feel like that is probably over 1% technical feasibility. But the world that we are in is so far. So far from From doing that, from trying, trying the way the work could actually work. Not like the try where like, oh, you know, like, well, we'd like to just do a bunch of RLHF to try to have it spit out output about this thing, but not about that thing and that, that, that, no, no, not that. Yeah, 1% that we, that humanity could do that if it tried and tried in just the right direction as far as I can. Perceive angles in this space. Yeah, I'm over 1% on that.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“It's kind of weird because Like small, large amounts of intelligence don't automatically make you a computer programmer, even if you are computer programmer, you don't automatically get security mindset. But it feels like there's some level of intelligence where you ought to automatically get security mindset. And I think that's about how hard you have to augment people to have them able to do alignment, like the level where they have security mindset not because they were special people with security mindset, but just because they're that intelligent that you just automatically have security mindset, I think that's about the level where a human could start to work on alignment, more or less.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Where do people have this notion of getting AIs to help you do your AI alignment homework? Why can we not talk about having enhance human intelligence instead?”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“No you have spoken it exists it cannot be called back there are no takebacks there is no going back there is no going back Go ahead.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“Okay, so somewhere on the internet. Is a list of hashes followed by the string hashed. Is a simple demonstration of how you can go on getting lower losses by throwing a hypercomputer at the problem. There are pieces of text on there that were not produced by humans talking in conversation, but rather by like lots and lots of work, to determine get extract experimental results out of reality. That text is also on the Internet Maybe there's not enough of it for the machine learning paradigm to work, but I'd sooner buy that like the The GPT system's just bottlenecks short of being able to predict that stuff better rather than that. But you can maybe buy that, but like the notion that you only have to be smart as a human to predict all the text as the internet as soon as you turn around and stare at that a bit is just transparently false.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“That's not how I would be quite surprised if that's how anything works. For one thing, because it's you know like. Like for an alien to be an actress playing all the humans on the internet. For another thing, well, first of all, you realize in principle that the task of minimizing losses on predicting human text does not have a, yeah, you understand that in principle this does not stop when you're as smart as a human, right? Like you can see that the computer science of that.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“A year, what are you going to do with that year before the next generation of systems come out that are not held in check by humans because they are not roughly in the same power intelligence range as humans? Maybe you can get a year with a year like that. Maybe that actually happens. What are you going to do with that year that prevents you from dying of the year after?”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“And other people go ahead and spend lots of money on it anyways. And everybody makes the same mistakes. Nate Soris has a post about it. I forget the exact title, but everybody coming into alignment makes the same mistakes.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“I know people who have a billion dollars. I don't know how to throw a billion dollars at outputting lots and lots of alignment stuff.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source
“It failed to gel. The alignment field failed to gel. That's my judgment to the like, well, you just thrown a ton of more money and then it's all solvable. Because I've seen people try to amp up the amount of money that goes into it and the stuff coming out of it has not To the places that I would have considered obvious a while ago, and I can print out all my interest sheets for it, and each time I do that, it gets a little bit harder to make the case next time.”
2023-04-06 · Dwarkesh Podcast · Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality · IDENTIFIED FROM THE TRANSCRIPT · source