YouSaid · the spoken record
Nick Bostrom
- lines on the record
- 138
- first
- 2020-03-26
- most recent
- 2020-03-26
- sittings or episodes
- 1
- sources
- podcast
Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections
“So it might not be so much the reality itself that we are after, but it's more that we are maybe involved in certain projects and relationships. And we have a self-identity and these things. That's our values are kind of connected with carrying that forward. And then whether it's inside a tank or outside a tank in Peru or whether inside a computer, outside a computer, that's kind of less important to what we ultimately care about.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Now, do you want to disconnect from this and enter the real world when you have no idea maybe what the real world is? Or maybe you could say, well, you're actually a farmer in Peru growing peanuts and you could live for the rest of your life in this or would you want to continue your dream alive as Alex Friedman going around the world, making podcasts and doing research? So, if the status quo was that they were actually in the experience machine, I think a lot of people might then prefer to live the life that they are familiar with rather than sort of bail out into.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So that could be one thing. I mean, like a big impact on history, maybe it's also something you won't have if you check into this experience machine. So some people might actually feel the life I want to have for me is one where I have a big positive impact on history unfolds. So you could kind of explore these different possible explanations for why it is you wouldn't want to go into the experience machine if that's what you feel. And one interesting observation regarding this NOSIC thought experiment and the conclusions he wanted to draw from it is how much is a kind of a status quo effect. So a lot of people might not want to Jet is on current reality to plug into this three machine. But If they instead were told, well, what you've experienced up to this point was a dream.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“What can you have in the experience machine? Well, I mean, That then becomes an interesting question to explore. But for example, real connection with other people, if the experience machine is a solo machine where it's only you, That's something you wouldn't have there. You would have this objective experience that would be fake people. But if you gave somebody flowers, that wouldn't be anybody they actually got happy. It would just be a little simulation of somebody smiling. But the simulation would not be the kind of simulation I'm talking about in the simulation argument where the simulated creature is conscious. It would just be a kind of smiley face that would look perfectly real to you.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“And so NoSic thought that most people would choose not to enter the experience machine. I mean, many might want to go there for a holiday, but they wouldn't want to check out of existence permanently. And so he thought that was an argument against certain views of value according to what we value is a function of what we experience. Because in the experience machine, you can have any experience you want, and yet many people would think that would not be much value. So therefore, what we value depends on other things than what we Experience”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and different people might have different preferences regarding that. Some might, even if you had any perfect virtual reality. Might still prefer not to spend the rest of their lives there. I mean, in philosophy, there's this experience machine, thought experiment. Have you come across this? So Robert Nozick had this thought experiment where you imagine some crazy super duper neuroscientists of the future have created a machine that could give you any experience you want if you step in there. And for the rest of your life, you can kind of pre-program it in different ways. So your fund streams could come true, you could, whatever you dream you want to be a great artist, a great lover, like have a wonderful life, all of these things. If you step into the experience machine will be your experiences constantly happy. And, but would you kind of disconnect from the rest of reality and you would float there in a tank?”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, I think that might be too low a bar. I mean, if you think, say, when people first had Pong or something like that, I'm sure there were people who wanted to keep playing it for a long time because it was fun and they wanted to be in this little world. I'm not sure whether we'd say it's immersive. I mean, I guess in some sense it is, but like an absorbing activity doesn't even have to be.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“The player character. And so similarly, if this environment that we perceive is similar, it might be that only the parts that come into our view are rendered at any given time. And a lot of aspects that never come into view, say the details of this microphone I'm talking into exactly what each atom is doing at any given point in time might not be part of the simulation only a more coarse grained representation.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“In order to generate the same kind of experiences that we have. And there is a bunch of stuff we don't know. I mean, if you look at, say, current virtual reality environments, one thing that's clear is that we don't have to simulate all details of them all the time in order for, say, the human player to have the perception that there is a full reality in there. You can have, say, procedurally generated, which might only render a scene when it's actually within the view.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“You asked another interesting question as well, which is, would it be Sufficient to just have, say, the brain, or would you need the environment?”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, you could say that, but then the question is, what's the class of computations such that when they are run, consciousness emerges? So if you just have something that adds one plus one plus one plus one, like a simple computation, you think maybe that's not going to have any consciousness. On the other hand, the competition is one like our human brains are performing where As part of the computation, there is like a global workspace, a sophisticated attention mechanism, there is like self-representations of other cognitive processes and a whole lot of other things that possibly would be conscious. And in fact, if it's exactly like ours, I think definitely it would. But exactly how much less than the full computation that the human brain is performing would be required is a little bit, I think, of an open question”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, so if you are a computationalist, you think that what creates consciousness is the implementation of a computation.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“A human with that brain would have, then I think that would be conscious. Now, it's possible you could also generate consciousness without having that detailed simulation. There I'm getting more uncertain exactly how much you could simplify or abstract away.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I guess there is a kind of premise. The simulation argument. Rolled in from philosophy of mind that is that it would be possible to create a conscious mind in a computer and that what determines whether some system is conscious or not is not like whether it's built from organic biological neurons, but maybe something like what the structure of the computation is that it implements. We can discuss that if we want, but I think it would be very, my view that it would be sufficient, say, if you had Computation that was identical to the computation in the human brain down to the level of neurons. So if you had a simulation with 100 billion neurons connected in the same way to human brain and you then roll that forward. With the same kind of synaptic weights and so forth. So you actually had the same behavior coming out of this.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, that would be the type of computer simulation that would contain people like those we think have lived on our planet in the past and like ourselves in terms of the types of experiences they have and where those simulated people are conscious. So like not just simulated in the same sense that a non player character would be simulated in the current computer game where it kind of has like an avatar body and then a very simple mechanism that moves it forward or backwards or but something where the simulated being has a brain let's say that simulated at a sufficient level of granularity that it would have the same subjective experiences as we have.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, certainly, I mean, yeah, ethical concerns. I mean, not really too difficult to do. I mean, in a sense, that's the first assumption that you get to technological maturity where you would have the ability using only a tiny fraction of your resources to create many simulations. So it wouldn't be the case that they would need to spend half of their GDP forever in order to create one simulation. And they had this difficult debate about whether they should invest half of their GDP for this. It would more be like, well, if any little fraction of the civilization feels like doing this at any point during maybe their millions of years of existence, then there would be millions of simulations. But certainly there could be many conceivable reasons for why there would be this many possible reasons for not running ancestor simulations.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, so the number two is the convergence hypothesis. That is that maybe a lot of some of these civilizations do make it through to technological maturity. But out of those who do get there, they all lose interest in creating these simulations. So they just have the capability of doing it, but they choose not to. Not just a few of them decide not to, but. Out of a million, you know, maybe not even a single one of them.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Could kill us. Like, if you look at existential risks, and it might be a different, like the best answer to what would be most likely to kill us might be a different answer than the best answer to the question if there is something that kills almost everyone, what would that be? Because that would have to be some risk factor that was kind of uniform overall possible civilization.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So then certainly there would be a lot of extraterrestrials out there. If a short of that, if the universe is very big, that might be a finite but large number. If we were literally the only one, yeah, then of course. We went extinct, then all civilizations at our current stage would have gone extinct before becoming technological mature. So then it kind of becomes trivially true that a very high fraction of those went extinct. But if we think there are many, I mean, it's interesting because there are certain things that plausibly”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, there are these set of basic questions that always come up in conversations with interesting people, like the Fermi paradox. Like there's like, You could almost define whether a person is interesting, whether at some point the question of the Fermi paradox comes up. So, for what it's worth, it looks to me that the universe is very big. I mean, in fact, according to the most popular current cosmological theories, infinitely big. And so then it would follow pretty trivially that it would contain a lot of other civilizations. In fact, infinitely many if you have some locals stochasticity and infinitely many. you know, infinitely many lumps of matter one next to another, there's kind of random stuff in each one, then you're gonna get all possible outcomes with probability one infinitely repeated.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, if thou are, yeah, then that would virtually all have to succumb in the same way. I mean, That leads off another. I guess there are a lot of little digressions that are interesting. Go there. Let's go there. Dragging us back.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Right. So that's kind of sad, but we have to think not just what might destroy us. I mean, so there could be some... Whatever disasters of meteors slamming the earth a few years from now, that could destroy us, right? But you'd have to postulate in order for this first disjunct to be true that almost all civilizations throughout the cosmos also failed to reach technological maturity.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, well, so it would shift around the probabilities between these three alternatives. If we are very, very far away from being able to create these simulations, if it's like, say, billions of years into the future, then it's more likely that we will fail ever to get there. There are more time for us to kind of go extinct along the way”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“A new ceilings. In terms of basic technological capabilities, I think there is like a finite set of those that can exist in this universe. I wouldn't be that surprised if we actually reached close to that level fairly shortly after we have, say, machine superintelligence. So I don't think it would take millions of years. For a human originating civilization to begin to do this, it's more likely to happen on historical timescales. But that's an independent speculation from... Simulation argument. I mean, for the purpose of the simulation argument, it doesn't really matter whether it goes indefinitely far up or whether there's a ceiling. As long as we know we could at least get to a certain level. And it also doesn't matter whether that's going to happen.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, okay, so I mean, it's a stronger concept than we really need for the simulation hypothesis. I just think it's interesting in its own right So it would be the idea that there is some stage of technological development for you basically maxed out that you developed all those general purpose widely useful technologies that could be developed or at least kind of come very close to 99.9% there or something. So that's an independent question. You can think either that there is such a ceiling or you might think it just because the technology tree just goes on forever.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So you can then combine these two. You could have this kind of nanomolecular ability to build things atom by atom and then say at a spatial scale that would be attainable through space colonizing technology. You could then start, for example, to characterize a lower bound on the amount of computing power that technologically matures civilization would have if it could grab resources in a planets and so forth and then use this molecular nanotechnology to optimize them for computing, you would get a very, very high lower bound on the amount of compute.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Computers, maybe it's easiest. One is we could just imagine bigger computers using exactly the same parts that we have, so you could kind of scale things that way, right? But you could also make processors a bit faster if you had this molecular nanotechnology that Eric Rexter described. He characterized a kind of crude computer built with his parts that would perform at a million times the human brain while being significantly smaller the size of a sugar cube. And he made no claim that that's the optimum computing structure. Like for what, you know, you could build a faster computers that would be more efficient, but at least you could do that if you had the ability to do things that were atomically precise.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, so these would be two examples of technical capability sets that we can have a high degree of confidence. Physically possible in our universe and that the civilization that was allowed to continue to develop its science and technology would eventually attain.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Technologies we already know are possible, then maybe they would discover other new physical phenomena as well that we haven't realized that would enable them to do even more. But at least there is this kind of basic set of capabilities.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“But you could say that, well, if we could put these atoms together in this way, then the system would be stable and it would, you know, rotate at this speed and have these computational characteristics. And he also outlined some pathways that would enable us to get to this kind of molecularly manufacturing in the fullness of time. And you could do other studies we've done. You could look at the speed at which, say, it would be possible to colonize the galaxy if you had mature technology. We have an upper limit, which is the speed of light. We have sort of a lower current limit, which is how fast current rockets go. We know we can go faster than that. making them bigger and have more fuel and stuff. And you can then start to describe the technical affordances that would exist once a civilization has had enough time to develop.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So, there's this interesting concept of technological maturity. It's kind of an interesting concept to have other purposes as well. We can see even based on our current limited understanding what some lower bound would be on the capabilities that you could realize by just developing technologies that we already see are possible. So for example, one of my research fellows here, Eric Drexler, back in the 80s, Studied molecular manufacturing that is, you could analyze using theoretical tools and computer modeling the performance of various molecularly precise structures that we didn't then and still don't today have the ability to actually fabricate.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Theoretically, it could be both. Now, I think it looks like we would technologically be able to get there in a time span that is short compared to, say, the lifetime of planets and other sort of astronomical processes.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so it's like a disjunction. It says at least one of these three is true. It doesn't on its own tell us which one. So the first one is that almost all civilizations at our current stage of technological development. Go extinct before they reach technological maturity. So there is some great filter that makes it so that Basically none of the civilizations throughout maybe a vast cosmos will ever get to realize the full potential of technological development.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Simulation hypothesis so far. There is also the simulation argument, which I tend to make a distinction. So simulation hypothesis, we are living in a computer simulation. Simulation argument, this argument that tries to show that one of three propositions is true, one of which is the simulation hypothesis, but there are two alternatives in the original simulation argument, which we can get to.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“I think not so much for computer science or physics per se. Certainly it would be of interest in philosophy, I think, also to say cosmology or physics in as much as you're interested in the fundamental building blocks of the world and the rules that govern it. If we are in a simulation, there is then the possibility that say physics at the level where the computer running the simulation could be different from the physics governing phenomena in the simulation. So I think might be interesting from point of view of religion or just for kind of trying to figure out what the heck is going on. So we mentioned the”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Is it useful? I guess it's more informative and interesting and maybe important. It's not designed to be useful for something else”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“The ones we have today would do fine. I mean, bigger, certainly. You'd need more. More memory and more processing power. I don't think anything else would be required. Now, it might well be that they do have additional, maybe they have quantum computers and other things that would give them even more oomph. It seems kind of plausible, but I don't think it's a necessary assumption in order to get to the conclusion that a technologically mature civilization would be able to create these kinds of computer simulations with conscious beings inside them.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, so the hypothesis is meant to be understood in a literal sense not that we can kind of metaphorically view the universe as an information processing physical system, but that there is some advanced civilization who built a lot of computers and that what we experience is an effect of what's going on inside one of those computers so that the world around us and our own brains, everything we see and perceive and think and feel would exist because this computer is running certain programs.”
2020-03-26 · Lex Fridman Podcast · #83 – Nick Bostrom: Simulation and Superintelligence · IDENTIFIED FROM THE TRANSCRIPT · source