YouSaid · the spoken record

Max Tegmark

lines on the record: 189
first: 2023-04-13
most recent: 2023-04-13
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Lot of people say that the two biggest risks from large language models are Spreading disinformation, harmful information of various types And second, being used for offensive Cyber weap I think that was not the two greatest threats. They're very serious threats, and it's wonderful that people are trying to mitigate them. It's a much bigger elephant in the room, is how disrupt our economy in a huge way, obviously, and maybe take away a lot of the most meaningful jobs. And an even bigger one is the one we spent so much time talking about here. Becomes the bootloader for the more powerful AI.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm not saying that. I have to say it feels in a bit weird to say it because MIT is like the cradle of the open source movement. And I love open source in general, power to the people, let's say. But there's always going to be some stuff that you don't open source. And, you know, it's just like you don't open source. So we have a three-month-old baby, right? When he gets a little bit older, we're not going to open source to him all the most dangerous things he could do in the house.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“The knowledge that you get. From open sourcing everything we do now. Might very well be powerful enough that people looking at that. Can use it to build the things that you're really threatening again. Let's get it. Remember, OpenAI GPT-4 is of ABAI. Baby sort of baby proto almost little bit AGI according to what Microsoft's recent paper said, right? It's not that they were scared of. What we're scared about is people taking that who are... Who might be a lot less responsible than the company that made it and just go into town with it? That's why An information hazard. There are many things which are not open sourced right now in society for a very good reason. How do you make? Certain kind of very powerful toxins out of stuff you can buy at Home Depot. We don't open source those things for a reason. And this is really no different.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Think the answer right now is no I think the answer early on was yes. We could bring in The wonderful creative thought process of everybody on this. But asking, should we open source GPT-4 now is just the same as if you say, well, should we open source? How to build really small nuclear weapons? Should we open source? How to make bioweapons? Should we open source? How to make... A new virus that kills 90% of everybody who gets it. Of course, we shouldn't”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and they're the same challenge. If we can keep AI safe, that's how we're going to get multiplanetary very efficiently.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“For billions of years to come, we totally understand now spread out into space, to other solar systems, to other galaxies, and flourish for billions and billions of years. And this to me is a very, very hopeful vision. That really motivates me to fight. And coming back to the end to something you talked about again, the struggle, how the human struggle is one of the things that also really gives meaning to our lives. If there's ever been an epic struggle, this is it. And isn't it even more epic if you're the underdog, if most people are telling you this is going to fail? It's impossible, right? And you persist. You succeed, that's what we can do together as a species on this one. A lot of pundits are ready to count this out.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“We have probably both lost someone we really cared about some disease that we were told was incurable. Well, it's not. There's no law of physics saying you have to die of that cancer or whatever. Of course you can cure it. And there are so many other things that with our human intelligence have also failed to solve on this planet, which AI could also very much help us with, right? So if we can get this right, just be a little more chill and slow down a little bit until we get it right Mind blowing how awesome our future can be. We talked a lot about stuff on Earth that can be great. But even if you really get ambitious and look up into the skies, right, there's no reason we have to be stuck on this planet for the rest of”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, was it Henry Ford who said that? If you tell yourself that it's impossible, it is Not make that mistake And this is a big mistake society is making, I think, all in all. Everybody is so gloomy and the media are also very biased towards if it bleeds, it leads and gloom and doom, right? Visions of the future we have or dystopian, which really demotivates people. We want to really, really, really focus on the upside also to give people the willingness to fight for it. AI, you and I mostly talked about gloom here again, but let's not forget that, you know.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“How you do psychological warfare, you persuade the other side that it's hopeless so they don't even fight. And then, of course, you win, right? Let's not do this psychological warfare on ourselves and say there's 100% probability we're all screwed anyway. Sadly, I do get that a little bit sometimes from some young people who are so convinced that we're all screwed that they're like, I'm just going to play computer games and do drugs because we're screwed anyway, right? It's important to keep the hope alive because it actually has a causal impact and makes it more likely that we're going to succeed.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“That's my vision. I'm not saying sitting here saying I'm confident 100% sure that it's going to work. But I don't think the chance is certainly not zero either. And it will certainly be possible to do for a lot of really cool AI applications that we're not using now. So we can have a lot of the fun that we're excited about if we do this. We're going to need a little bit of time. That's why it's good to pause and put in place requirements. One more thing also, I think. Someone might think 0% chance we're going to survive. Let's just give up, right? That's very dangerous. Because there's no more guaranteed way to fail than to convince yourself that it's impossible and not try. When you study history and military history, the first thing you learn is that that's”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Discover the knowledge in the first place, but we don't stop with a black box and use that. We then do a second round of AI where we use automated systems to extract out the knowledge and see what are the insights it's had. And then we put that knowledge into a completely different kind of architecture programming language or whatever that's made in a way that can be both really efficient and also Is more amenable to formal verification.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“No, I think it's hopeless. That's like trying to prove verb about phi spaghetti. What I think, well, the vision I have for success is instead that just like we human beings were able to look at our brains and distill out the key knowledge, Galileo. When his dad threw him an apple when he was a kid, he was able to catch it because his brain could, in this funny spaghetti kind of way, predict how parabolas are going to move, his Kanaman system one, right? But then he got older and it's like, wait, this is a parabola. It's y equals x squared. I can distill this knowledge out, and today you can easily program it into a computer and it can simulate not just that, but how to get to Mars and so on, right? I envision a similar process where we use the amazing learning power of neural networks.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“For the first time, actually, since the many, many years that have passed since I and many others started warning about this, I feel. Most people are getting it now. It's just talking. This guy in the gas station, they were a house the other day. And he's I think we're getting replaced. And I think that's positive that we're finally seeing this reaction, which is the first step towards solving the problem. Second, I really think that this vision of only running AI's really, if the stakes are really high, that can prove to us that they're safe. It's really just virus checking in reverse again. I think it's scientifically doable. I don't think it's hopeless. We might have to forfeit some of the technology that we could get if we were putting blind faith in our AIs, but we're still going to get amazing stuff.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“First of all, I tremendously respect Elias Jurkowski and his thinking. Second, I do share his view that there's a pretty large chance that we're not going to make it as humans. There won't be any humans on the planet. Not the distant future. And that makes me very sad. We just had a little baby, and I keep asking myself. How old is even going to get? And I ask myself. It feels, I said to my wife recently, it feels a little bit like I was just diagnosed with some sort of. Has some risk of dying from and some risk of surviving. Except this is a count of cancer which will kill all of you humanity. So I completely take seriously his concerns. Think Don't absolutely think it's hopeless. I think there is, first of all, a lot of momentum now.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“I want to win the war on life. I think we humans together with all our fellow living things on this planet will be better off if we can. Remain in control over the non living things and make sure that they work for us. I really think it can be done.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Not a psychologist. I'm out of my depth here, but I suspect that part of the reason why teen suicide and suicide in general in the US is that record-breaking levels is actually caused by, again, AI technologies and social media, making people spend less time and actually just human interaction. We've all seen a bunch of good-looking people in restaurants staring into the rectangles instead of looking into each other's eyes. So that's also a part of the war on life that we're replacing so many. Really life affirming things by technology. We're putting technology between us, the technology that was supposed to connect us is actually distancing ourselves from each other. And oh and then we're giving ever more power to things which are not alive. These large corporations are not living things, right? They're just maximizing profit there.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Big parts of it, even though most people thought that was a bad idea, always start doing ocean acidification, wiping out all sorts of species. Oh, now we have all these close calls. You almost had a nuclear war. And we're replacing more and more of the biosphere with non-living things. We're also replacing in our social lives a lot of the things which were so valuable to humanity. A lot of social interactions now are replaced by people staring into their rectangles, right?”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Virtually agrees on that, hey, you know, like having no humans on the planet. In the near future, let's not do that, right? Looked at something like the United Nations Sustainable Development Goals. Some of them were quite ambitious. Basically, all the countries agree, US, China, Russia, Ukraine, they all agree. So instead of quibbling about the little things we don't agree on, let's start with the things we do agree on and get them done. Instead of being so distracted by all these things we disagree on that Moloch wins because frankly Moloch going wild now. It feels like a war on life. Playing out in front of our eyes. If you just look at it from space, you know. We're on this planet, beautiful, vibrant ecosystem. Now we start chopping down.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, for starters, begin with a simple problem of just making sure that the system that you own that's supposed to be loyal to you has to prove to itself that it's always going to do the things that you actually want it to do. And if it can't prove it, maybe it's still going to do it, but you won't run it. So you just forfeit some aspects of all the cool things AI can do. I bet you dollars of donuts, it can still do some incredibly cool stuff for you There are other things too that we shouldn't sweep under the rug. Like, not every human agrees on exactly what direction we should go with humanity, right? You've talked a lot about geopolitical things on this on your podcast to this effect. Think that shouldn't distract us from the fact that there are actually a lot of things that everybody in the world”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“The way your proof checker works is too limited, and we have this new hypermath, and it's true. But then I would just take the attitude, okay, I'm going to forfeit some of the supposedly super cool technologies. I'm only going to go with the ones that I can prove in my own trusted proof gesture. Then I think it's fine. Of course, this is not something anyone has successfully implemented at this point, but I think I just give it as an example of hope. We don't have to do all the work ourselves. This is exactly the sort of very boring and tedious task that is perfect to outsource to an AI. And this is a way in which less powerful agents like us can actually Continue to control and trust more powerful ones.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Leap is when it runs away. I don't buy that argument. I think no matter how super intelligent an AI is, it's never going to be able to prove to me that there are only finitely many primes, for example. It just can't. Can try to snow me by making up all sorts of new weird rules of deduction that And say, trust me, you know.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“To lie to you and give me a proof that I'm gonna think is correct. But it's not me. It's lying to that trick my proof checker.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Why don't you flip this around, and this is an idea, give credit to Steve Amandro for, so that it will only run the code if it can prove, instead of not running it, if it can prove that it's not trustworthy, if it will only run it if it can prove that it's trustworthy. So it asks the code, prove to me that you're going to do what you say you're going to do. And it gives you this proof. And you, a little proof taker, can check it. Now you can actually trust an AI that's much more intelligent than you are, right? It's its problem to come up with this proof that you could never have found, that you should trust it.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Prove that they can be trustworthy. They're forgetting that AI can help us prove that things work. And there's this very fundamental fact that in math, it's much harder to come up with a proof that it is to verify that the proof is correct. You can actually write a little proof checking code. It's quite short, but you can assume and understand. And then it can check the most monstrously long proof ever generated even by a computer and say, yeah, this is valid. So, right now, This approach with virus checking software, that it looks to see if there's something you should not trust it. And if it can prove to itself that you should not trust that code, it warns you, right?”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“I think AI has the power to really heal a lot of the rifts we're seeing by creating trust systems. It has to get away from this idea today with some fact checking sites which might themselves have an agenda and you just trust it because of its reputation. Want to have it these sort of systems they earn their trust and they're completely transparent. This, I think, would actually help a lot. That can, I think, help heal the very dysfunctional conversation that humanity has about how it's going to deal with all its... Biggest challenges in the world today. On the technical side, another common sort of gloom I get from people saying, we're just screwed, there's no hope is, well, things like GPT-4 are way too complicated for a human to ever understand.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Project that I'm working on right now is the outgrowth of improved news foundation together with a metaculous folks is seeing if we can really scale this up a lot with more powerful AI. Because I would love it for there to be a really powerful truth-seeking system. Is trustworthy because it keeps being right about stuff. And people who come to it Maybe look at its latest trust ranking of different pundits and newspapers, et cetera. If they want to know why someone got a low score, they can click on it and see all the predictions that they actually made and how they turned out. This is how we do it in science. You trust scientists like Einstein who said something everybody thought was bullshit and turned out to be right get a lot of trust point. And he did it multiple times even.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Think something AI can help enormously with. Example Little baby step in this direction is this website called Metaculus, where People bet and make predictions, not for money, but just for their own reputation. And it's kind of funny, actually. You treat the humans like you treat AI as you have a loss function where they get penalized if they're super confident on something and then the opposite happens Whereas if you're kind of humble and then you're like, I think it's 51% chance this is going to happen, and then the other happens, you don't get penalized much. And what you can see is that some people are much better at predicting than others. They've earned your trusts, right?”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“didn't see that reply actually But I really resonate with it because AI is not evil. It caused people around the world to hate each other much more. But that's because we made it in a certain way. It's a tool. We can use it for great things and bad things. And we could just as well have AI systems. And this is part of my vision for success here, truth-seeking AI that really brings us together again. Why do people hate each other so much between countries and within countries? It's because These have totally different versions of the truth, right? They all had the same truth that they trusted for good reason because they could check it and verify it and not have them believe in some self-proclaimed authority, right? They wouldn't be nearly as much hate. There'd be a lot more understanding instead.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Third of all teenage girls in the US have been thinking about suicide. Those are steps in they're totally the wrong direction. And it's important to keep our eyes on the prize here. We have the power now for the first time in the history. Harness artificial intelligence to help us really flourish. Help bring out the best in our humanity rather than the worst of it to help us have really fulfilling experiences that feel truly meaningful. And you and I shouldn't sit here and dictate the future generations what they will be. Let them figure it out. But let's give them a chance to live and not foreclose all these possibilities for them by just messing things up, right?”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“That'll only happen if we steer in that direction. That's absolutely not the default outcome. That's why income inequality keeps going up. That's why the life expectancy in the US has been going down. Now I think it's four years in a row. I just read a heartbreaking study from the CDC about how some people...”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Sometimes that sweeten got a little better off because it got a little bigger, and then Denmark got a little bit better off because Sweden got a little bit smaller. But then technology came along and we both got just dramatically wealthier without taking away from anyone else. It was just a total win for everyone. And AI can do that on steroids. You can build safe AGI, if you can build superintelligence. Basically, all the limitations that cause harm today can be completely eliminated. It's a wonderful possibility. And this is not sci-fi. This is something which is clearly possible according to laws of physics. And we can talk about ways of making it safe also. But unfortunately,”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Pitching the rebranding again from Homo sapiens to Homo sapiens. The meaningful experiences. And just to inject some optimism in this year so we don't sound like a bunch of gloomers, you know, we can totally have our cake and eat it. You hear a lot of totally bullshit claims that we can't afford having more teachers. Have to cut the number of nurses. That's just nonsense, obviously. With anything, even quite far short of AGI, we can dramatically improve. Grow the GDP and produce this wealth of goods and services. It's very easy to create a world where everybody is better off than today, including the richest people can be better off as well, right? It's not a zero-sum game technology. Again, you can have two countries like Sweden and Denmark had all these ridiculous wars century after century.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Taking care of children in the daycare center, maybe doing art, etc., etc. And even if it were possible to automate that wavelength, we don't need to do that, right? We built these machines.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“You said you can code five times faster, then you need five times fewer programmers. Maybe there will be more output also, but you'll still end up using fewer program, needing fewer programmers than today. And I love coding. I think it's super cool. So we need to stop and ask ourselves why, again, are we doing this as humans? I feel that AI should be built by humanity for humanity. And let's not forget that. It shouldn't be by Moloch for Moloch. Or what it really is now is kind of biomoch, which doesn't make any sense. It's for us that we're doing it. And it would make a lot more sense if we develop, figure out gradually safely how to make all this tech. And then we think about what are the kind of jobs that people really don't want to have, you know, automate them all. And then we ask what are the jobs that people really find meaning in?”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Some arguments which are too glib. Sometimes people say, well, that's all that's going to happen. We're getting rid of the boring, tedious, dangerous jobs. It's just not true. There are a lot of really interesting jobs that are being taken away now. Journalism is going to get crushed. Coding is going to get crushed. I predict the job market for programmers, salaries are going to start dropping”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“And the stable diffusion and techniques like this, they're really beginning to blow away some real, some jobs that people really love having. It was a heartbreaking article just posted just yesterday on social media I saw about this guy who was doing 3D modeling for gaming and he All of a sudden now they've got this new software says prompts and he feels this whole job that he loved has lost its meaning, you know. I asked GPT 4 to rewrite Twinkle, Twinkle, Little Star in the style of Shakespeare. Couldn't have done such a good job. It was really impressive. You've seen a lot of the art coming out here, right? So I'm all for Automating away the dangerous jobs and the boring jobs, but I think.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“If we get to the point where most humans aren't needed anymore, I think it's quite naive to think that they're going to still be treated well. You know, we say that, yeah, yeah, everybody's equal and the government will always protect them. But if you look in practice, groups that are very disenfranchised and don't have any actual power Usually gets screwed. And now in the beginning, so industrial revolution. We automated away muscle work. But that worked out pretty well eventually because we educated ourselves and started working with our brains instead and got usually more interesting, better paid jobs. But now we're beginning to replace brain work. So we replaced a lot of boring stuff. We got the pocket calculator so you don't have people multiplying numbers anymore at work. Fine. There were better jobs they could get. But now GPT-4.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“To be blunt, my personal guess, and people can quibble with this we're just gonna, there won't be any humans. That's it. That's what I mean by lose. We can see in history, once you have some species or some group of people who aren't needed anymore Doesn't usually work out so well for them, right? There were a lot of horses that were used for traffic in Boston, and then the car gotnamed, and most of them got, well, we don't need to go there. And if you look at... Humans, right now, why did the labor movement succeed? And after the Industrial Revolution, because it was needed. Even though we had a lot of Molochs and there was Chin labor and so on, you know. Company still needed to have workers. And that's why strikes had power”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“We said six months because we figured the number one pushback we were going to get in the West was like, but China. And everybody knows there's no way that China is going to catch up with the West on this in six months. So that argument goes off the table. And you can forget about geopolitical competition and just focus on. A real issue. That's why we put this”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, we have this pause from time to time and look into our hearts and ask, why are we doing this? Am I still going towards Austin or have I gone too far? Maybe we should change direction.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“So, this is Moloch again showing up, and I just want to anyone here who has any concerns about late stage capitalism having gone a little too far, you should worry about superintelligence because it's the same villain in both cases. It's Moloch.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Take over entire continents just so we can get more supercomputer facilities in there and figure out how to cure cancer backwards. And then you're like, wait, that's not what I wanted, right? And the issue with capitalism and the issue with runaway AI have kind of merged now because the Moloch I talked about is exactly the capitalist Moloch that we have built an economy that has optimizing for only one thing. And that worked great back when things were very inefficient, and then now it's getting done better. And it worked great as long as the companies were small enough that they couldn't capture the regulators. That's not true anymore, but they keep optimizing, and now They realize that these companies can make even more profit by building ever more powerful AI, even if it's reckless. But optimize more and more and more, more, more.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“If you have an AI that actually has power over the world and you just give it one goal and just like keep optimizing that, most likely everybody can be like, yay, this is great in the beginning. Things are getting better But it's almost impossible to give it exactly the right direction to optimize in. Eventually, our hair breaks loose, right? Nick Bostrom and others are given examples that sound quite silly. Like, what if you just want to tell it? Cure cancer or something, and that's all you tell it. Maybe it's going to decide.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Capitalism is exactly like that too. We wanted to get stuff done more efficiently that people wanted. Introduce the free market Things got done much more efficiently than they did in, say, communism. And it got better. But then it just kept optimizing and kept optimizing. And you got ever bigger companies and ever more efficient information processing and now also very much powered by IT. And eventually, a lot of people are beginning to feel, wait, we're kind of optimizing a bit too much. Why did we just chop down half the rainforest? And why did suddenly these regulators get captured by lobbyists and so on? It's just the same optimization that's been running for too long.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Back to Austin, for example, and you're like, okay, yeah, let's just go south, but you put in exactly the right sort of the right direction. Just optimize that. South is possible. You get closer and closer to Austin. But there's always some little error. So you're not going exactly towards Austin, but you get pretty close. But eventually you start going away again. And eventually you're going to be leaving the solar system. And they proved it's a beautiful mathematical proof. This happens generally. And this is very important for AI because even though Stuart Russell has written a book and given a lot of talks on why it's a bad idea to have AI just blindly optimize something, that's what pretty much all our systems do. Have something called the loss function that we're just minimizing or reward function we just minimize maximizing.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“So I was very interested in how you could use market forces to just get stuff done more efficiently, but give the right incentives to the market so that it wouldn't do really bad things. So Dylan had Phil Manel, who's a professor and colleague of mine at MIT. Wrote this really interesting paper with some collaborators recently where they proved mathematically that if you just take one goal, that you just optimize for. On and on and on indefinitely that you think is going to You in the right direction. What basically always happens is in the beginning it will make things better for you. If you keep going, at some point, it's going to start making things worse for you again. And then gradually it's going to make it really, really terrible. So just as a simple, the way I think of the proof is suppose you want to go from here.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“And this carbon chauvinism, as I like to call it, that you can only have our level of intelligence in humans, that there's something magical about it. Whereas the people in the tech companies... Build this stuff, they all realize that intelligence is information processing of a certain kind. And it really doesn't matter at all whether the information is processed by carbon atoms in neurons and brains or by silicon atoms and some technology we build. You brought up capitalism earlier, and there are a lot of people who love capitalism and a lot of people who really, really... don't and It struck me recently that what's happening with capitalism here is exactly analogous to the way in which superintelligence might wipe us out You know, I studied economics for my undergrad, Stockholm School of Economics, yay.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“But it's not in anyone's incentive that we go over the cliff. And it's not like everybody's in their own car. All the cars are connected together with a chain. So if anyone goes over, they'll start dragging others down, the others down too. And so ultimately it's in the selfish... Interests also of the people in the companies to slow down when you start seeing the contours of the cliff there in front of you. And the problem is that even though the people who are building the technology, And the CEOs, they really get it. The shareholders and these other market forces, they are people who don't, honestly, understand that the cliff is there. They usually don't, you have to get quite into the weeds to really appreciate how powerful this is and how fast. And a lot of people are even still stuck again in this idea that...”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“For us, again, there's this cliff Gets quite scenic. The closer the cliff they go, the more money there is, the more gold ingots throw on the ground. You can pick up or whatever. So you want to drive there”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source
“Absolutely. I mean, we've seen that many other sectors where you've had the free market produce quite good things without causing particular harm. When the guardrails are there and they work, you know, Capitalism is a very good way of optimizing for just getting the same things done more efficiently. But it was good. And like in hindsight, I never met anyone Even on parties way over on the right in any country who thinks it was a terrible idea to ban child labor, for example.”
2023-04-13 · Lex Fridman Podcast · #371 – Max Tegmark: The Case for Halting AI Development · IDENTIFIED FROM THE TRANSCRIPT · source