YouSaid · the spoken record

Andrej Karpathy

lines on the record: 266
first: 2022-10-29
most recent: 2022-10-29
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Yeah, I think it's always been a bit of an arms race between sort of the attack and the defense. So the attack will get stronger, but the defense will get stronger as well, our ability to detect that”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, today, all this interaction is mostly on the level of HTML, CSS, and so on. That's done because of computational constraints. But I think ultimately everything is designed for human visual consumption. And so at the end of the day, there's all the additional information is in the layout of the webpage and what's next to you and what's a red background and all this kind of stuff and what it looks like visually. So I think that's the final frontier as we are taking in pixels and we're giving out keyboard, mouse commands. But I think it's impractical still today.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“But now what's happened is it is time to revisit that, and OpenAI is interested in this. Companies like Adept are interested in this and so on. And the idea is coming back because the interface is very powerful. But now you're not training an agent from scratch. You are taking the GPT as an initialization. So GPT is pre-trained on all of text. And it understands what's booking. It understands what's a submit. It understands quite a bit more. And so it already has those representations. They are very powerful. And that makes all of the training significantly more efficient and makes the problem tractable.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Just too many options, and it's too sparse of a reward signal. And you're starting from scratch at the time, and so you don't know how to read, you don't understand pictures, images, buttons, you don't understand what it means to make a booking.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Networks from scratch using reinforcement learning directly. It turns out that reinforcement learning is extremely inefficient way of training neural networks because you're taking all these actions and all these observations and you get some sparse rewards once in a while. So you do all this stuff based on all these inputs. And once in a while, you're like told you did a good thing. You did a bad thing. And it's just an extremely hard problem. You can't learn from that. You can burn a forest and you can sort of brute force through it. And we saw that, I think, with, you know, with Go and Doda and so on. And it does work. But it's extremely inefficient and not how you want to approach problems, practically speaking. And so that's the approach that at the time we also took to world of bits. We would have an agent initialize randomly. So with keyboard mash and mouse mash and try to make a booking. And it's just like revealed the insanity of that approach very quickly where you have to stumble by the correct booking in order to get a reward of you did it correctly and you're never going to stumble by it by chance at random.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“They're kind of like similar philosophy in some way, where the human, the world, the physical world is designed for the human form and the digital world is designed for the human form of seeing the screen and using keyboard and mouse. And so it's the universal interface that can basically command the digital infrastructure we've built up for ourselves. And so it feels like a very powerful interface to command and to build on top of. Now to your question as to what I learned from that, it's interesting because the world of bits was basically too early, I think, at OpenAI at the time. This is around 2015 or so. And the zeitgeist at that time was very different in AI from the zeitgeist today. At the time, everyone was super excited about reinforcement learning from scratch. This is the time of the Atari paper where neural networks were playing Atari games and beating humans in some cases, AlphaGo and so on. So everyone's very excited about training your”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. Well, it's the universal interface in the digital realm, I would say. And there's a universal interface in the physical realm, which in my mind is a humanoid form factor kind of thing. We can later talk about Optimus and so on. But I feel like there's”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“So basically you perceive the input of the screen pixels and basically the state of the computer is sort of visualized for human consumption in images of the web browser and stuff like that. And then you give the neural network the ability to press keyboards and use the mouse. And we're trying to get it to, for example, complete bookings and interact with user interfaces.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that's probably the final frontier for a lot of these models because, so as you mentioned, when I was at OpenAI, I was working on this project World of Bits. And basically it was the idea of giving neural networks access to a keyboard and a mouse.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so text by itself, I'm a little bit suspicious about. There's a ton of things we don't put in text in writing just because they're obvious to us about how the world works and the physics of it and that things fall. We don't put that stuff in text because why would you? We share that understanding. And so text is a communication medium between humans and it's not a all-encompassing medium of knowledge about the world. But as you pointed out, we do have video and we have images and we have audio. And so I think that definitely helps a lot. But we haven't trained models sufficiently across both across all of those modalities yet. So I think that's what a lot of people are interested in.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so I think the internet has a huge amount of data. I'm not sure if it's a complete enough set. I don't know that text is enough for having a sufficiently powerful AGI as an outcome.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Is doing some understanding in its weights. It understands, I think, a lot about the world, and it has to in order to predict the next word in a sequence.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so basically it gets a thousand words and is trying to predict 1,000 first. And in order to do that very, very well over the entire data set available on the internet, you actually have to basically kind of understand the context of what's going on in there. And it's a sufficiently hard problem that you, if you have a powerful enough computer like a transformer, you end up with interesting solutions. And you can ask it to do all kinds of things. It shows a lot of emergent properties like in-context learning. That was the big deal with GPT and the original paper when they published it, is that you can just sort of prompt it in various ways and ask it to do various things. And it will just kind of complete the sentence. But in the process of just completing the sentence, it's actually solving all kinds of really interesting problems that we care about.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Lungage modeling for a long time. So, really, what's new or interesting or exciting is just realizing that when you scale it up with powerful enough neural net transformer, you have all these emergent properties where basically what happens is if you have a large enough data set of text. You are in the task of predicting the next word. You are multitasking a huge amount of different kinds of problems. You are multitasking understanding of chemistry, physics, human nature. Lots of things are sort of clustered in that objective. It's a very simple objective, but actually you have to understand a lot about the world to make that prediction.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so language model just basically the rough idea is just predicting the next word in a sequence, roughly speaking. So there's a paper from, for example, Benjio and the team from 2003 where for the first time they were using a neural network to take, say, like three or five words and predict the next word. And they're doing this on much smaller data sets. And the neural net is not a transformer. It's a multi-erceptron. But it's the first time that a neural network has been applied in that setting. But even before neural networks, there were language models, except they were using Ngram models. So Ngram models are just count-based models. So if you try to take two words and predict a third one, you just count up how many times you've seen any two-word combinations and what came next. And what you predict as coming next is just what you've seen the most of in the training set. And so language modeling has been around for a long time. Neural networks have done.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Basically, the way GPT is trained, right, is you just download a massive amount of text data from the internet and you try to predict the next word in the sequence, roughly speaking, you're predicting will work chunks, but roughly speaking, that's it. And what's been really interesting to watch is basically it's a language model. Language models have actually existed for a very long time. There's papers on language modeling from 2003, even earlier.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“That kind of stuff. Definitely the zeitgeist today is just pushing, like basically right now the zeitgeist is do not touch the transformer, touch everything else. So, people are scaling up the data sets, making them much, much bigger. They're working on the evaluation, making the evaluation much, much bigger, and they're basically keeping the architecture unchanged. And that's the last five years of progress in AI, kind of.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Definitely looks like the transformers taking over AI, and you can feed basically arbitrary problems into it. And it's a general differentiable computer, and it's extremely powerful. And this conversions in AI has been really interesting to watch for me personally.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“In simultaneously optimizing for lots of properties of a desirable neural network architecture. And I think people have been trying to change it, but it's proven remarkably resilient. But I do think that there should be even better architectures potentially.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Transformer, typically they would be much shorter, say 20. So if 20 lines of code, then you can do something in them. And so think of during the optimization, basically what it looks like is first you optimize the first line of code and then the second line of code can kick in and the third line of code can kick in. And I kind of feel like because of the residual pathway and the dynamics of the optimization, you can sort of learn a very short algorithm that gets the approximate answer. But then the other layers can sort of kick in and start to create a contribution. And at the end of it, you're optimizing over an algorithm that is 20 lines of code, except these lines of code are very complex because this is an entire block of a transformer. You can do a lot in there. Well, it's really interesting is that this transformer architecture actually has been remarkably resilient. Basically, the transformer that came out in 2016 is the transformer you would use today, except you reshuffle some of the layer norms. The layer normalizations have been reshuffled to a pre-norm formulation. And so it's been remarkably stable, but there's a lot of bells and whistles that people have attached on it and try to improve it. I do think that basically it's a big...”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Right, think of it as basically a transformer is a series of blocks. And these blocks have attention and a little multilayer perceptron. And so you go off into a block and you come back to this residual pathway. And then you go off and you come back. And then you have a number of layers arranged sequentially. And so the way to look at it, I think, is because of the residual pathway in the backward path, the gradients sort of flow along it uninterrupted because addition distributes the gradient equally to all of its branches. So the gradient from the supervision at the top just floats directly to the first layer. And all the residual connections are arranged so that in the beginning during initialization, they contribute nothing to the residual pathway. So what it kind of looks like is imagine the transformer is kind of like a Python function, like a def. And you get to do various kinds of lines of code. Say you have 100 layers deep.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“They prefer lots of parallelism, so you don't want to do lots of sequential operations. You want to do a lot of operations serially. And the Transformer is designed with that in mind as well. And so it's designed for our hardware and it's designed to both be very expressive in a forward pass, but also very optimizable in the backward pass.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, exactly. Transformer is much more than just the attention component. It's got many pieces architectural that went into it. The residual connection of the way it's arranged, there's a multi-layer perceptron in there, the way it's stacked and so on. But basically, there's a message passing scheme where nodes get to look at each other, decide what's interesting, and then update each other. And so I think when you get to the details of it, I think it's a very expressive function so it can express lots of different types of algorithms in a forward pass. Not only that, but the way it's designed with the residual connections, layer normalizations, the soft max attention and everything, it's also optimizable. This is a really big deal because there's lots of computers that are powerful that you can't optimize or they're not easy to optimize using the techniques that we have, which is back propagation ingredient sent. These are first order methods, very simple optimizers, really. And so you also need it to be optimizable. And then lastly, you wanted to run efficiently on our hardware. Our hardware is a massive throughput machine like GPUs.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“You want to have a general purpose computer that you can train on arbitrary problems, like say the task of next word prediction or detecting if there's a cat in an image or something like that. And you want to train this computer, so you want to set its weights. And I think there's a number of design criteria that sort of overlap in the transformer simultaneously that made it very successful. And I think the authors were kind of deliberately trying to make this really powerful architecture. So basically it's very powerful in the forward path because it's able to express very general computation as sort of something that looks like message passing. You have nodes and they all store vectors. And these nodes get to basically look at each other. And it's each other's vectors and they get to communicate and basically nodes get to broadcast, hey, I'm looking for certain things. And then other nodes get to broadcast. Hey, these are the things I have. Those are the keys and the values.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Honestly, yeah, there is an element of me that honestly agrees with you and prefers it this way. If it was too grand, it would overpromise and then under deliver potentially. So you want to just meme your way to greatness.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“That it was going to have. Yeah, I'm not sure if the authors were aware of the impact that that paper would go on to have. Probably they weren't, but I think they were aware of some of the motivations and design decisions behind the transformer, and they chose not to, I think, expand on it in that way in a paper. And so I think they had an idea that there was more than just the surface of just like, oh, we're just doing translation and here's a better architecture. You're not just doing translation. This is like a really cool differentiable, optimizable, efficient computer that you've proposed. And maybe they didn't have all of that foresight, but I think it's really interesting.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, the one that I've been thinking about recently, the most probably is the Transformer architecture. So basically neural networks have a lot of architectures that were trendy, have come and gone for different sensory modalities like for vision, audio, text. You would process them with different looking neural nuts. And recently we've seen this convergence towards one architecture, the transformer. And you can feed it video or you can feed it images or speech or text and it just gobbles it up. And it's kind of like a general purpose computer that is also trainable and very efficient to run on our hardware. And so this paper came out in 2016, I want to say.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think it's unsettling. I think it's a deterministic system. I think that things that look random, like say the collapse of the wave function, et cetera, I think they're actually deterministic, just entanglement and so on. And some kind of a multiverse theory, something, something.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Trying to get energy from it. We're just kind of like these particles in the wave that I feel like is mostly deterministic and takes universe from some kind of a Big Bang to some kind of a superintelligent replicator, some kind of a stable point in the universe given these laws of physics.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Will behave in some very strange way to us because they're beyond they're playing the meta game. And the metagame is probably, say, like arranging quantum mechanical systems in some very weird ways to extract infinite energy, solve the digital expansion of pi to whatever amount they will build their own little fusion reactors or something crazy. Like they're doing something beyond comprehension and not understandable to us and actually brilliant under the hood.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it's very likely that these things, for example, like, say you have these AGIs, it's very likely that, for example, they will be completely inert. I like these kinds of sci-fi books sometimes where these things are just completely inert. They don't interact with anything. And I find that kind of beautiful because they probably figured out the metagame of the universe in some way potentially. They're doing something completely beyond our imagination. And they don't interact with simple chemical life forms. Like why would you do that? So I find those kinds of ideas compelling.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, no person will discover it, I think, by the way. I think it's going to have to be some kind of a super intelligent AGI of a third generation. We're building the first generation AGI, you know.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, we'll find some way to extract infinite energy. For example, when you train reinforcement learning agents in physical simulations and you ask them to say run quickly on a flat ground, they'll end up doing all kinds of weird things in part of that optimization, right? They'll get on their back leg and they'll slide across the floor. And it's because the optimization, the enforcement learning optimization on that agent has figured out a way to extract infinite energy from the friction forces and basically their poor implementation. And they found a way to generate infinite energy and just slide across the surface. And it's not what you expected. It's just a... Sort of like a perverse solution. And so maybe we can find something like that. Maybe we can be that little dog in this physical simulation.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it's possible that physics has exploits and we should be trying to find them, arranging some kind of a crazy quantum mechanical system that somehow gives you buffer overflow, somehow gives you rounding error in the floating point.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“The puzzle is basically like alerting the creator that we exist. Or maybe the puzzle is just to break out of the system and just stick it to the creator in some way. Basically, like if you're playing a video game, you can somehow find an exploit and find a way to execute on the host machine, any arbitrary code. For example, I believe someone got a game of Mario to play Pong just by exploiting it and then creating basically writing code and being able to execute arbitrary code in the game. And so maybe we should be, maybe that's the puzzle is that we should. Find a way to exploit it. So I think some of these synthetic AIs will eventually find the universe to be some kind of a puzzle and then solve it in some way. And that's kind of like the end game somehow.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“It is a really interesting thing to think about what the puzzle of the universe is. Did the creator of the universe give us a message? Like, for example, in the book, Contact Carl Sagan, there's a message for any civilization in digits, in the expansion of pi in base 11 eventually, which is kind of interesting thought. Maybe we're supposed to be giving a message to our creator. Maybe we're supposed to somehow create some kind of a quantum mechanical system that alerts them to our intelligent presence here. Because if you think about it from their perspective, it's just, say, like quantum field theory, massive, like cellular autonomaton-like thing. And like, how do you even notice that we exist? You might not even be able to pick us up in that simulation. And so how do you prove that you exist, that you're intelligent and that you're part of the universe?”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“We're stepping through an explosion, and we're living day to day, and it doesn't look like it. But it's actually, if you, I saw a very cool animation of Earth and life on Earth, and basically nothing happens for a long time. And then the last two seconds, like basically cities and everything, and the low Earth orbit just gets cluttered and just the whole thing happens in the last two seconds, and you're like, this is exploding. This is a statement explosion.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I don't know what determinating conditions are, but definitely there's a trend line of something. And we're part of that story. Where does it go? So we're famously described often as a biological bootloader for AIs. And that's because humans, I mean, we're an incredible biological system and we're capable of computation and love and so on. But we're extremely inefficient as well. Like we're talking to each other through audio. It's just kind of embarrassing, honestly, that we're manipulating like seven symbols, serially. We're using vocal cords. It's all happening over multiple seconds. It's just kind of embarrassing when you step down to the frequencies at which computers operate or are able to operate on. And so basically it does seem like synthetic intelligences are kind of like the next stage of development. And I don't know where it leads to. Like at some point, I suspect the universe is some kind of a puzzle. And these synthetic AIs will uncover that puzzle.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Definitely, there seems to be, yeah, you're sort of. Pretty incredible that these self replicating systems will basically arise from the dynamics and then they perpetuate themselves and become more complex and eventually become conscious and build a society. And I kind of feel like in some sensitive wave that kind of just like happens on any sufficiently well-ranged system like Earth. And so I kind of feel like there's a certain sense of inevitability in it. And it's really beautiful.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Maybe eventually they will. Currently, NPCs are really dumb, but once they're running GPTs, maybe they will be like, hey, this is really suspicious. What the hell”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm suspicious of this idea of like a deliberatemeremia. As you described it, sort of I don't see a divine intervention in some way in the historical record right now. I do feel like the story in these books like Nick Lane's books and so on sort of makes sense and it makes sense how life arose on earth uniquely. And yeah, I don't need to reach for a more exotic explanations right now.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Happens. I think it should be very interesting to scientists, other alien scientists, what happened here. And, you know, what we're seeing today is a snapshot. Basically, it's a result of a huge amount of computation. Over like billion years or something like that”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think you would need like a very good reason, I think, to destroy it. Why don't we destroy these ant farms and so on? It's because we're not actually really in direct competition with them right now. We do it accidentally and so on, but there's plenty of resources. And so why would you destroy something that is so interesting and precious”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think the question, it's really hard. I will say, for example, for us, we have lots of primitive life forms on Earth next to us. We have all kinds of ants and everything else, and we share space with them. And we are hesitant to impact on them, and we're trying to protect them by default because they are amazing interests. Interesting and special. And I don't know that you want to destroy that by default. And so I like complex dynamical systems that took a lot of time to evolve. I think I like to preserve it if I can afford to. And I'd like to think that the same would be true about the galactic resources and that they would think that we're kind of incredible, interesting story that took time took a few billion years to unravel and you don't want to just destroy it.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I'm suspicious basically of our ability to measure life, and I'm suspicious of the ability to just permeate all of space in the galaxy or across galaxies. And that's the only way that I can currently see a way around it.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Atoms and the little particles of dust are basically massive kinetic energy at those speeds. And so basically you need some kind of shielding. You have all the cosmic radiation. It's just like brutal out there. It's really hard. And so my thinking is maybe interstellar travel is just extremely hard.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Very suspicious of our ability to find these intelligences out there and to find these earth radio waves, for example, are terrible. Their power drops off as basically one over R square. So I remember reading that our current radio waves would not be the ones that we are broadcasting, would not be measurable by our devices today. Only like, was it like one-tenth of a light year away? Like not even basically tiny distance because you really need like a targeted transmission of massive power directed somewhere for this to be picked up on long distances. And so I just think that our ability to measure is not amazing. I think there's probably other civilizations out there. And then the big question is why don't they build van Neumann probes and why don't they interstellar travel across the entire galaxy? And my current answer is it's probably interstellar travel is like really hard. You have the interstellar medium if you want to move at closer speed of light. You're going to be encountering bullets along the way because even like tiny hydrogen.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“We don't have really good mechanisms for seeing this life. I mean, by what? So I'm not an expert just to preface this, but just from what I think”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I currently think that there's no major drop offs basically. And so there should be quite a lot of life. And basically where that brings me to then is the only way to reconcile the fact that we haven't found anyone and so on is that we just can't see them. We can't observe them.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“You have an active Earth and you have your alkaline vents, and you have lots of alkaline waters mixing with devotion, and you have your proton gradients, and you have little porous pockets of these alkaline vents that concentrate chemistry. And basically as he steps through all of these little pieces, you start to understand that actually this is not that crazy. You could see this happen on other systems. And he really takes you from just a geology to primitive life. And he makes it feel like it's actually pretty plausible. And also the origin of life didn't was actually fairly fast after formation of Earth. If I'm recording just a few hundred million years or something like that, after basically when it was possible, life actually arose. And so that makes me feel like that is not the constraint. That is not the limiting variable and that life should actually be fairly common. And then where the drop-offs are is very interesting to think about.”
2022-10-29 · Lex Fridman Podcast · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI · IDENTIFIED FROM THE TRANSCRIPT · source