YouSaid · the spoken record

George Hotz

lines on the record: 283
first: 2023-06-30
most recent: 2023-06-30
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“I mean, you know, a rocket and an ICVM, you have a rocket that can land anyway. Isn't that an ICBM? Well, you know, don't ask too many questions.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“And it will have a, I mean, this is true for the AIs too, right? There's no turning the AIs off. There's no button. You can't press it. Now, does Mark Zuckerberg have that button for Facebook.com?”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Sure? They have the technical power, but do they have the authority? Let's say Sundar Pashai made this his sole I came into Google tomorrow and said, I'm going to shut Google.com down. Think you keep this position too long”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“When I listen to your Sam Altman podcast, he talked about the button. Everyone who talks about AI talks about the button, the button to turn it off, right? Do we have a button to turn off Google? Is anybody in the world capable of shutting Google down?”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“See what I mean? Like, that's just like Mark Zuckerberg. This is Mark Zuckerberg reading that Paul Gramassing and being like, I'm going to show everyone how alive we are. I'm going to change the name.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Look, I would put a lot more money on Mark Zuck Because Mark Zuckerberg's alive. Like, this is all Paul Graham's essay. Startups are either alive or dead. Google's dead.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Some startups going to figure it out. I think if you ask me, Google's still the number one webpage, I think by the end of the decade Google won't be the number one webpage anymore.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“When I started comma, I said over and over again, I'm going to win self driving cars. I still believe that. I have never said I'm going to win search with the Tiny Corp, and I'm never going to say that because I won't.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, if you are writing an absolute basic React app with a button, it's not going to hallucinate chair. No, there's kind of ways to fix hallucination problem. I think Facebook is an interesting paper. It's called Atlas. And it's actually weird the way that we do language models right now where all of the information is in the weights. Human brains don't really like this. We have like a hippocampus and a memory system. So, why don't LLMs have a memory system? And there's people working on them. I think future LLMs are going to be smaller, but are going to run looping on themselves and are going to have retrieval systems. And the thing about using a retrieval system is you can side sources explicitly.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“I was at Twitter. I tried to use ChatGPT to ask some questions. Like, was the API for this? And it would just hallucinate. It would just give me completely made up API functions that sounded real.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I guess if I was really writing maybe today, if I wrote a lot of data parsing stuff. I mean, I don't play CTFs anymore, but if I still play CTFs, a lot of is just like you have to write a parser for this day. I wonder, or like advent of code. I wonder when the models are going to start to help with that kind of code. And they may. They may. And the models also may help you with speed. The model's very fast, but where the models won't, my programming speed is not at all limited by my typing speed. And in very few cases it is. Yes. If I'm writing some script to just like parse some weird data format. My programming speed is limited by my typing speed”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Is now 2700 lines and it can run llama and stable diffusion. And all of this stuff is in 2700 lines. Boilerplate and abstraction indirections and all these things are just bad code.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, it's Heid E's, the languages, the runtimes. It's everything. And programming is tool complete. So almost if codecs or copilot are helping you, that actually probably means that your framework or library is bad and there's too much boilerplate in it.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“So, think about the difference between a car from 1980 and a car from today. No difference, really. It's got a bunch of pedals, it's got a steering wheel. Great. Maybe now it has a few ADAS features, but it's pretty much the same car. You have no problem getting into a 1980 car and driving it. Take a programmer today who spent their whole life doing JavaScript and you put him in an Apple IIE prompt and you tell them about the line numbers in Basic. But how do I insert something between line 17 and 18? Oh, well.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Think it's like level two driving. I think driving is not tool complete and programming is. Meaning, you don't use the best possible tools to drive. You're not like. Cars have basically the same interface for the last 50 years. Computers have a radically different interface.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“That's the scariest kind of code. I spend 5% of time typing and 95% of time debugging. The last thing I want is close to correct code. I want a machine that can help me with the debugging, not with the typing.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“I disagree, actually I disagree. I think things like mu zero and alpha go are so much more impressive because these things are playing beyond the highest human level. Language models are writing middle school level essays and people are like, wow, it's a great essay. It's a great five paragraph essay about the causes of the Civil War.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“I think we dramatize everything. I think that you ask the milkman when he saw refrigerators, and they're going to have one of these in every home?”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“The poor milkman, the day he learned about refrigerators, he's like, I'm done. You tell me you can just keep the milk in your house? You don't need me to deliver it every day? I'm done.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“I'd like to go back to when calculators first came out or computers. And like I wasn't around, look, I'm 33 years old and to see how that affected. Like society”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“But no, no, no, you don't understand humans are created by God and machines are created by humans. Therefore, that'll be the last niche we have.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Humans are always going to define a niche for themselves. Like, well, you know, we're better than the machines because we can, you know, and like they tried creative for a bit, but no one believes that one anymore.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“The problem is if you go back to 1960 and you tell them that you have a machine that can play amazing chess, of course someone in 1960 will tell you that machine is intelligent. Someone in 2010 won't. What's changed, right? Today, we think that these machines that have language are intelligent, but I think in 20 years we're going to be like, yeah, but can it reproduce?”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Making brilliancies and chess, which feels a lot like thought, whatever new thing that AI can do everybody thinks is brilliant. And then like 20 years go by and they're like, well, yeah, but chess, that's like mechanical. Like adding, that's like mechanical”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Not a data Data centers are clouds. We don't have clouds. Data centers have air conditioners. We have fans. That makes it a compute cluster.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“40 kilowatts at idle, our data center dive crazy. We have 40 kilowatts just burning just when the computers are idle. Sorry, sorry, compute cluster.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Also, have a very fancy simulator. They're probably saying all the same things we are. They're probably saying we just need to optimize what is the reward. We get negative reward for disengagement. Everyone kind of knows this. It's just a question who can actually build and deploy the system.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Tesla is always one two years ahead of us. They've always been one to two years ahead of us, and they probably always will be because they're not doing anything wrong.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“They're just like stupid bugs. And also, we might just need more scale. We just massively expanded our compute cluster at comma. We now have about two people worth of compute, 40 peta flops.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Hard to say we haven't completely closed the loop yet, so we don't have anything built that truly looks like that architecture yet. We have prototypes and there's bugs. So we are a couple bug fixes away. Might take a year, might take 10.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Usually, there's some that are just I felt like driving, and those are always fine too, but they're just going to look like noise in the data.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“With a reward function, not asking is this close to the human policy, but asking would a human disengage if you did this behavior?”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, actually, our simulator is conditioned on the pose, so it's actually a simulator. You can put in like a state action pair and get out the next state. And then once you have a simulator, you can do RL in the simulator, and RL will get us that human policy.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“And then an RNN in the middle, right? You take an automoter, you compress the picture, you use an RNN, predict the next date, and these things were a laughably bad simulator. This is 2015 error machine learning technology. Today, we have VQVAE and Transformers. Building Drive GPT basically.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Solving means how do you build a model that outputs a human policy? Do you build a model that given a reasonable set of sensors outputs a human policy for driving? So, you have companies like Wayman Cruz, which are hand coding these things that are quasi human policies. Then you have Tesla, and maybe even to more of an extent, asking, okay, how do we just learn the human policy from data? The big thing that we're doing now, and we just put it out on Twitter. At the beginning of Published a paper called Learning a Driving Simulator. Way this thing worked was it's a it was an auto encoder”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Almost no one talks about FSD anymore, and even less people talk about OpenPilot. We've solved the problem. Like, we solved it years ago”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“To be able to leverage it in a way that it's completely generic, right? So there's a lot of, Xiaomi has a pretty good open source library for mobile GPUs called MACE, where they can generate, where they have these kernels, but they're all hand coded, right? So that's great if you're doing three by three confs. That's great if you're doing dense mat mols, but the minute you go off the beaten path a tiny bit, well, your performance is nothing.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“A Snapdragon 845. So, this is using the GPU. So, the GPU is an adreno GPU. There's like different things. There's a really good Microsoft paper that talks about mobile GPUs and why they're different from desktop GPUs. One of the big things is in a desktop GPU, you can use buffers on a mobile GPU image textures a lot faster.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“So I'm much more of a like. Build it the right way and worry about performance later. There's a bunch of things where I haven't even really dove into performance. The only place where Tiny Grad is competitive performance-wise right now is on Qualcomm GPUs. So, Tiny Grid is actually used in OpenPilot to run the model. So the driving model is Tiny”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“My goal is step one build an equally performance stack to PyTorch on NVIDIA and AMD. With way less lines. And then step two is okay, how do we make an accelerator? But you need step one. You have to first build the framework before you can build the accelerator.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Think that there's a spectrum. And like on one side, you have Mojo, and on the other side, you have like GGML. GGML is this like we're gonna run llama fast on Mac. And okay, we're gonna expand out to a little bit, but we're gonna basically go like depth first, right? Mojo is like, we're gonna go breath first. We're gonna go so wide that we're gonna make all of Python fast. Tiny grads We are going to make neural networks faster.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“No, it doesn't have a costume performance. But yeah, no, I think that it's that's what I mean about Tiny Gred's front end being cleaner.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“There's a really simple way to get these models into Tiny Grad, and you can just export them as Onyx, and then TinyGrad can run Onyx. So the ports that I did of Llama Stable Diffusion and now Whisper are more academic to teach me about the models, but they are cleaner than the PyTorch versions. You can read the code. I think the code is easier to read. It's less lines. There's just a few things about the way TinyGred writes things. Here's a complaint I have about PyTorch. NN.relU is a class. So when you create out, when you create an NN module, you'll put your nn reluse in init. And this makes no sense. RelU is completely stateless. Why should that be a class”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“No Look, I want to give you a great out of the box experience. I want you to lift this thing out. I want it to be like the Mac, you know.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“It is the I do not want it to be large according to UPS. I want it to be shippable as a normal package, but that's my constraint there.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“A key trick is to actually make it big. Ironically, it's called the tiny box. But if I can make it big, a lot of that noise is generated because of high pressure If you look at a one U server, a one U server has these super high pressure fans that like super deep and they're like gen engines versus if you have something that's big, well I can use a big, you know, you know they call them big ass fans, those ones that are like huge on the ceiling and they're completely silent.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“No, no, quiet because you want to put this thing in your house and you want it to coexist with you. If it's screaming, it's 60 dB, you don't want that in your house. You'll kick it out.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“And I'll tell you why it's six. So, AMD Epic processors have 128 lanes of PCI. I want to leave enough lanes for some drives. And I want to leave enough lanes for some networking.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, look, I mean, look, I know you can look at porn viewership numbers, right? A lot more men watch porn than women. Can ask why”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source
“I also think that men are probably more desperate and more likely to buy my product, so maybe they're a better target market.”
2023-06-30 · Lex Fridman Podcast · #387 – George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God · IDENTIFIED FROM THE TRANSCRIPT · source