YouSaid · the spoken record

Dylan Patel

lines on the record: 170
first: 2024-10-02
most recent: 2024-10-02
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“It would be very easy to just like shut down. I think the largest aluminum mill in the world is there, and it's like north of five gigawatts alone. It's like, oh, could we tell if they stopped making aluminum there and instead started making AIs there or making AI there? Like, I don't know if we could tell, right? Because they could also just easily spawn like 10 other aluminum mills, make up for the production, and be fine, right? So there's many ways for them to hide compute as well.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“Actually, I think the other thing is, could we notice it? I don't think so, because the amount of factories that are being spun up, the amount of other... Construction, manufacturing, etc., that's being built. A gigawatt is actually like a drop in the bucket, right? Like a gigawatt is not a lot of power. 10 gigawatts is not an absurd amount of power, right? It's okay. Yes, it's like hundreds of thousands of homes, right? Yeah, millions of people, but it's like you got 1.4 billion people. You got like most of the world's like extremely energy intensive, like refining and like, you know, rare earth refining and all these manufacturing industries are here. It would be very easy to hide it.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“Think between not just the dam itself, but like also all of the coal, there's some nuclear reactors there, I believe as well between all of and like renewables like solar and wind, between all of that in that region, there is an absurd amount of concentrated power that could be built. I don't think it's like, I'm not saying it's like one button, but it's like, hey, within X mile radius, right? It's more of like the correct way to frame it. And that's how the labs are also framing it, right? In the US.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“But I mean, on the flip side, right? Like, there is obviously immense gains from being centralized because of the scaling loss, right? And then the flip side is compute efficiency is obviously going to be hurt because you can't experiment and have different people lead and try their efforts as much if you're less centralized or more centralized. So it's like there is a balancing act there.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“The question is, does that slow down your research? Do you like crushed people deep seek who are clearly not being influenced by the government and put some idiot?”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't know if it's like that. Is that a Chinese accent, by the way? Just to be clear, John is East Asian. He is Chinese. I am of East Asian Half Taiwanese, half Chinese. Right, that is right, but like, I think, I don't know if that's as simple as that to like. Because training systems are like fire, like they're watergated, firewalled, what is it called? Not firewalled. I don't know. There's a word for that.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“Which we're talking about gigawatt data centers are coming over 2627 in the year or 26 in the US or 27, right? Know, sort of, this is an absurd scale relatively, right? We don't have gigawatt data centers ready, but like China could just build it in six months, I think, around the Three Gorgeous Dam or many other places, right? Because they have the ability to do the substations, they have the power generation capabilities. Everything can be like done like a flip of a switch, but they haven't done it yet. And then they can centralize the chips like crazy, right now, oh, oh, a million chips that NVIDIA is shipping in Q3 and Q4, the H20 Let's just put them all in this one data center. They just haven't had that centralization effort. But what you can”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“XYZ company is now in charge, and every GPU goes to one place. And then you don't have the same issues as the U.S., right? In the US, we have a big problem with like being able to build big enough data centers, being able to build substations and transformers and all this that are large enough in a dense area. China has no issue with that at all because their supply chain adds like as much power as half of Europe every year, right? Or some absurd statistics, right? So they're building transformer substations, they're building new power plants constantly. So they have no problem with like getting power density. And you go look at Bitcoin mining, right? Around the Three Gorges Dam. At one point, at least, there was like 10 gigawatts of like Bitcoin mining estimated, right?”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“No company has had the centralization push to have a cluster that large and train on it yet, at least publicly well known. And the best models seem to be from a company that has got like 10,000 GPUs, right? Or 16,000 GPUs, right? So it's not quite as centralized as the US companies are. And the U.S. companies are quite decentralized. If you're Xi Jinping and your scale pill, do you just say...”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“You have sanctions on how many NVIDIA GPUs you can get in. Now, they're still north of a million a year, right? Even post October last year sanctions. We still have more than a million H20s and other Hopper GPUs getting in through other means. But legally, like H20s. And then on top of that, you have... Your domestic chips, right? But that's less than a million chips. So then when you look at it, it's like, oh, well, we're still talking about a million chips. The scale of data centers people are training on today slash over the next six months is 100,000 GPUs, right? OpenAI, XAI, right? These are like quite well documented and others. But in China, they have no individual system of that scale yet, right? So then the question is like, how do we get there?”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“I think there is a true question of how decentralized do you go versus centralized, right? And if you look in the US, right, as far as labs and such, OpenAI, XAI, anthropic, and then Microsoft having their own effort, Anthropic having their own efforts despite having their partner, and then Meta. And you go down the list, it's like there is quite a decentralization and then all the startups interesting startups that are out there doing stuff. There's quite a decentralization of efforts. Today in China, it is still quite decentralized, right? It's not like Alibaba, Baidu, you are the champions, right? You have like deep seek, like, who the hell are you? Does government even support you? Like, doing amazing stuff, right? If you are Xi Jingping and scale pilled, you must now centralize the compute resources, right?”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“Not with Jensen Jesus. No, let's just color around a hopper, Blackwell, and margins. It's quite boring stuff for most people. I think it's interesting though.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“They don't even know about the end chip a lot of times. It's like, oh, I am an etch engineer, and all I focus on is how hydrogen fluoride etches this, right? And that's all I know. And like if I do it at different pressures, if I do it at different temperatures, if I do it with a slightly different recipe of chemicals, it changes everything.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“It's yield, its performance, its power, it's not just a one, it's not just better or worse, right? It's a multivariable search space.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“You're basically multivariable problem that each, every single tool sequentially you're processing the whole thing, you turn up knobs up and down on every single tool. You can increase the pressure on this one specific deposition tool there.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“And so Samsung at one point was better than TSMC. And then, yeah, he goes to Smick, and Smick is now better than, or not better, but they caught up rapidly as well after.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“I think you should tell that story actually the TSMC guy that went to Samsung and Smic and all that. I think you should tell that story.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“Them and give them a much better life, and they have right a lot of SMIC's employees are TSMC, you know, Taiwanese nationals, right? A lot of the really good ones, high up ones especially, right? And then you go up like the next layers of the stack, and it's like, I think, yeah, of course there's tons of model secrets, but then how many of those model secrets do you not already have and you just haven't deployed or implemented, you know, organized, right? That's the one thing I would say is like China just hasn't, they clearly are still not scale-pilled in my view.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I guess you have to stack every layer, right? And I think the beautiful thing about AI is because it's growing so freaking fast, every layer is being stressed. Some incredible degree. Of course, China has been. Hacking ASML for over five years, and ASML is kind of like, oh, it's fine. The Dutch government's really pissed off, but it's fine, right? I think it's, they already have those files, right? In my view. It's just a very difficult thing to build, right? I think the same applies for fab recipes, right? They can poach Taiwanese nationals very not that difficult, right? Because TSMC employees do not make absurd amounts of money. You can just”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source
“It's unfortunate my parents decided between my older brother and me to give me a white name. I could have been Dwark Ashton. How amazing it would have been if we had the same name? Butterfly effect at all that would probably have all turned out the same way, but like maybe.”
2024-10-02 · Dwarkesh Podcast · @Asianometry & Dylan Patel — How the semiconductor industry actually works · IDENTIFIED FROM THE TRANSCRIPT · source