YouSaid · the spoken record

David Patterson

lines on the record: 127
first: 2020-06-27
most recent: 2020-06-27
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“I can tell you the nine magic words that you need to say to your partner. To keep a good relationship. In the nine magic words, I was wrong. You were right. I love you. And you got to say all nine. You can't say I was wrong. You were right. You're a jerk. So freely acknowledging that you made a mistake, the other person was right, and that you loved them really gets over a lot of bumps in the road. So that's what I pass along.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“That's probably not the things that are afterwards you're going to care about. When I was, Just when I got the offer from Berkeley before I showed up, I read a book where they interviewed a lot of people in all works of life. And what I got out of that book was the people who felt good about what they did was the people who affected people as opposed to things that were more transitory. So I came into this job assuming that it wasn't going to be the papers. It was going to be relationships with the people over time that I would value. And that was a correct assessment, right? It's the people you work with, the people you can influence, the people you can help is the things that you feel good about towards into your career. It's not the stuff that's more transitory.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“And you know, this is a clich ⁇ people don't die wishing they'd spent more time in the office, right? As I reflect upon my career, there have been, you know, a half a dozen things say I've been proud of. A lot of them aren't papers or scientific results. Certainly my family, my wife, we've been married more than 50 years, kids and grandkids. That's really precious. Education things I've done. I'm very proud of, you know. books and courses. I did some help with underrepresented groups that was effective. So it was interesting just seeing what were the things I reflected. You know, I had hundreds of papers, but some of them were the papers, like the risk and rate stuff I'm proud of, but a lot of them were not those things. So people who are just spend their lives going after the dollars or going after all the papers in the world.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“A friend of mine, Randy Kat, said this. He said, you know, when it's time to sign off, the measure isn't the number of zeros in your bank account. It's the number of inches in the obituary in the New York Times as he said it. I think having.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Teams of faculty, large teams of faculty grad students pull all together for a common goal and often be successful at it. But I got both of those things from wrestling. Also, I think I heard this line about if people are in kind of collision, you know, sports with physical contact like wrestling or football and stuff like that, people are a little bit more assertive or something. And so I think that also comes through is, you know, I didn't shy away from the risk cisc debates. I enjoyed taking on the arguments and stuff like that. So it was, I'm really glad I did wrestling. I think it was really good for my self-image and I learned a lot from it. So I think that's, you know, sports done well. There's really lots of positives you can take about it of leadership, you know, how to form teams and how to. Be successful”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and the same things happen intellectually. If there's something going on where people are saying something that's not true, I feel it's my job to stand up just like I would in the street if there was something going on. Somebody attacking some woman or something, I'm not standing by and letting that get away. So I feel it's my job to stand up. So it's kind of ironically translates. The other things that turned out for both, I had really great college and high school coaches. And they believed even though wrestling's an individual sport that we would be more successful as a team if we bonded together do things that we would support each other rather than everybody in wrestling as a one-on-one. And you could be everybody's on their own. But he felt if we bonded as a team, we'd succeed. So I kind of picked up those skills of how to form successful teams and how to, from wrestling. And so I think most people would say one of my strengths is I can create.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Something going on in the streets that is bad physically, I'm not going to ignore it, right? I'm going to stand up and try and straighten that out.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“That we're still together, we're right now reaching out to other members of the wrestling team if we can get together for a reunion. But in terms of me, it was a huge difference. I was kind of the age cut off. It was December 1st. And so I was almost always the youngest person in my class. And I matured later on, you know, our family matured later. So I was almost always the smallest guy. So, you know, I took kind of nerdy courses, but I was wrestling. So wrestling was huge for my, you know, self-confidence in high school. And then, you know, I kind of got bigger at El Camino and then college. And so I had this kind of physical self-confidence. And it's translated into research self-confidence. And also kind of I've had this feeling even today in My 70s, you know, if something”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, that's in fact I wrestled at UCLA, but also at El Camino Community College. And just right now, we were in the state of California. We were state champions at El Camino. And in fact, I was talking to my mom, and I got into UCLA, but I decided to go to the community college, which is much harder to go to UCLA than the community college. And I asked, why did I make that decision? Because I thought it was because of my girlfriend. She said, well, it was the girlfriend, and you thought the wrestling team was really good. And we were right. We had a great wrestling team. We actually wrestled against UCLA at a tournament and we beat UCLA as a community college, which is just freshmen and sophomores. And part of the reason I brought this up is I'm going to go, they've invited me back at El Camino to give a lecture next month. And so my friend who was on the wrestling team.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Effective. I mean, in general, I think people are surprised. I've seen a lot of people who don't think they like teaching come give guest lectures or teach a course and get hooked on seeing the lights turn on, right? Is people, you can explain something to people that they don't understand and suddenly they get something, you know, that's not important and difficult. And just seeing the lights turn on is a real satisfaction there. There's any specific example of that. It's just the general joy of seeing them understand.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“I was going to bring up Feynman. I mean, he criticized the Institute of Advanced Studies. So the Institute of Advanced Studies was this thing that was created near Princeton where Einstein and all these smart people went. And when he was invited, he thought it was a terrible idea. This is a university. It was supposed to be heaven, right? A university without any teaching. But he thought it was a mistake. It's getting up in the classroom and having to explain things to students and having them ask questions. Like, well, why is that true? Makes you stop and think. So he thought, and I agree, I think that interaction between a great research university and having students with bright young men, asking hard questions the whole time is synergistic. And, you know, a university without teaching wouldn't be as vital and exciting a place. And I think it helps stimulate the research.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“That's the opposite, right? What's happened is they're highly correlated. The other way to think of it, if you're very successful people or maybe successful at everything they do, it's not an either or.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“But when you run into a really bad professor, a really bad teacher, I think the students think, well, this guy must be a great researcher because why else could he be here? So after 40 years at Berkeley, we had a retirement party and I got a chance to reflect. And I looked back at some things. That is not my experience. I saw a photograph of five of us in the department who won the Distinguished Teaching Award from campus. A very high honor, you know, when I've got one of those, one of the highest honors. So there are five of us on that picture. There's Manuel Blum. Richard Carp, me, Randy Cass, and John Osterhau, contemporaries of mine. I mentioned Randy already. All of us are in the National Academy of Engineering. We've all run the Distinguished Teaching Award. Blum Carp and I have touring awards”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, I think what happens is when you're a college student, you know there's this kind of tenure system in doing research. This model that is popular in America, I think America really made it happen, is we can attract these really great faculty to research universities because they get to do research as well as teach. And that, especially in fast moving fields, this means people are up to date and they're teaching those.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“This is the worst. Oh my God. So if you have a laptop and you're not backing it up on the cloud or something like that and your disk drive breaks, which it can do, you'll lose all that information and you just go crazy, right? So the importance of reliability for storage is tremendously higher than the importance of reliability for computation because of the consequences of it. So yes, so rate ideas are still very popular even with the switch of the technology. Although flash drives are more reliable. If you're not doing anything like backing it up to get some redundancy so they handle it, you're taking great risks.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, the people will still use great ideas because it's kind of what's different, a kind of interesting kind of psychologically if you think about it. People have always worried about the reliability of computing since the earliest days. But if we're talking about computation, if your computer makes a mistake in the computer says the computer has ways to check and say, oh, we screwed up, we made a mistake. What happens is that program that was running, you have to redo it, which is a hassle. For storage, if you've sent important information away and it loses that information, you go nuts.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Fraction of all the computers in the world are using semiconductor for storage, the flash drive, instead of being magnetic, well, their semiconductor writing of information very densely. And that's been a huge difference. So all the cell phones in the world use flash. Most of the laptops use flash. All the embedded devices use flash instead of storage. Still in the cloud magnetic disks are more economical than flash, but they use both in the cloud. So it's been a huge change in the storage industry. Switching from primarily disk to being primarily semiconductor.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, the big thing that happened, well, a couple of things that happened, what we did had a modest amount of storage as redundancy as people built bigger and bigger storage systems. They've added more redundancy so they could add more failures. And the biggest thing that happened in storage is for decades it was based on things physically spinning called hard disk drives where it used to turn on your computer and it would make a noise. What that noise was was the disk drives spinning and they were rotating at like 60 rpm and it's like if you remember the vinyl records, if you have ever seen those, that's what it looked like. And there was like a needle like on a vinyl record that was reading it. So the big drive change is switching that over to a simulator technology called flash. So within the last, I'd say about decade is increasing.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“IBM was the big pusher of these disk drives in the real world where the technical advantage get turned into a business advantage or not. It proved to be true. And so we thought we were sound technically and it was unclear whether the business side, but we kind of as academics, we believe that the technology should win and it did.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“I'd say I think I'm naturally an optimist, but I thought our ideas were right. I thought kind of like Morris Law, it seemed to me if you looked at the history of the disk drives, they went from washing machine-sized things and they were getting smaller and smaller. And the volumes were with the smaller disk drives because that's where the PCs were. So we thought that was a technological trend that disk drives the volume of disk drives was going to be getting smaller and smaller devices, which were true. They were the size of a, I don't know, eight inches diameter, then five inches, then three inches in diameter. And so that it made sense to figure out how to deal things with an array of disks. So I think it was one of those things where logically we think the technological forces were on our side that it made sense. So we expected DIT to catch on, but there was that same kind of business question.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Turns out if you put a modest number of extra discs in one of these arrays, it could actually not only be as faster and cheaper than one of these washing machine disks, it could be actually more reliable because you could have a couple of breaks even with these cheap disks, whereas one failure with the washing machine thing would knock it up.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Of these devices, each of which are kind of PC quality, so they're not as good as these IBM washing machines, IBM dominated the storage consistency. So reliability is going to be awful. And so when we calculated it out instead of it breaking on average once a year, it would break every two weeks. So we thought about the idea and said, well, we got to address the reliability. So we did it originally performance, but we had to do reliability. So the name redundant array of inexpensive disks is array of these disks, inexpensive life for PCs, but we have extra copies. So if one breaks, we won't lose all the information. We'll have enough redundancy that we could let some break and we can still preserve the information. So the name is an array of inexpensive disks. This is a collection of these PCs. And the R part of the name was the redundancy so they'd be reliable.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Next to his Mac. Gee, these are brand new small things. Before that, for the big computers, the disk would be the size of washing machines. And here's something, the size of the size of a book or so. I wonder what we could do with that. Well, Randy was involved in the fourth generation RISC project here at Berkeley in the 80s. So we figured out a way how to make the computation part, the processor part go a lot faster. But what about the storage part? Can we do something to make it faster? So we hit upon the idea of taking a lot of these disks developed for personal computers at Macintosh's and putting many of them together instead of one of these washing machine size things. And so we wrote the first draft of the paper and we'd have 40 of these little PC disks instead of one of these washing machine size things. And they would be much cheaper because they're made for PCs. And they could actually kind of be faster because there was 40 of them rather than one of them. And so we wrote a paper like that and sent it to one of our former Berkeley students at IBM. And he said, well, this is all great and good. But what about the reliability of these things? Now you have 40.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“It's right. So, this is work I did with my colleague Randy Katz and a star graduate student Garth Gibson. So we had just done the fourth generation RISC project. And Randy Katz, which had an early Apple Macintosh computer, at this time, everything was done with floppy disks, which are old technologies that could store things that didn't have much capacity. And you had to get any work done. You're always sticking your little floppy disk in and out because they didn't have much capacity. They started building what are called hard disk drives, which is magnetic material that can remember information storage for the Mac. And Randy asked the question when he saw this.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, there are a bunch of most people kind of use clock rate as a measure performance. It's not a perfect one. But if you've noticed, clock rates are more or less the same as they were five years ago. Computers are a little better than they haven't made zero progress, but they've made small progress. So there's some indications out there in their behavior, right? Nobody buys the next laptop because it's so much faster than the laptop from the past. For cell phones, I think I don't know why people buy new cell phones because a new one's announced. The cameras are better, but that's kind of domain specific, right? They're putting special purpose hardware to make the processing of images go much better. So that's the way they're doing it. They're not particularly, it's not that the ARM processor in there is twice as fast as much as they've added accelerators.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“It's a warning or call for arms that the world is changing right now, and a lot of people, a lot of computer science PhDs are unaware of that. So a way to try and get their attention is to say that Moore's Law is slowing down, and that's going to affect your assumptions. And we're trying to get the word out. And when companies like TSMC and Intel say, oh, no, no, no, Mars law is fine. Then people think, okay, I don't have to change my behavior. I'll just get the next servers. If they start doing measurements, they'll realize what's going on.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“But what's happening right now, talking to people who have working in national offices and stuff like that, a lot of the computer science community is unaware that this is going on That we are in an era that's going to need radical change at lower levels that could affect the whole software stack. If Intel, if you're using cloud stuff and the servers that you get next year are basically only a little bit faster than the servers you got this year, you need to know that. And we need to start innovating to start delivery on it. If you're counting on your software going to add a lot more features, assuming the computer's going to get faster, that's not true. So are you going to have to start making your software stack more efficient? Or are you going to have to start learning about machine learning? So it's kind of a...”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Whether it's marketing or it's, well, it could be affecting business, but it could also be affecting the imagination of engineers. If Intel employees actually believe that we're frozen in 2019, well, that's... That would be bad for Intel.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Like, wow, what an amazing prediction! It's so easy to understand the implications are amazing. And that's why it was so famous as a prediction. And this reinterpretation of what it meant and changing is revisionist history. And I'd be happy. And they're not claiming there's a new Morris law. They're not saying, by the way, instead of every two years, it's every three years. I don't think they've I don't think they want to say that. I think what's going to happen is the new technology commissions H1 is going to get a little bit slower. So it is slowing down. The improvements won't be as great. And that's why we need to do new things.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Many people industry have told me that's not true anymore, that basically in more recent technologies that got more complicated, the actual cost of transistor went up. So even the corollary might not be true, but certainly Moore's law, that was the beauty of Morris' law. It was a very simple, it's like equals mc squared, right?”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Is this like bond and politics? Does everybody get their own facts? Or do we have Moore's Law was a crisp, you know, a more... Carver Meade looked at his Morris conversations drawing on a log log scale, a straight line, and that's what the definition of Moore's law is. There's this other, what Intel did for a while, interestingly before Jim joined them, they said, oh no, Moore's law isn't the number of doubling, isn't really doubling transistors every two years. Moore's law is the cost of the individual transistor going down, cutting in half every two years. Now that's not what he said, but they reinterpreted it because they believed that the cost of transistors was continuing to drop even if they couldn't get twice as many ships.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Okay, I understand the perception problem when I say Morris law is stopped. Okay, so now I say Morris Law is slowing down. And I think Jim. Which is another way of, if it's predicting every two years and I say it's slowing down, then that's another way of saying it doesn't hold anymore. And I think Jim wouldn't disagree that it's slowing down because that sounds like things are still getting better and just not as fast, which is another way of saying Moore's Law isn't working anymore.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“That's what Moore's law is. There's the, I don't know, there's been this aura associated with Moore's law that they've enjoyed for 50 years about, look at the field we're in, we're doubling transistors every two years. What an amazing field, which is an amazing thing that they were able to pull off. But even as Gordon Moore said, you know, no exponential can last forever. It lasted for 50 years, which is amazing. And this is a huge impact on the industry because of these changes that we've been talking about. So he claims because he's trying to act, he claims, you know, Patterson says Moore's law is no more. And look at it, it's still going. And TSMC, they say it's no longer. But there's quantitative evidence that Moore's law is not continuing. But I say now to try and...”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Semiconductor improving, and Morris Law in many people's minds are the same thing. So when I say, and I'm factually correct, that Moore's law is no longer holds, we are not doubling transistors every year's years. The downside for a company like Intel is people think that means it stopped, that technology has no longer improved And so Jim is trying to counteract the impression that semiconductors are frozen in 2019 are never going to get better. So I never said that. All I said was. Moore's law is no more. And I'm”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“This is just marketing. So what Gordon Moore said is a quantitative prediction. We can check the facts, right? Which is doubling the number of transistors every two years. So we can look back at Intel for the last five years and ask him, let's look at DRAM chips six years ago. So that would be three two-year periods. So then our DRAM chips have eight times as many transistors as they did six years ago. We can look up Intel microprocessors six years ago. If Moore's law is continuing, it should have eight times as many transistors as six years ago. The answer in both those cases is The problem has been because Moore's law was kind of genuinely embraced by the semiconductor industry is they would make investments in semiconductor equipment to make Morris Law come true.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I mean, I think this is such a, you know, given that we've had Moore's law. I just feel comfortable trying to do projects that are thinking about the next decade. I admire people who are trying to do things that are 30 years out, but it's such a fast-moving field. I just don't know how to, I'm not good enough to figure out what's the problem's going to be in 30 years. 10 years is hard enough for me.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Are expecting quantum computing to deliver in the next few years. It's probably further off. I'd be happy to be wrong. It'd be great if quantum computing is going to commercially viable, but it will be a set of applications. It's not a general purpose computation. So it's going to do some amazing things, but there'll be a lot of things that probably, you know, the old-fashioned computers are going to keep doing better for quite a while.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Two reports to me say probably it'll be 2030 before quantum computing is something that could happen. And when it does happen, this is going to be big science stuff. This is micro Kelvin, almost absolute zero things that if they vibrate, if truck goes by, it won't work, right? So this will be in data center stuff. We're not going to have a quantum cell phone. And it's probably a 2030 kind of thing. So I'm happy that our people are working on it, but just, you know, it's hard with all the news about it not to think that it's right around the corner. And that's why we need to do something as Moore's Law is slowing down to provide the computing, keep computing getting better for this next decade. And we shouldn't be betting on quantum computing.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“You know, I actually quantum computing idea has been around for a while, and I actually thought, well, sure hope I retire before I have to start teaching this. I would say because I talk about, give these talks about the slowing of Moore's Law and we need to change by doing domain-specific accelerators. Common questions say what about quantum computing. The reason that comes up, it's in the news all the time. So I think to keep in, the third thing to keep in mind is quantum computing is not right around the corner. There have been two national reports, one by the National Academies of Engineering, another by the computing consortium where they did a frank assessment of quantum computing and both of those reports said, you know, as far as we can tell, before you get error-corrected quantum computing, it's a decade away. So I think of it like nuclear fusion, right? There have been people who've been excited about nuclear fusion a long time. If we ever get nuclear fusion, it's going to be fantastic for the world. I'm glad people are working on it. But it's not right around the corner.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“You can just believe you're the best in the world. I think the people. I think they weren't purposely misleading. I think if you don't have benchmarks, I mean, how do you know you could have your intuition is kind of like the way we used to do computer architecture. Your intuition is that this is the right instruction set to do this job. I believe in my experience, my hunch is that's true. We had to get and make things more quantitative to make progress. And so I just don't know how in fields that don't have benchmarks, I don't understand how they figure out how they're making progress.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“And Dave Martin told me they don't have benchmarks. Everybody has their own vision algorithm and the way here's my image. Look at how well I do. And everybody had their own image. So David Martin back when he did his dissertation figured out a way to do benchmarks. He had a bunch of graduate students identify images and then ran benchmarks to see which algorithms run well. And that was as far as I know kind of the first time people did benchmarks in computer vision which was predated all the things that eventually led to ImageNet and stuff like that. But then the vision community got religion. And then once we got as far as image net, then that let the guys in Toronto be able to win the ImageNet competition and then that changed the whole world.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“It is incredible. Historically, it wasn't always that way. I had a graduate student working with me, David Martin. So in computer, in some fields benchmarking is been around forever. So computer architecture, databases, maybe operating systems. benchmarks are the way you measure progress. But he was working with me and then started working with Gender Malik and Jender Malik in computer vision space. I guess you've interviewed.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“20 years ago, it was a dirty word at Microsoft, and today Microsoft is one of the big proponents of open source software, kind of that's the standard way most software gets built, which really shows off your engineers because you can see, if you look at the source code, you can see who are making the commits, who's making the improvements, who are the engineers at all these companies who are really great programmers and engineers and making really solid contributions, which enhances their reputations and the reputation of the companies.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“I think what I was trying to understand that, you know, what's changed from the 80s in this era? I think because of things like social networking, Twitter, and stuff like that, if you put up bullshit stuff, right, that's just purposely misleading, you can get a violent reaction in social media pointing out the flaws in your arguments. And so from a marketing perspective, you have to be careful today that you didn't have to be careful that there'll be people who put out the flaws. You can get the word out about the flaws and what you're saying much more easily today than in the past. It used to be easier to get away with it. And the other thing that's been happening in terms of sharing off engineers is just on the software side, people have largely embraced open source software. It was.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“In 50 companies involved, so pretty much MLPERF is the way you measure machine learning performance. And it didn't exist even two years ago.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“Thing would happen with ML Perf, you know, companies like NVIDIA were maybe worried that it was some kind of trap, but eventually we all got together to create a set of benchmarks and do the right thing, right? And we agree on the results. And so we can see whether TPUs or GPUs or CPUs are really faster and how much the faster. And I think from an engineer's perspective, as long as the results are fair, you can live with it. Okay, you kind of tip your hat to your colleagues at another institution. Boy, they did a better job than this. What you hate is if it's false, right? They're making claims and it's just marketing bullshit and that's affecting sales. So from an engineer's perspective, as long as it's a fair comparison and we don't come in first place, that's too bad, but it's fair. So we wanted to create that environment for MLPRF. And so now there's 10 companies, I mean, 10 universities.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“But one of the interesting metrics is what we thought it was. I was involved in the start that Peter Matson is leading there from Google. Google got it off the ground, but we had to reach out to competitors and say, there's no benchmarks here. We think this is bad for the field. It'll be much better if we look at examples like in the risk days, there was an effort to create a for the people in the risk community got together. Competitors got together were building risk microprocessors to agree on a set of benchmarks that were called spec. And that was good for the industry. It's rather before the different risk architectures were arguing, well, you can believe my performance others, but those other guys are liars. And that didn't do any good. So we agreed on a set of benchmarks. And then we could figure out who was faster between the various risk architectures. But it was a little bit faster. But that grew the market rather than, you know, people were afraid to buy anything. So we argued this.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“And the ones who did invest in software stack did, and we're seeing kind of in computer architecture, this is what happens. You have these arguments about risk versus sisk. People spend billions of dollars in the marketplace to see who wins. It's not a perfect comparison, but it kind of sorts things out. And we're seeing companies go out of business and then companies like there's a company in Israel called Habana, they came up with machine learning accelerators. They had good MLPerf scores. Intel had acquired a company earlier called Nirvana a couple years ago. They didn't reveal the MLP perf scores, which was suspicious. But a month ago, Intel announced that they're canceling the Nirvana product line, and they've bought Habana for $2 billion, and Intel is going to be shipping Habana chips, which have hardware and software and run the MLP programs pretty well. And that's going to be their product line of the”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“And so there'll be both the compiler piece and the hardware piece underneath it. So as you kind of, the fatal flaw for hardware people is to create really great hardware, but not have brought along the compilers. And what we're seeing right now in the marketplace, because of this enthusiasm around hardware for machine learning, is getting probably billions of dollars invested in startup companies. We're seeing startup companies go belly up because they focus on the hardware but didn't bring the software stack along. We talked about benchmarks earlier. So I participated in machine learning, didn't really have a set of benchmarks. I think just two years ago, they didn't have a set of benchmarks. And we've created something called MLPERF, which is machine learning benchmark suite. And pretty much the companies who didn't invest in the software stack couldn't run ML Perf very well.”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source
“It's going to be, I mean, if you think about the old wrist cyst debate, it was both software hardware. It was the compilers improving as well as the architecture improving. And that's likely to be the way things are now. With machine learning, they're using domain-specific languages, languages like TensorFlow and PyTorch are very popular with machine learning people. Those are raising the level of abstraction, it's easier for people to write machine learning in these domain-specific languages like PyTorch in...”
2020-06-27 · Lex Fridman Podcast · #104 – David Patterson: Computer Architecture and Data Storage · IDENTIFIED FROM THE TRANSCRIPT · source