YouSaid · the spoken record

Jeremy Howard

lines on the record: 135
first: 2019-08-27
most recent: 2019-08-27
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Zach Cahane over in Boston, Cambridge has a number of students now who are data science experts, deep learning experts, and actual medical doctors. Quite a few doctors have completed our fast AI course now and are publishing papers and creating journal reading groups in the American Council of Radiology. And like it's just starting to happen. But it's going to be a long process. The regulators have to learn how to regulate this. They have to build. Guidelines and then the lawyers at hospitals have to. Sometimes it makes sense for data to Looked at in raw form in large quantities in order to create world-changing results.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Oh, there's a lot of reasons. I mean, one is it's pretty new. I only started Analytic in like 2014, and before that, like. It's hard to express to what degree the medical world was not aware of the opportunities here. So I went to RSNA, which is the world's largest radiology conference, and I told everybody I could, you know, like I'm doing this thing with deep learning, please come and check it out. And no one had any idea what I was talking about, no one had any interest in it. So we've come from absolute zero, which is hard. And then the whole regulatory framework education system, everything is just set up to think of doctoring in a very different way. So today there is a small number of people who are Deep learning practitioners and doctors at the same time. And we're starting to see the first ones come out of their PhD program.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, that's what the triage would do, right? So the triage would say, okay, 99%. Sure, there's nothing here. So that can be done on device, and they can just say, okay, go home. So the experts are being used to look at the stuff which has some chance it's worth looking at, which most things is not. It's fine.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Don't see the point of even thinking about that because we have such a shortage of people. Why would we not want to find a way not to use them We have people, so the idea of even from an economic point of view, if you can make them 10x more productive, getting rid of the person doesn't impact your unit economics at all, and it totally ignores the fact that there are things people do better than machines. So it's just to me that's not a useful way of framing the problem.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“But actually, in India, for example, and China, almost no x rays are read by anybody, by any trained professional because they don't have enough. So if instead we had an algorithm that could take the most likely high risk 5% and say triage basically say, okay, someone needs to look at this, it would massively change the kind of way that what's possible with medicine in the developing world. And remember, they have increasingly they have money. The developing world, they're not the poor world. They're developing world. So they have the money. So they're building the hospitals. They're getting the diagnostic equipment, but there's no way for a very long time will they be able to have the expertise.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Can get very high quality assessment of whether the patient might be at risk and tell, you know, okay, we'll send them off to a hospital. So for example, in Africa, outside of South Africa, there's only five pediatric radiologists for the entire continent, so most countries don't have any. So if your kid is sick and they need something diagnosed through medical imaging, the person, even if you're able to get medical imaging done, the person that looks at it will be Know a nurse at best.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Not much happening today in terms of stuff that's actually out there. It's very early, but in terms of the opportunity, it's to take markets like India and China and Indonesia, which have big populations, Africa, small numbers of doctors. And provide diagnostic, particularly treatment planning and triage kind of on device so that if you do a test for malaria or tuberculosis or whatever, you immediately get something that even a healthcare worker that's had a month of training.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“So I guess the founding story is. Heavily tied to my previous startup, which is a company called Inletic, which was the first company to focus on deep learning for medicine. And I created that because I saw there was a huge opportunity to, there's about a 10x shortage of the number of doctors in the world, in the developing world that we need. Expected it would take about 300 years to train enough doctors to meet that gap, but I guess that maybe if we used deep learning for some of the analytics, we could maybe make it so you don't need as highly trained doctors. Diagnosis.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“It's even worse. So, TPUs, Google actually made an explicit decision to make them almost entirely unprogrammable because they felt there was too much IP in there and if they gave people direct access to program them, people would learn their secrets. So, you can't actually directly program the memory in a TPU. You can't even directly like Create code that runs on and that you look at on the machine that has the GPU, it all goes through a virtual machine. So all you can really do is kind of cookie cutter thing of like Plug in to high level stuff together, which is just super tedious and annoying and totally unnecessary”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Unfortunately, at the moment it does, but one of the nice things about MLIR, if AMD ever gets their act together, which they probably won't, is that they or others could write MLIR backends for other GPUs or other tensor computation devices. Which today there are increasing number like GraphCore or Vertex AI or whatever. Being able to target lots of backends would be another benefit of this, and the market really needs competition because at the moment Nvidia is massively overcharging for their... Kind of enterprise class cards because there is no Serious competition because nobody else is doing the software properly.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“So that's like, so these other things are kind of sitting on top of that kind of research, and MLIR is pulling a lot of those best practices together. And now we're starting to see work done on making all of that directly accessible through Swift so that I could use Swift to kind of write those domain-specific languages and hopefully we'll get then Swift Courage written in a very expressive and concise way that looks a bit like J in APL and then Swift layers on top of that and then a Swift UI on top of that. And it'll be so nice if we can get to that point.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Tensor computations. These are the kinds of things we do generally on the GPU for deep learning, and then have a compiler which can optimize that tensor computation. A lot of this work is actually sitting on top of a project called Halide, which is a mind-blowing project where they came up with such a domain specific language. In fact, two, one domain-specific language for expressing this is what my tensor computation is. And another domain-specific language for expressing, this is the kind of the way I want you to structure the compilation of that, like do it block by block and do these bits in parallel. They were able to show how you can compress the amount of code by 10x compared to optimized GPU code and get the same performance.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“And that's just because if you have to deal with the fact that I've got 10,000 threads and I have to synchronize between them all and I have to put my thing into grid blocks and think about warps and all this stuff, it's just so much boilerplate that to do that well you have to be a specialist at that and it's going to be a year's work to Optimize that algorithm in that way, but with things like tensor comprehensions and tile and MLIR and TVM, there's all these various projects which are all about saying, Let's let people create domain specific languages for”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Mean no one's at fault, just nobody's got around to it yet, or it's just hard, right? And I mean, part of the fault is that we ignored that whole. APL kind of direction. Nearly everybody did for 60 years, 50 years Recently, people have been starting to Reinvent pieces of that and kind of create some interesting new directions in the compiler technology. So the place where that's Particularly happening right now is something called MLIR, which is something that, again, Chris Latner, the SWIFT guy, is leading. And because it's actually not going to be swift on its own that solves this problem. Because the problem is that currently writing an acceptably fast GPU program is too complicated regardless of what language you use.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“So, like, there's just big gaps in what people actually research on, what people actually implement because of the programming language problem.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Huge role. Yeah, so currently Python has a big gap in terms of our ability to innovate, particularly around recurrent neural networks and natural language processing because it's so slow. The actual loop where we actually loop through words, we have to do that whole thing in CUDA C. So we actually can't innovate with the kernel, the heart of that most important algorithm. It's just a huge problem, and this happens all over the place. So we hit research limitations. Another example, convolutional neural networks, which are actually the most popular architecture for lots of things, maybe most things in deep learning. We almost certainly should be using sparse convolutional neural networks. only like two people are because to do it you have to rewrite all of that co-ta level stuff and Yeah, just researchers and practitioners”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“It's everything. I mean, in the end, I want to be productive as a. Practitioner. So that means that. So, like at the moment, our understanding of deep learning is incredibly primitive. There's very little we understand. Most things don't work very well, even though it works better than anything else out there. So many opportunities to make it better. So you look at any domain area like, I don't know, speech recognition with deep learning or natural language processing classification with deep learning or whatever. Every time I look at an area where deep learning, I always see like, oh, it's terrible. There's lots and lots of obviously stupid ways to do things that need to be fixed. So then I want to be able to jump in there and quickly. Experiment and make them better.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“I hope Swift is successful. Because the goal is swift the way Chris Latner describes it is to be infinitely hackable. And that's what I want. I want something where me and the people I do research with and my students can look at and change everything from top to bottom. There's nothing mysterious and magical and inaccessible. Unfortunately with Python, it's the opposite of that because Python's so slow, it's extremely unhackable. You get to a point where it's like, okay, from here on down at C, so your debugger doesn't work in the same way, your profiler doesn't work in the same way, your build system doesn't work in the same way. It's really not very hackable.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I mean, Perl did dominate. It was. Everything everywhere, but then the guy that ran Pearl, Lurry Wall kind of Didn't put the time in. Anymore. No project can be successful if there isn't particularly one that started with a strong leader that loses that strong leadership. So then Python has kind of replaced it. Python is a lot less elegant language in nearly every way, but it has the data science libraries and a lot of them are pretty great. So, I kind of Because it's the best we have, but it's definitely not.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, you know, it's just much more pragmatic. The end, you kind of have to end up where The libraries are, you know, like, because for me, my focus is on productivity. I just want to get stuff done and solve problems. So Perl was great. I created an email company called FastMail, and Perl was great because back in the late 90s, early 2000s, it just had a lot of stuff it could do. I still had to write my own monitoring system and my own web framework, my own whatever, because none of that stuff existed, but it was a super flexible language to do that in.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“It's pretty heavily focused on computation. I mean, so much of programming is data processing by definition, and so there's a lot of things you can do with it. But yeah, there's not much work being done on making like... User interface toolkits or whatever. I mean, there's some, but they're not great.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Interesting. The APL created two main branches, K and J. J is this kind of open source niche community of crazy enthusiasts like me. And then the other path K was fascinating. It's an astonishingly expensive programming language which many of the world's most ludicrously rich hedge funds use. So the entire K machine is so small it sits inside level three cache on your CPU and it easily wins every benchmark I've ever seen in terms of data processing speed but you don't come across it very much because it's like $100,000 per CPU to run it But it's like this, this, this, this path of programming languages is just so much, I don't know, so much more powerful in every way than the ones that almost anybody uses every day.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Array oriented means that you generally don't use any loops, but the whole thing is done with kind of an extreme version of broadcasting, if you're familiar with that numpy slash Python concept. You do a lot with one line of code. It looks a lot like math notation. Highly compact The idea is that you can because you can do so much with one line of code, a single screen of code is very unlikely to, you very rarely need more than that to express your program. And so you can kind of keep it all in your head and you can kind of clearly communicate it.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Math theory paper, not a programming paper, it was called Notation as a Tool for Thought, and it was the development of a new type of math notation. And the idea is that this math notation would much more flexible, expressive, and also well-defined than traditional math notation, which is none of those things. Math notation is awful. And so he actually turned that into a programming language. because this was the early 50s, late 50s, all the names were available. So he caught his language, a programming language or APL. So APL is a implementation of notation as a tool for thought by which he means math notation. And Ken and his son went on to do many things, but eventually they actually produced a new language that was built on top of all the learnings of APL. And that was called J. And J is the most expressive composable”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Okay, so At all surprising you're not familiar with it because it's not well known, but it's actually one of the main Families of programming languages going back to the late 50s, early 60s. Was a couple of major directions. One was the kind of lambda calculus Alonso church direction, which I guess kind of Lisp and And whatever, which has a history going back to the early days of computing. A second was the kind of Imperative algo simula going on to C, C++, so forth. There was a third which are called array-oriented languages, which started with a paper by a guy called Ken Iverson, which was actually a”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“At least programming environments again. Pascal's not a nice language. If you wanted to know specifically about what languages I like, I would definitely pick J as being an amazingly wonderful language.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, but a compiled fast version. Sure, there's anything quite like it anymore if you took like C sharp or Java and got rid of the virtual machine and I feel like it's where Swift could get to with the new Swift UI and the cross-platform development going on. Like that's one of my Is that we'll hopefully get back to where Delphi was. There is actually a free Pascal project nowadays called Lazarus, which is also attempting to kind of recreate Delphi. They're making good progress.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Always been interested in doing useful things for myself and for others, which generally means getting some data and doing something with it and putting it out there again. So that's been my. Interest throughout, so I also did a lot of stuff with AppleScript back in the early days. So it's kind of nice being able to The computer and computers to talk to each other and to do things for you. I think the programming language I most loved then would have been Delphi, which was object Pascal. Created by Anders Halzberg, who previously did Turbo Pascal and then went on to create.NET, and then went on to create TypeScript. Delphi was amazing because it was like a compiled, fast language that was as easy to use as Visual Basic”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Tie together, and it's just a lot more awkward than it should be. There are people that are trying to make it easier so that in particular I think of F sharp, you know, Don Syme, who him and his team have done a great job of Making something like a database appear in the type system, so you actually get tab completion for fields and tables and stuff like that. Anyway, so that was kind of that whole VBA office thing, I guess, was a starting point, which I still miss. And I got into standard Visual Basic, which”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Connection between So very close. So access kind of was the relational database equivalent, if you like. So people still do a lot of that stuff that should be an access in Excel because they know it. Excel's great as well. But it's just not as Rich a programming model as VBA combined with a relational database. And so I've always loved relational databases, but today Programming on top of relational databases is just a lot more of a headache. You generally either need to kind of need something that connects, that runs some kind of database server unless you use SQLite, which has its own issues. You kind of often, if you want to get a nice programming model, you'll need to create an ORM on top and then, I don't know, there's all these pieces.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“And that kind of withered, but basically it lets you in a totally graphical way create tables and relationships and queries and tie them to forms and set up event handlers and calculations. And it was very complete, powerful system designed for not massive scalable things, but for like... Useful little applications that I loved.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“My favorite programming environment. Most certainly was Microsoft Access back in the earliest days. So that was official basic for applications, which is not a good programming language, but the programming environment is fantastic. It's like the ability to create. User interfaces and tie data and actions to them and create reports and all that as I've never seen anything as good. There's things nowadays like Airtable which are like Small subsets of that which people love for good reason, but unfortunately nobody's ever Achieved anything like that.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“It's not where I wish it was. Various reasons couldn't really keep it going, particularly because I had a lot of problems with RSI with my fingers, and so I had to kind of cut back anything that used to. Hands and fingers I hope one day I'll be able to get back to it health wise.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Like you want an actual exactly three to two ratio, whereas with a 12. Interval scale, it's not exactly three to two, for example. So that's”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Program I wrote that I remember would be at high school. I did an assignment where I decided to try to find out if there were some better musical scales than the normal 12 interval scale. So I wrote a program on my Commodore 64 in Basic that searched through other scale sizes to see if it could find one where there were more accurate harmonies.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source