YouSaid · the spoken record

Jeremy Howard

lines on the record: 135
first: 2019-08-27
most recent: 2019-08-27
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Do think that every data scientist working with Deep learning needs to recognize they have an incredibly high leverage tool that they're using that can influence society in lots of ways. And if they're doing research, that that research is going to be used by people doing this kind of work. And they have a responsibility to consider the consequences and to think about things like How will humans be in the loop here? How do we avoid runaway feedback loops? How do we ensure an appeals process for humans that are impacted by my algorithm? How do I ensure that the constraints of my algorithm are adequately explained to the people that end up using them? There's all kinds of human issues which only data scientists are actually in the right place to educate people about. data scientists tend to think of themselves as”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I'm desperately concerned. And you see already. That the changing workplace has lived to a hollowing out of the middle class. You're seeing that students coming out of school today have a less rosy financial future ahead of them than their parents did, which has never happened in recent, in the last few hundred years. We've always had progress before. And you see this turning into... Anxiety and despair and even violence. So I very much worry about that”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I mean, for example, there are problems that AI creates, right? So more specifically, Labor force displacement is going to be huge, and people keep making Frivolous econometric argument of being like, oh, there's been other things that aren't AI that have come along before and haven't created massive labor force displacement, therefore AI won't slightly.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“It's just like there's so many societally important problems to solve right now. Don't find it a really interesting question to even answer”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Have no way to know. I don't know. I don't know why people make predictions about this because there's no data and nothing to go on.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Don't think it's possible to predict. I think Think what we already have is an incredibly powerful platform to solve lots of socially important problems that are currently unsolved. So I just hope that people will, lots of people will learn this toolkit and try to use it. I don't think we need a lot of new technological breakthroughs to do a lot of great work right now.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“So, you have to, you know, you won't remember things well if they don't have some context, and yeah, you won't remember them well if you don't regularly practice them, whether it be just part of your day-to-day life or the Chinese for me flashcards. I mean, the other thing is let yourself fail sometimes. So I've had various medical problems over the last few years, and basically my flashcards just stopped for about three years. There have been other times I've stopped for a few months, and it's so hard because you get back to it, and it's like you have $18,000 due. So you just have to go all right. Well, I can either stop and give up everything or just decide to do this every day for the next two years until I get back to it. The amazing thing has been that even after three years, I...”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“You got to stick with it, which is again the number one thing that 99% of people don't do. So the people I started learning Chinese with none of them were still doing it 12 months later. I'm still doing it 10 years later. I tried to stay in touch with them, but no one did it. Something like Chinese study how human learning works. So every one of my Chinese flashcards is associated with a story and that story is specifically designed to be memorable and we find things memorable which are like funny or disgusting or sexy or related to people that we know or care about. So I try to make sure all the stories that are in my head have those characteristics.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Which is all my colleagues have always hated because it always looks like I'm not working on what I'm meant to be working on, but it always means I do everything faster because I've been practicing a lot of stuff. So I kind of give myself a lot of opportunity to practice new things. And so I find now. Yeah, I don't often kind of find myself wishing I could remember something because if it's something that's useful, then I've been using it a lot. It's easy enough to look it up on Google. But speaking Chinese, you can't look it up on Google.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Even if I don't get much of a chance to exercise it, because I'm not often in China, so I don't, or else something like programming languages or papers, I have a very different approach, which is I try not to learn anything from them, but instead I try to identify the important concepts and actually ingest them. So really understand that concept deeply and study it carefully decide if it really is important. If it is like incorporate it into our library, you know, incorporate it into how I do things, or decide it's Not worth it. So I find Find I then remember the things that I care about because Using it all the time. So for the last. 25 years I've committed to spending at least half of every day learning or practicing something new.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Today's ebbing house is a guy called Pietur Wozniak who developed a system called Super Memo and he's been basically trying to become like World's greatest Renaissance man over the last few decades. He's basically lived his life with space repetition learning for everything. And Michael's only very recently got into this, but he started really getting excited about doing it for a lot of different things. For me personally, I actually don't use it for anything except Chinese. And the reason for that is that Chinese is specifically a thing. I made a conscious decision that I want to continue to remember.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Unfortunately, though, it's also like it doesn't let you fool yourself if you're not learning something. You know your revisions will just get more and more. So you have to find ways to learn things productively and effectively, like treat your brain well. So using mnemonics and stories and context and stuff like that. So yeah, it's a super great technique. It's like learning how to learn is something which everybody should learn before they actually learn anything, but almost nobody does.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Remember. So spaced repetition learning is using this simple algorithm, which is something like revise something after a day and then three days and then a week and then three weeks and so forth. And so if you use a program like Anki, as you know, it will just do that for you. And it will say, did you remember this? And if you say no, it will reschedule it back to be a peer again like 10 times faster than it otherwise would have. It's a kind of a way of being guaranteed to learn something because by definition if you're not learning it, it will be rescheduled and revised more quickly.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“So space repetition An idea created by a psychologist named Ebbinghouse, I don't know, must be a couple hundred years ago or something, 150 years ago. He did something which sounds pretty damn tedious. He wrote down random sequences of letters on cards and tested how well he would remember those random sequences a day later or a week later whatever. He discovered that there was this kind of curve where his probability of remembering one of them would be dramatically smaller the next day and then a little bit smaller the next day and a little bit smaller the next day. What he discovered is that if he revised those cards after a day the probabilities would decrease at a smaller rate and then if you revise them again a week later they would decrease at a smaller rate again and so he basically figured out a roughly optimal equation for when you should revise something you want to”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, they can be, but not at the VC level because the VC exit needs to be Thousand X, so where else the lifestyle exit, if you can sell something for $10 million and you've made it, right? So it depends. If you want to build something that's going to, you're kind of happy to do forever, then fine. If you want to build something, you want to sell. Three years' time, that's fine too. I mean, they're both perfectly good outcomes”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, sure. I mean, it's scary before you jump in, and I just, I guess I was comparing it to the scarediness of VC. I felt like with VC stuff, it was more scary, kind of much more in somebody else's hands, you know, will they fund you or not? And what do they think of what you're doing? I also found it very difficult with VC backed startups to actually do the thing which I thought was important for the company rather than doing the thing which I thought would make the VC happy. And VCs always tell you not to do the thing that makes them happy. But then if you don't do the thing that makes them happy, they get sad.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so I kept my cost down, and once I needed to spend more money, I asked people to spend the money for me. Was that basically from then on? We were making money and I was profitable from then. Optimal decisions, it was a bit harder because we were trying to sell something that was more like a $1 million sale, but what we did was we would sell scoping projects. So kind of like Prototypy projects, but rather than doing it for free, we would sell them for $50,000 to $100,000. So again, we were covering our costs and also making the client feel like we were doing something valuable. So in both cases, we were profitable from... Six months in”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“So, yeah, so I started fast mail and optimal decisions at the same time in 1999 with two different friends Fast Guess I spent $70 a month on the server. And when the server ran out of space, I put a payments button on the front page and said if you want more than 10 mega space, you have to pay $10 a year.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“No VC startups are much more scary because you have these people on your back who do this all the time and who have done it for years telling you grow, grow, grow, grow. And they don't care if you fail. They only care if you don't grow fast enough. So that's scary. Where else doing the ones myself with partners? We're friends is nice because we just went along at a pace that made sense and we were able to build it to something which was big enough that we never had to work again but was not big enough that any VC would think it was impressive. And that was enough for us to be excited, you know. So I thought that's a much better way to do things for most people.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Problem. You picked your PhD topic because it was an interesting kind of engineering or math or research exercise. But yeah, if you've actually spent time as a recruiter and you know that most of your time was spent sifting through resumes and you know that most of the time you're just looking for certain kinds of things and you can try doing that with a model for a few minutes and see whether that's something which the model seems to be able to do as well as you could then you're on the right track to creating a startup. And then I think just yeah being Be pragmatic and And stay away from venture capital money as long as possible, preferably forever.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Same thing as becoming successful deep learning practitioner, which is not giving up Run out of money or run out of time or run out of something, you know. But if you keep costs super low and try and save up some money beforehand so you can afford to have some time. Just sticking with it is one important thing. Doing something you understand and care about is important. By something, I don't mean The biggest problem I see with deep mining people is Do a PhD in deep learning and then they try and commercialize their PhD, which is a waste of time because that doesn't solve an actual”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“And it's a problem because you know when you've actually driven, you know, these are the things that happen to me when I was driving.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, to me, I would compare it to Studying self-driving cars having never looked at a car or been in a car or turned a car on. Know which is like the way it is for a lot of people, they'll study some academic data set. where they literally have no idea about”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“If you're not working on a real problem that you understand, how do you know if you're doing it any good? How do you know if your results are good? How do you know if you're getting bad results, why you're getting bad results? Is it a problem with the data? How do you know you're doing anything useful? Yeah, to me, the only really interesting research is not the only, but the vast majority of interesting research is try and solve an actual problem and solve it really well.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Train lots of models. But specifically, train lots of models in your domain area. So an expert what, right? We don't need more expert like Create slightly evolutionary research in areas that everybody's studying. We need experts at using deep learning to diagnose malaria. Or we need experts at using deep learning to analyze Language to study media bias, or we need experts in Analyzing fisheries to identify problem areas in the ocean. That's what we need. So become the expert in your passion area. And this is a tool which you can use for just about anything. And you'll be able to do that thing better than other people, particularly by combining it with your passion and domain expertise.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“After lesson one. And then there's others which are just more kind of fun, like somebody is doing Trinidad and Tobago hummingbirds. She said that's kind of their national bird. And she's got something that can now classify Trinidad and Tobago hummingbirds. So yeah, train models, fine-tuned models with your data set and then study their inputs and outputs.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“It does it with basically 100% accuracy. Took me about four minutes to scrape the images from Google search from the script. There's a little graphical widgets we have in the notebook that help you clean up the data set. There's other widgets that help you study the results to see where the errors are happening. And so now we've got over a thousand replies in our share your work here thread of students saying here's the thing I built and so there's people who like and a lot of them are state of the art. Like somebody said, oh, I tried looking at Devengari characters and I couldn't believe it. The thing that came out was more accurate than the best academic paper.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“No, you've got to fine-tune the models. So that's the critical thing, because at that point, you now have a model that's in your domain area. So there's no point running somebody else's model because it's not your model. So it only takes five minutes to find tune a model for the data you care about. And in lesson two of the course, we teach you how to create your own data set from scratch by scripting Google Image Search. So we show you how to actually create a web application running online. So I create one in the course that differentiates between a teddy bear, a grizzly bear, and a brown bear.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Train lots of models. That's how you learn it. So I think it's not just me. I think our course is very good, but also lots of people independently have said it's very good. It recently won the COGX Award for AI courses as being the best in the world. So I'd say come to our course, course.fast.ai. And the thing I keep on harping on in my lessons is train models print out the inputs to the models, print out to the outputs to the models, like study change the inputs a bit, look at how the outputs vary just run lots of experiments to get a intuitive understanding of what's going on.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Of the ones who don't give up pretty much everybody succeeds, even if at first I'm just kind of thinking like, wow, they really aren't quite getting it yet, are they? But eventually people get it and they succeed. So I think that's, I think they're both things I've liked to believe was true, but I don't feel like I really had strong evidence for them to be true. But now I can say I've seen it again and again.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Say taught me, but convinced me something that I liked to believe was true, which was anyone can do it. So there's a lot of kind of snobbishness out there about only certain people can learn to code, only certain people are going to be smart enough to do AI. That's definitely bullshit. You know, I've seen so many people from so many different backgrounds get state-of-the-art results in their domain areas now. It's definitely taught me that the key differentiator between people that succeed and people that fail is tenacity. That seems to be basically the only thing that matters. The people, a lot of people give up.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“A lot. It's a key reason for me to teach the courses. I mean, obviously, it's going to be necessary to achieve our goal of getting domain experts to be familiar with deep learning, but it was also necessary for me to achieve my goal of being really familiar with deep learning. I mean, to see so many domain experts from so many different backgrounds, it's definitely”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“I would say coding Yeah, I would say the people who are strong coders pick it up the best Although another bottleneck is people who have a lot of experience of Classic statistics can really struggle because the intuition is so the opposite of what they're used to, they're very used to like trying to reduce the number of parameters in their model. Looking at individual coefficients and stuff like that. So I find people who have a lot of coding background and know nothing about statistics are generally going to be the best off.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“That's it. Okay. But a lot of people I know take a year off to study FastAI full time. Say at the end of the year they feel pretty competent because generally there's a lot of other things you do like generally they'll be entering Kaggle competitions, they might be reading Ian Goodfellow's book, they might, you know, they'll be doing a bunch of stuff. Often, you know, particularly if they are a domain expert, their coding skills might be a little on the pedestrian side, so part of it's just doing a lot more writing.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Off their ass and start appreciating this because currently all of their low level libraries are not written in SWIFT. They're not particularly Swifty at all stuff like CoreML. They're really pretty rubbish. So yeah, so there's a long way to go. But at least one nice thing is that Swift for TensorFlow can actually directly use Python code and Python libraries in a literally the entire lesson one notebook of FastAI runs in Swift right now in Python mode. So that's a nice intermediate thing.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Not for a few years, particularly because Swift has Data science community libraries. And the SWIFT community has Total lack of appreciation and understanding of numeric computing, so they keep on making stupid decisions. You know, for years they've just done dumb things around performance and prioritization. That's clearly changing now because the developer of Swift, Chris Lattner, is working at Google on Swift for TensorFlows. So that's a priority. It'll be interesting to see what happens with Apple because Apple hasn't shown any sign of caring about numeric programming in Swift. So hopefully they'll...”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, we obviously recommend FastAI and PyTorch because we teach new students, and that's what we teach with. So we would very strongly recommend that because it will let you get on top of the concepts much more quickly. So then you'll become an action. And you'll also learn the actual state of the art techniques. So you'll actually get world-class results. Honestly, it doesn't much matter what library you learn because switching from Chainer to MXNet to TensorFlow to PyTorch is going to be a couple of days work as long as you understand the foundations well”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Not too different now, thanks to TensorFlow Eager, that's not too different, but because so many things take so long to run. You wouldn't run it at 10 times slower. Like, you could just go, like, oh, this is taking too long Also, there's a lot of things that are just less programmable, like tf.data, which is the way data processing works in TensorFlow, is just this big mess. It's incredibly inefficient. TPU problems I described So I just, you know, I just feel like they've got this huge technical debt which they're not going to solve without starting from scratch.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, it does the job if you're using predefined things that somebody's already written If you actually compare, like I've had to do because I've been having to do a lot of stuff with TensorFlower recently, you actually compare something from scratch and you're like, I just keep finding it's like, oh, it's running 10 times slower than PyTorch.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, well, particularly the way TensorFlow was written, it was written by a lot of people very quickly in a very disorganized way. So when you actually look in the code as I do often, I'm always just like, oh God, what were they thinking? It's just, it's pretty awful. So I'm really extremely negative about the potential future for. That's Swift for TensorFlow can be a different beast altogether. It can be like, it can basically be a layer on top of MLIR that takes advantage of all the great compiler stuff that Swift builds on with LLVM. And I think it will be absolutely fantastic.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Because a Python for TensorFlow is not going to cut it. It's just a disaster. What they did was they tried to replicate Bits that people were saying they like about PyTorch, this kind of interactive computation, but they didn't actually change their foundational runtime components. So they kind of added this syntax sugar they call TFEGA, TensorFlow Eager, which makes it look a lot like PyTorch, but it's 10 times slower than PyTorch to actually... Do a step. So, because they didn't invest the time in retooling the foundations because their code base is so horribly complex”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“So, in the latest incarnation of the course and with some of the research we're still now starting to do, we're starting to do stuff, some stuff in Swift. I think we're three years away from that being super practical, but I'm in no hurry. I'm very happy to invest the time to get there. With that, we actually already have a nascent version of the FastAI library for vision running on Swift TensorFlow.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“And which kind of talks to an API, which talks to an API, which talks to an API, which like you can dive into at any level and get progressively closer to the machine kind of levels of control And this is the fast AI library. That's been critical for us and for our students and for lots of people that have won big learning competitions with it and written academic papers with it. It's made a big difference. Still limited though by Python Particularly this problem with things like recurrent neural nets where you just can't change things unless you accept it going so slowly that it's impractical.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, it was critical for us for something like Dawn Bench to be able to rapidly try things. It's just so much harder to be a researcher and practitioner when you have to do everything up front and you can't inspect it. Problem with PyTorch is It's not at all accessible to newcomers because you have to write your own training loop and manage the gradients and all this stuff. It's also not great for researchers because you're spending your time dealing with all this boilerplate and overhead rather than thinking about your algorithm. So we ended up writing this very multi-layered API that at the top level you can train a state-of-the-art neural network in three lines of code.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Were great, but were much harder to teach and to do research and development on because they define what's called a computational graph up front, a static graph, where you basically have to say here are all the things that I'm going to eventually do in my model. And then later on you say, okay, do those things with this data. And you can't debug them, you can't do them step by step, you can't program them interactively in a Jupyter notebook and so forth. PyTorch was not the first, but PyTorch was certainly the strongest entrant to come along and say, let's not do it that way. Let's just use normal Python. And everything you know about in Python is just going to work, and we'll figure out how to make that run on the GPU as and when necessary”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“So, in terms of what we've done our research on and taught in our course, we started with Thiano. And Keras, and then we switch to TensorFlow and Keras. And then we switch to PyTorch, and then we switch to PyTorch and FastAI. And that kind of reflects a growth and development of the ecosystem of deep learning libraries. Theano and TensorFlow.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“GCP gives you $300 of compute for free, which is really nice. As I say, salamander and paper spacer are even easier still.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“So things are so much better now, and like we actually have if you go to course.fast.ai, the first thing it says is here's how to get started with your GPU and there's like you just click on the link and you click start”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“Has all the courses pre installed, it has Jupyter Notebook pre running Jupyter Notebook is this wonderful interactive computing system which everybody basically should be using for any kind of data-driven research. But then even better than that, there are platforms like Salamander, which we own, and Paperspace, where literally you click a single button and it pops up a Jupyter notebook straight away without any kind of installation or anything and all the course notebooks are all pre-installed. So like for me, this is one of the things we spent a lot of time kind of curating and working on. When we first started our courses, the biggest problem was people dropped out of lesson one because they couldn't get an AWS instance running.”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source
“So, from a hardware point of view, Google's TPUs and the best NVIDIA GPUs are similar, maybe the TPUs are like 30% faster, but they're also much harder to program There isn't a clear leader in terms of hardware right now, although much more importantly the NVIDIA's GPUs are much more programmable. They've got much more written for all of them, so that's the clear leader for me and where I would spend my time as a researcher and practitioner. But then in terms of the platform, We're super lucky now with stuff like Google GCP, Google Cloud, and AWS that you can access a GPU pretty quickly and easily. I mean, for AWS, it's still too hard. You have to. Find an AMI and get the instance running and then install the software you want and blah, blah, blah. GCP is currently the best way to get started on a full server environment”
2019-08-27 · Lex Fridman Podcast · Jeremy Howard: fast.ai Deep Learning Courses and Research · IDENTIFIED FROM THE TRANSCRIPT · source