YouSaid · the spoken record

Andrew Ng

lines on the record: 94
first: 2020-02-20
most recent: 2020-02-20
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“You and then second is helping other people. I think to me, I think the meaning of life is helping others achieve whatever are their dreams. And then also to try to move the world forward by making humanity more powerful as a whole. So the times that I felt most happy, most proud was when I felt someone else Allow me the good fortune of helping them a little bit on the path to their dreams.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“You know, I've made so many mistakes. It feels like every time I discover something, I go, why didn't I think of this, you know, five years earlier or 10 years earlier? And sometimes I read a book and I go, I wish I read this book 10 years ago. Oh, my life would have been so different. Although that happened recently, and then I was thinking, if only I read this book when we're starting at Coursera, I could have been so much better. When I discovered the book had not yet been written or starting Coursera, so that made me feel But I find that the process of discovery we keep on finding out things that seem so obvious in hindsight, but always takes us so much longer than I wish to figure it out.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and I think just some companies and a regulator comes to you and says, look, your product is messing things up, fixing it may have a revenue impact. Well, it's much more fun to talk to them about how you promise not to wipe out humanity than to face the actually really hard problems we face.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I worry about some teams, maybe accidentally, and I hope not deliberately, making a lot of noise about things that problems in the distant future rather than focusing on senses of much harder problems.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Inequality, the AI and internet are causing an acceleration of concentration of power because we can now centralize data, use AI to process it. And so industry after industry, we've affected every industry. So the internet industry has a lot of win and take most of win and take all dynamics. But we've infected all these other industries. So also giving these other industries win and take most of Winnipeg all flavors. So look at what Uber and Lyft did to the taxi industry. So we're doing this type of thing. So we're creating tremendous wealth, but how do we make sure that the wealth is fairly shared? I think that how do we help people whose jobs are displaced? I think education is part of it. There may be even more that we need to do than education. I think bias is a serious issue. They're adverse users of AI like deepfakes being used for various nefarious purposes.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“If you take self-driving cars, the biggest problem with self-driving cars is not that there's some trolley dilemma and you teach this. So, you know, how many times when you're driving your car did you face this moral dilemma? Who did I crash into? So I think self-driving cars will run into that problem roughly as often as we do when we drive our cars. The biggest problem with South Giant Cars is when there's a big white truck across the road and what you should do is brake and not crash into it. Self-driving car fails and it crashes into it. So I think we need to solve that problem for us. I think the problem with some of these discussions about AGI alignment, the paperclip problem is that is a huge distraction from the much harder problems that we actually need to address today. Some of the hard problems need to address today. I think bias is a huge issue. I worry about wealth.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I do worry about the long-term fate of humanity. I do wonder as well, I do worry about overpopulation on the planet Mars, just not today. I think there will be a day when maybe someday in the future Mars will be polluted. There are all these children dying and someone look back at this video and say, Andrew, how was Andrew so heartless? He didn't care about all these children dying on the planet Mars. And I apologize to the future viewer. I do care about the children, but I just don't know how to productively work on that today.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I would love to get to AGI, and I think humanity will, but whether it takes 100 years or 500 or 5,000, I find hard to estimate.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“But if you have a system got in a factory, there are not 20 machine learning engineers sitting around you can pay the duty and have them fix it. So how do you deal with the maintenance or the dev offs or the MOOCs or the other aspects of this? So these are concepts that I think landing AI and a few other teams on the cutting edge of but we don't even have systematic terminology yet to describe some of the stuff we do because I think we're indenting it on the fly.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Is working upon us to think through all the things beyond just the machine learning model of running a Jupyter notebook, but build the entire system, manage the change process, and figure out how to deploy this in a way that has an actual impact. The processes that the large software tech companies use for deploying don't work for a lot of other scenarios. For example, when I was leading launch speech teams, if the speech recognition system goes down, what happens? Well, alarms goes off, and then someone like me would say, hey, U20 engineers, please fix this. And it would get.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, and the software system needs to interface with people's workloads. So machine learning is automational steroids. If we take one task out of many tasks that done in factory, so a factory does lots of things. One task is visual inspection. If we automate that one task, it can be really valuable, but you may need to redesign a lot of other tasks around that one task. For example, say the machine learning algorithm says this is defective. What are you supposed to do? Do you throw it away? Do you get a human to double check? Do you want to rework it or fix it? So you need to redesign a lot of tasks around that thing you've now automated. So planning for the change management and making sure that the software you write is consistent with the new workflow. And you take the time to explain to people what needs to happen. So I think what landing AI has become good at, and I think we learn by making mistakes. painful experiences, what would become good at.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“The test set on your hard drive versus what it takes to work well in a deployment setting. Some common problems. Robustness and generalization, you deploy something in the factory, maybe they chop down a tree outside the factory so the tree no longer covers a window and the lighting is different. So the test set changes. In machine learning, and especially in academia, we don't know how to deal with test set distributions that are dramatically different than the training set distribution. There's research, there's stuff like domain annotation, transfer learning, the people working on it, but we're really not good at this. So how do you actually get this to work? Because your test set distribution is going to change. And I think also if you look at the number of lines of code in the software system, the machine learning model is maybe 5% or even fewer relative to the entire software.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think building and deploying machine learning systems is hard. There's a huge gulf between something that works in a Jupyter notebook on your laptop versus something that runs a production deployment setting in a factory or agriculture or plant or whatever. So I see a lot of people get something to work on your laptop and say, wow, look what I've done. And that's great. That's hard. That's a very important first step. But a lot of teams underestimate the rest of the steps needed. So for example, I've heard this exact same conversation between a lot of machine learning people and business people. The machine learning person says, look, my algorithm does well on the test set and it's a clean test set at the peak. And the business person says, thank you very much, but your algorithm sucks. It doesn't work. And the machine learning person says, no, wait, I did well on the test set. And I think there is a gulf between what it takes to do well.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think the early small scale projects, it helps the teams gain faith, but also helps the teams learn what these technologies do. I still remember when our first GPU server, it was a server under some guy's desk. And then that taught us early important lessons about how do you have multiple users share a set of GPUs, which is really non-obvious at the time. But those early lessons were important. We learned a lot from that first GPU server that later helped the teams think through how to scale it up to much larger deployments.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I publish a document called the AI Transformation Playbook that's online and taught briefly in the AI for everyone course on Coursera about the long-term journey that companies should take. But the first step is actually to start small. I've seen a lot more companies fail by starting too big than by starting too small. Take even Google. Most people don't realize how hard it was and how controversial it was in the early days. So when I started Google Brain It was controversial. People thought deep learning, Neuronet tried it, didn't work. Why would you want to do deep learning? So my first internal customer within Google”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Recently there was a factory in which a bird threw through the factory and pooped on something. And so that changed stuff. And so increasing our river robustness. So all the changes happen in the factory. I find that we run a lot of practical problems that are not as widely discussed in academia and is really fun kind of being on the cutting edge solving these problems before maybe before many people are even aware that there is a problem there.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Frankly, I think as all of them. Some of the ones I'm spending a lot of time on are manufacturing agriculture looking to healthcare. For example, in manufacturing, we do a lot of work in visual inspection where today there are people standing around using the eye human eye to check if this plastic pot or this smartphone or this thing has a scratch or a dense or something in it. We can use a camera to take a picture, use a algorithm, deep learning and other things to check if it's defective or not and does help factories improve yield and improve quality and improve throughput. It turns out the practical problems we run into are very different than the ones you might read about in most research papers. The data sets are really small, so we face small data problems. The factories, keypoint changing the environment. So it works well on your test set, but guess what? Something changes in the factory. The lights go on.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Impact will be outside the software internet sector. So we need more teams to work with these companies to help them adopt AI. And I think this is one of the things that will help drive global economic growth and make humanity more powerful.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“AI is a general purpose technology, and I think it will transform every industry. Our community has already transformed to a large extent the software internet sector. Most software internet companies are outside the top, five or six or three or four already have reasonable machine learning capabilities or getting there. There's still room for improvement. But when I look outside the software internet sector, everything from manufacturing, agriculture, healthcare, logistics, transportation, there's so many opportunities that very few people are working on. So I think the next way for AI is for us to also transform all of those other industries. There was a McKinsey study estimating $13 trillion of global economic growth. US GDP is $19 trillion, so $13 trillion is a big number, or PWC has been $16 trillion. So whatever number is large. But the interesting thing to me was a lot of that.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I feel like it's really important that we build companies that move the world forward. For example, within the AI Fun team, there was once an idea for a new company that if it had succeeded, would have resulted in people watching a lot more videos in a certain narrow vertical type of video. I looked at it, the business case was fine, the revenue case was fine, but I looked at it and just said, I don't want to do this. It wasn't educational. It was educational, maybe. And so, and so I code the idea on the basis that I didn't think it would actually help people. Whether building companies or work of enterprises or doing personal projects, I think. It's up to each of us to figure out what's the difference we want to make in the world.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Facing with these key decisions like trying to hire your first VP of engineering, what's good selection criteria? How do you solve? Should I hire this person or not? By having an ecosystem around the entrepreneurs, the founders to help. I think we help them at the key moments and hopefully significantly make them more enjoyable and higher success rate.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Very much so. You know, starting a company to most entrepreneurs is a really lonely thing. And I've seen so many entrepreneurs not know how to make certain decisions. When do you need to, how do you do B2P sales, right? If you don't know that, it's really hard. Or how do you market this efficiently other than buying ads, which is really expensive? Are there more efficient tactics that? Or for a machine learning project, basic decisions can change the course of whether machine learning product works or not. And so there are so many hundreds of decisions that entrepreneurs need to make and making a mistake in a couple of key decisions can have a huge impact on the faith of the company. So I think it's not a studio provides a support structure that makes starting a company much less of a lonely experience. And also when”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Is actually a place for us to build startups from scratch. So we often bring in founders and work with them or maybe even have existing ideas that we match founders with. And then this launches hopefully into successful companies.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Now do with AI. I think the ability to build new teams, to go after this rich space of opportunities is a very important way, very important mechanism to get these projects done that I think will move the world forward. So I've unfortunately built a few teams that had a meaningful positive impact. And I felt that we might be able to do this in a more systematic, repeatable way. So a startup studio is a relatively new concept. There are maybe dozens of startup studios right now, but I feel like all of us, many teams are still trying to figure out how do you systematically build companies with a high success rate. So I think even a lot of my venture capital friends are seem to be more and more building companies rather than investing companies. But I find a fascinating thing to do to figure out the mechanisms by which we could systematically build.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I remember when I was leading the AI group at Baidu, I had two jobs, two parts of my job. One was to build an AI engine to support the existing businesses. And that was running. It just ran, just performed by itself. The second part of my job at the time, which was to try to systematically initiate new lines of businesses using the company's AI capabilities. So the self-driving car team came out of my group, the smart speaker team, similar to what is Amazon Echo or Alexa in the US, but we actually announced it before Amazon did. So I do wasn't following Amazon, that came out of my group. And I found that to be actually the most fun part of my job. So what I want to do was to build AI fund as a startup studio to systematically create new startups from scratch. With all the things we”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think so. I think startups that are very customer focused, customer-sized, deeply understand the customer and oriented to serve the customer are more likely to succeed. With a provisio, I think all of us should only do things that we think create social good and moves the world forward. I personally don't want to build addictive digital products just to sell a lot of ads. There are things that could be lucrative that I won't do But if we can find ways to serve people in meaningful ways, I think those can be great things to do. Even the academic setting or in a corporate setting or a startup setting.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“In Silicon Valley, a lot of stars of failure has come from building products that no one wanted. So cool technology, but who's going to use it? I think I tend to be very outcome driven and customer obsessed. Ultimately, we don't get to vote if we succeed or fail. It's only the customer, that they're the only one that gets a thumbs up or thumbs down vote in the long term. In the short term, there are various people that get various votes, but in the long term, that's what really matters.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And actually, in my standard class, CS230, as well as an ACN talk, I think I gave an hour-long talk on career advice, including on the job search process and some of these. So you can find those videos online.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think when someone interviews at a university or the research lab or the launch corporation, it's good to insist on just asking who are the people, who is my manager, and if you refuse to tell me, I'm going to think, well, maybe that's because you don't have a good answer. It may not be someone I like.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Some team doing really work that doesn't excite you. And then that's actually a really bad experience. So this is true both for universities and for large companies. For small companies, you can kind of figure out who you work with quite quickly. And I tend to advise people if a company refuses to tell you who you would work with. So if you say, oh, join us, the rotation system, we'll figure it out. I think that's a worrying answer because it means you may not get sense to you may not actually get to a team with great peers and great people to work with.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“The thing that affects your experience more is less are you in discomfort versus that company or academia versus industry. I think the thing that affects your experience most is who are the people you're interacting with in a daily basis. So even if you look at some of the large companies, the experience of individuals and different teams is very different. And what matters most is not the logo above the door when you walk into the giant building every day. What matters to Moses, who are the 10 people, who are the 30 people you interact with every day? So I actually tend to advise people, if you get a job from a company, ask who is your manager? Who are your peers? Who are you actually going to talk to? We're all social creatures. We tend to become more like the people around us. And if you're working with great people, you will learn faster. Or if you get admitted, if you get a job at a great company or a great university, maybe the logo you walk in, you know, is great, but you're actually stuck in.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that there are multiple good options of which doing a PhD could be one of them. I think that if someone's admitted to a top PhD program at MIT, Stanford, top schools, I think that's a very good experience. Or someone gets a job at a top organization, at the top AI team, I think that's also a very good experience. There are some things you still need a PhD to do. If someone's aspiration is to be a professor, you're the top academic university. You just need a PhD to do that. But if it goes to start a company, build a company, do great technical work, I think a PhD is a good experience, but I would look at the different options available to someone, where the places where you can get a job, where the places can get in a PHP program, and kind of weigh the pros and cons of those.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Organizational level for a company to become good at machine learning, sometimes the right thing to do is not to tackle the giant project, is instead to do the small project that lets the organization learn and then build up from there. But this is true both for individuals and for companies.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Causes because it's too cutting edge, the cause hasn't been created yet. There's some practical experience that we're not yet that good as teaching in a course. And I think after exhausting the efficient coursework, then most people need to go on to either ideally work on projects and then maybe also continue their learning by reading blog posts and research papers and things like that. Doing projects is really important. Again, I think it's important to start small and just do something. Today you read about deep learning. If you're like, oh, all these people are doing such exciting things. What if I'm not building a neural network that changes the world? Then what's the point? Well, the point is sometimes building that tiny neural network, you know, be it MNIS or upgrade to a fashion MNINIS to whatever. Doing your own fun hobby project, that's how you gain the skills to let you do bigger and bigger projects. I find this to be true at the individual level and also at the”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, Muscle Pont's thing is to get started I think in the early parts of a career coursework, like the deep learning specialization is a very efficient way to master this material. instructors, be it me or someone else or Lawrence Moroni, teachers are TensorFlow specialization or other things we're working on, spend effort to try to make it time efficient for you to learn new concept. So coursework is actually a very efficient way for people to learn concepts in the beginning parts of break into new field. In fact, one thing I see at Stanford, some of my PhD students want to jump in the research right away and I actually tend to say look in your first couple years of PhD student spend time taking courses because it lays the foundation. It's fine if you're less productive in your first couple of years, you'll be better off in the long term. Beyond a certain point, there's materials that doesn't exist in.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, one of the things I do both in creating videos or when we write the badge is I try to think, is one minute spent with us going to be a more efficient learning experience than one minute spent anywhere else? And we really try to make it time efficient for the learners because everyone's busy. So when we're editing, I often tell my teams every work needs to fight for his life. And if we can delete a word, let's just delete it and not wait. Let's not waste the learn this time.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And because you can't write as many words, you have to take what they said and summarize it into fewer words. And that summarization process requires deeper processing of the meaning, which then results in better attention. And I spent, I think, because of Coursera, I spent so much time studying pedagogy, it's actually one of my passions. I really love learning how to more efficiently help others learn.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“There have been a few studies if you search online, you find some of these studies that taking handwritten nose, because handwriting is slower, as we're saying just now. It causes you to recode the knowledge in your own words more, and that process of recoding promotes long-term retention. This is as opposed to typing, which is fine. Again, typing is better than nothing and taking a class and not taking an analysis better than not taking any class at all. But comparing handwritten notes and typing, you can usually type faster for a lot of people that you can handwrite nose. And so when people type, they're more likely to transcribe verbatim what they heard and that reduces the amount of recoding and that actually results in less long-term retention.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“One thing I still do when I'm trying to study something really deeply is take handwritten notes. It varies. I know there are a lot of people that take the deep learning courses during a commute or something where maybe more awkward to take notes. So I know it may not work for everyone. But when I'm taking courses on Coursera and I still take some every now and then, the most recent one I took was a Course on Clinical Trials because I was interested about that. I got out my little Moleskin notebook and was sitting on my desk was just taking down notes of what the instructor was saying. And that act, we know that that act of taking notes preferably handwritten notes increases retention.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and I think it's often not about the births of sustained efforts and the all-nighters because economically. I think reading two research papers is a nice thing to do, but the power is not reading two research papers, it's reading two research papers a week for a year, then you read 100 papers and you actually learn a lot when you read 100 papers.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Much harder if we have to make a decision every morning. And actually, that's the reason why we're the same thing every day as well. It's just one less decision. I just get up and wear my blue shirt. But I think if you can get that habit, that consistency of studying, then it actually feels easier.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think getting in the habit of learning is key and that means regularity. So for example, we send out a weekly newsletter The Batch every Wednesday so people know it's coming Wednesday, you can spend all the time on Wednesday catching up on the latest news through the batch on Wednesday. And for myself, I've picked up a habit of spending some time every Saturday and every Sunday reading or studying. And so I don't wake up on a Saturday and have to make a decision. Do I feel like reading or studying today or not? It's just what I do. And the fact is a habit makes it easier. So I think if someone can get into that habit, it's like, you know, just like we brush our teeth every morning. I don't think about it. If I thought about it, it's a little bit annoying to have to spend two minutes doing that, but it's a habit that it takes no cognitive load. But this would be so.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Official length of the D length specialization is, I think, 16 weeks, so about four months, but go at your own pace. So if you subscribe to the D-Lang specialization, there are people that have finished it in less than a month by working more intensely and studying more intensely. So it really depends on the individual. When we created the DeVank specialization, we wanted to make it very accessible and very affordable. And with Coursera and DVLI's education mission, one of the things that's really important to me is that if there's someone for whom paying anything is a financial hardship, then just apply for financial aid and get it for free.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Think there's a lot more work that one could explore around this theme of ideas and other ideas to come a better algorithms.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't think there's a whole enchilada. I think it's just a piece of it. And I think this one piece self supervised learning is starting to get traction. We're very close to it being useful. Well, word embeddings are really useful. I think we're getting closer and closer to just having a significant real world impact, maybe in computer vision and video. But I think this concept, and I think there'll be other concepts around it. Other unsupervised learning things that I worked on, I've been excited about I was really excited about smart coding and ICA, slow feature analysis. I think all of these are ideas that various of us were working on about a decade ago before we all got distracted by how well supervised learning was doing.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Of the ways we learn word embeddings is another example, and I think there's now this portfolio of techniques for generating these made up tasks. Another one called Jigsaw would be if you take an image, cuts it up into a three by three grid, so like a nine, three by three puzzle piece, jump out the nine pieces and have a neural network predict which of the nine factorial possible permutations it came from. So many groups, including OpenAI, PDRP has been doing some work on this too. Facebook, Google Brain, I think DeepMind, Aaron Van der Old has great work on the CPC objective. So many teams are doing siting work, and I think this is a way to generate label data. And I find this very exciting piece of unsupervised learning.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So here's the example of self supervised learning. Let's say we grab a lot of unlabeled images off the internet, so with infinite amounts of disturbed data, I'm going to take each image and rotate it by a random multiple of 90 degrees. And then I'm going to train a supervised neural network to predict what was the original orientation. So has this been rotated 90 degrees, 180 degrees, 270 degrees, or zero degrees. So you can generate an infinite amount of labeled data because you rotate the image so you know what's the ground truth label. And so various researchers have found that by taking unlabeled data and making up labeled data sets and training a large neural network on these tasks, you can then take the hidden layer representation and transfer it to a different task very powerfully. Learning where the embeddings where we take a sentence, delete a word, predict the missing word, which is how we learn.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that if my only job was being an academic researcher, if an unlimited budget and didn't have to worry about short-term impact and only focus on long-term impact, I probably spent all my time doing research on unsupervised learning. I still think unsupervised learning is a beautiful idea. At both this past Europe and ICML, I was attending workshops or listening to various talks about self-supervised learning, which is one vertical segment, maybe of sort of unsupervised learning that I'm excited about. Maybe just to summarize the idea, I guess you know the idea I'll describe briefly.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“That's really cool, and I feel like the world actually needs all sorts, even within machine learning. I feel like deep learning is so exciting, but AI team shouldn't just use deep learning. I find that my teams use a portfolio of tools. Maybe that's not”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source
“One of my teams is looking to reinforce the learning for some real party control tasks. So I see the applications, but if you look at it as a percentage of all of the impact of the types of things we do, at least today outside of playing video games in a few other games, the scope. Actually at Neuros, a bunch of us were standing around saying, hey, what's your best example of an actual deploy reinforcement learning application? And among senior machine learning researchers. And again, there are some emerging ones, but there are not that many great examples.”
2020-02-20 · Lex Fridman Podcast · #73 – Andrew Ng: Deep Learning, Education, and Real-World AI · IDENTIFIED FROM THE TRANSCRIPT · source