YouSaid · the spoken record

Tom Mitchell

lines on the record: 60
first: 2019-06-19
most recent: 2019-06-19
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Right, and shaming them into adopting a new paradigm. And so one of the slogans, one of the phrases you kept hearing when people started working on probabilistic statistical, probabilistic methods, they would never call them that. They would have called them instead. Principled probabilistic method. Just to kind of shine a light on the distinction between neural nets, which are just somehow tuning a gazillion parameters and the principled methods that were being used. And so that became really the dominant paradigm in the late 90s and kind of remained in charge of the field up through Till about 2009, 2010, when, now, as everybody kind of knows, Deep Networks made a very serious revolution showing that they could do all kinds of amazing things that hadn't been done before.”
2019-06-19 · a16z Podcast · a16z Podcast: The History and Future of Machine Learning · IDENTIFIED FROM THE TRANSCRIPT · source
“Around the late 80s, early 90s, that started competing with the idea of symbolic representations. But then in the late 90s, the statisticians moved in and probabilistic methods became very popular. And at the time, there was this, if you look at this history, you can't help but realize what a social phenomenon technology advance is in sciences and technology.”
2019-06-19 · a16z Podcast · a16z Podcast: The History and Future of Machine Learning · IDENTIFIED FROM THE TRANSCRIPT · source
“Right. And so around 19, in the late 90s, in fact, so if you look at the history of machine learning, there's an interesting trajectory where in maybe up to the mid-80s things were pretty much focused on symbolic representations. Actually, if you go back to the 60s, in other words, the perceptron, but then it got swallowed up by the end of the 60s by symbolic representations and trying to reason that way and trying to learn those kind of symbolic structures. Then when the neural net wave came in the...”
2019-06-19 · a16z Podcast · a16z Podcast: The History and Future of Machine Learning · IDENTIFIED FROM THE TRANSCRIPT · source
“And so one of the big changes between the 80s and 2019 is that We no longer really think in the field of AI that inference is proving things. Instead, it's building a plausible chain of argument. And it might be wrong, and if it goes wrong, if there is a banana in the tailpipe, you'll deal with it when it happens and when you figure it out.”
2019-06-19 · a16z Podcast · a16z Podcast: The History and Future of Machine Learning · IDENTIFIED FROM THE TRANSCRIPT · source
“That's right. And so the point of the banana and the tailpipe is there are an infinite number of other things that you don't say when you spin out a plan like that. And any proof, if it's a proof really, is going to have to cover all those conditions. And that's kind of infinitely intractable problem.”
2019-06-19 · a16z Podcast · a16z Podcast: The History and Future of Machine Learning · IDENTIFIED FROM THE TRANSCRIPT · source
“Like, how would you get from here to the airport? Well, you'd walk to your car, you'd turn, put the key in, turn the car on, you'd drive out of the parking lot, get on the interstate, go to the airport exit, et cetera. But what if there's banana in the tailpipe?”
2019-06-19 · a16z Podcast · a16z Podcast: The History and Future of Machine Learning · IDENTIFIED FROM THE TRANSCRIPT · source
“So back in the 80s and the 90s, And I have to say that some of the really senior people in the field were totally devoted to this paradigm of logical inference, logical representations. People like John McCarthy, for example, were very strong proponents of this. And really essentially just saw That reasoning is theorem proving, and therefore, if I'm going to get computers to do it, that's what we have to do. There were some problems with that, and there still are. One that I remember from back then that was an example was the banana in the tailpipe problem. These logical systems were used to reason to do things like how would you plan a sequence of actions to achieve a goal?”
2019-06-19 · a16z Podcast · a16z Podcast: The History and Future of Machine Learning · IDENTIFIED FROM THE TRANSCRIPT · source
“The Empire wanted us to do research on knowledge representation and inference and first order logic. I remember as an undergrad, I took this computer-aided class that John E. Mendi wrote called Tarski's World, where we learned all about First World Logic, right? What could you prove? What could you not prove? And so that's what the establishment, quote-unquote, was teaching. And then Jeff was the rebel off in neural network land and he gets his reprise later. Take us back to the world of knowledge representation because I'm actually seeing a lot of startups these days who are trying to bring back some of these techniques to complement deep learning because, you know, there are well-known challenges with deep learning, right? Like we're not encoding any priors, we're learning everything for the first time. We need tons of data labeled data sets to make progress.”
2019-06-19 · a16z Podcast · a16z Podcast: The History and Future of Machine Learning · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, Jay was Jay McClellan as a psychologist here at CMU. Rummel Hart, kind of a neuroscientist. And more, he was a very broad person. And Jeff. So the three of them were kind of the rebels who were taking things off in a different paradigm.”
2019-06-19 · a16z Podcast · a16z Podcast: The History and Future of Machine Learning · IDENTIFIED FROM THE TRANSCRIPT · source
“That's right. Well, at the time, this was the 1980s, so artificial intelligence at that point was dominated by what we would call symbolic methods. Where things like formal logic would be used to do inference. And much of machine learning was really about learning symbolic structures, symbolic representations of knowledge. But there was this kind of young whipper snapper, Jeff Hinton, who had a different idea. And so he was working on a book with Rommel Hart McClellan that became a very well-known parallel data processing book that kind of launched the field of neural nets.”
2019-06-19 · a16z Podcast · a16z Podcast: The History and Future of Machine Learning · IDENTIFIED FROM THE TRANSCRIPT · source