YouSaid · the spoken record

Gary Marcus

lines on the record: 109
first: 2019-10-03
most recent: 2019-10-03
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Thanks for the clarification. I guess John Lennon's such an intriguing person and I mean a troubled person but an intriguing one so beautiful”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Those would be different, but I'll say that my favorite rock song to listen to is probably all along the watchtower that Jimi Hendrix version.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“The demo that you can try. If you can't try it at home, maybe it doesn't really work that well yet. So if we don't have this example in the book, but if Sundar Pinchai says we have this thing that allows it to sound like human beings in conversation, you should ask, can I try it? And you should ask how general it is. And it turns out at that time, I'm alluding to Google Duplex when it was announced, it only worked on calling hairdressers, restaurants, and finding opening hours. That's not very general. That's narrow AI.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Part of the reason we wrote the book was to try to inform those committees. So part of the reason we wrote the book was to inspire a future generation of students to solve what we think are the important problems. So a lot of the book is trying to pinpoint what we think are the hard problems where we think effort would most be rewarded. And part of it is to try to train people Who talk about AI but aren't experts in the field to understand what's realistic and what's not.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“We're fooling ourselves in thinking that we can make trustworthy AI if we can't translate harm into something that we can execute. And if we can't, then we should be thinking really hard. How could we ever do such a thing? Because if we're going to use AI in the ways that we want to use it, to make job interviews or to do surveillance, not that I personally want to do that or whatever. I mean, if we're going to use AI in ways that have practical impact on people's lives or medicine, it's got to be able to understand stuff like that.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Translated into something that machines can work with. And that means there has to be a way of working the translation. And right now we don't. We don't have a way. So let's say you and I were the committee and we decide that Asimov's first law is actually right. And let's say it's not just two white guys, which would be kind of unfortunate and that we have a broad, and so we have a representative sample of the world or however we want to do this. And the committee decides eventually, okay, Azimo's first law is actually pretty good. There are these exceptions to it. We want to program in these exceptions. But let's start with just the first one, and then we'll get to the exceptions. First one is first do no harm. Well, somebody has to now actually turn that into a computer program or a neural network or something. And one way of taking the whole book, the whole argument that I'm making, is that we just don't know how to do that. And we're fooling ourselves if we think that we can build trustworthy AI if we can't even specify in any kind of, you know, we can't do it in Python and we can't do it in TensorFlow.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I might run it in the reverse direction, but roughly speaking, I agree with you. So we probably need to have committees of wise people, ethicists, and so forth, think about what these rules ought to be. And we shouldn't just leave it to software engineers. It shouldn't just be software engineers, and it shouldn't just be, you know, people who own large mega corporations that are good at technology, ethicists and so forth would be involved. There should be some assembly of wise people, as I was putting it, that tries to figure out what the rules ought to be. And those have to get translated into code. You can argue, or code or neural networks, or something. They have to be.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“First thing we have to do is to replace deep learning with deep understanding. So we need, you can't have alignment with a system that traffics only in correlations and doesn't understand concepts like bottles or harm. So Asima talked about these famous laws. And the first one was first do no harm. You can quibble about the details of Asmo's laws, but we have to if we're going to build real robots in the real world, have something like that. That means we have to program in a notion that's at least something like harm. That means we have to have these more abstract ideas that deep learning is not particularly good at. They have to be in the mix somewhere. I mean, you could do statistical analysis about probabilities of given harms or whatever, but you have to know what a harm is in the same way that you have to understand that a bottle isn't just a collection of pixels.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“There's a famous saying about being people overselling technology in the short run and underselling it in the long run. And so I actually end the book, or Ernie Davis and I end our book with an optimistic chapter, which kind of killed Ernie because he's even more pessimistic than I am. He describes me as a contrarian and him as a pessimist. But I persuaded him that we should end the book with a look at what would happen if AI really did incorporate, for example, the common sense reasoning and the nativism and so forth, the things that we counseled for. And we wrote it. And it's an optimistic chapter that AI suitably reconstructed so that we could trust it, which we can't now, could really be world-changing.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Found out some things to do with naked emperors. I mean, it's not like stuff is worthless. I mean, they're not really naked. It's more like they're in their briefs than everybody thinks. And so they are great at speech recognition. But the problems that I said were hard, because I didn't literally say the emperor has no clothes. I said, this is a set of problems that humans are really good at. And it wasn't couched as AI. It was couched as cognitive science. But I said, if you want to build a neural model of how humans do certain class of things, you're going to have to change the architecture. And I stand by those claims.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“This is what again I predicted in 1998 for that matter, Chomsky and Miller made the same prediction in 1963. I was just updating their claim for a slightly new tech. So those particular architectures that don't have any built-in knowledge, they're basically just a bunch of layers doing correlational stuff. They're not going to solve these problems.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And there's been advances in replacing sigmoids with other functions and so forth. There's all kinds of advances. But the fundamental architecture hasn't changed and the fundamental limit hasn't changed. And what I said then is kind of still true. And then here's a second example. I recently had a piece in Wired that's adapted from the book. And the book went to press before GP2 came out, but we describe this children's story and all the inferences that you make in this story about a boy finding a lost wallet. And for fun, in the wired piece, we ran it through GP2. GPT2 at something called talk to transformer.com, and your viewers can try this experiment themselves. Go to the WiredPiece that has the link and it has the story. And the system made perfectly fluent text that was totally inconsistent with the conceptual underpinnings of the story, right?”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“GPT2. And my example was something like a roses, a rose, a tulip is a tulip, a lily is a blank. And he got it to actually do that, which was a little bit impressive. And I wrote back and I said, that's impressive. But can I ask you a few questions? I said, was that just one example? Can it do it generally? And can it do it with novel words, which is part of what I was talking about in 1998 when I first raised the example? So ADAX is a DAX, right? And he sheepishly wrote back about 20 minutes later. And the answer was, well, it had some problems with those. So I made some predictions 21 years ago that still hold. In the world of computer science, that's amazing, right? Because there's a thousand or a million times more memory and, you know, computations a million times more operations per second, you know.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And I'll just give you two examples. One is a guy at DeepMind thought he had finally outfoxed me. At Zergi Lord, I think, is his Twitter handle And he said, he specifically made an example. Marcus said that such and such. He fed it into GP2, which is the AI system that is so smart that OpenAI couldn't release it because it would destroy the world, right? You remember that a few months ago. So he feeds it into.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm more worried about rebranding as a kind of political thing. So, I mean, what's going to happen, I think, is that deep learning is going to start to encompass symbol manipulation. So I think Hinton's just wrong. Hinton says we don't want hybrids. I think people will work towards hybrids and they will relabel their hybrids as deep learning. We've already seen some of that. So AlphaGo is often described as a deep learning system, but it's more correctly described as a system that has deep learning but also Monte Carlo Tree Search, which is a classical AI technique. And people will start to blur the lines in the way that IBM blurred Watson. First Watson meant this particular system and then it was just anything that IBM built in their cognitive division.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, you could build it in innately, or you could have your system watch a lot of films. If you can do this at all, but with a wide range of films, not just one film and one genre, but even if you could do it for all Westerns, I'd be reasonably impressed.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“My famous thing about the classic seven plots or whatever. I don't care if you want to build in the system, boy meets, girl, boy, loses girl, boy finds girl, that's fine. I don't mind having some headstone. Innate knowledge?”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, if you do it, there are lots of ways you could cheat. Like, you could build a Spartacus machine that works on that film. Like, that's not what I'm talking. I'm talking about you can do this with essentially arbitrary films from a large bibe.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Everybody or everybody minus one has to be lying. They can't all be Spartacus. We have enough common sense knowledge to know they couldn't all have the same name. We know that they're lying and we can infer why they're lying, right? They're lying to protect someone and to protect things they believe in. You get a machine that can do that. They can say, this is why these guys all got up and said, I am Spartacus. I will sit down and say AI has really achieved a lot. Thank you.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“human level or better than humans at answering kind of arbitrary questions you know why did this person pick up the stone what were they thinking when they picked up the stone were they trying to knock down glass and i mean ideally these wouldn't be multiple choice either because multiple choice is pretty easily gamed so if you could have relatively open-ended questions and you can answer why people are doing this stuff I would be very impressed and of course humans can do this right if you watch a well constructed movie and somebody picks up a rock everybody watching the movie knows why they picked up the rock right they all know oh my gosh he's gonna you know hit this character or whatever we have an example in the book about when a whole bunch of people say I am Spartacus you know this famous scene you know the viewers understand first of all that”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So in New Yorker, I proposed an alternative, I guess. And the one that I proposed there was a comprehension test Must like breaking back because I've already given you one breaking bad example. And in that article, I have one as well, which was something like if Walter, you should be able to watch an episode of Breaking Bad, or maybe you have to watch the whole series to be able to answer the question and say, if Walter White took a hit out on Jesse, why did he do that? So if you could answer kind of arbitrary questions about characters' motivations, I would be really impressed with that. I mean, you build software to do that. They could watch a film, or there are different versions. And so ultimately, I wrote this up with Praveen Peritosh in a special issue of AI magazine that basically was about the Turing Olympics. There were like 14 tests proposed. The one that I was pushing was a comprehension challenge, and Praveen, who's at Google, was trying to figure out how we would actually run it. And so we wrote a paper together. And you could have a text version too, or you could have an auditory podcast version. You could have a written version. But the point is that you win at this test if you can do, let's say,”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“For me personally, I would like to see a kind of comprehension that relates to what you just said. So I wrote a piece in The New Yorker in I think 2015, right after Eugene Gustman, which was a software package, won a version of the Turing test. And the way that it did this is it, well, the way you win the Turing test, so called Win It, is the Turing test is you fool a person into thinking that a machine is a person is you're evasive, you pretend to have limitations so you don't have to answer certain questions and so forth. So this particular system pretended to be a 13-year-old boy from Odessa who didn't understand English and was kind of sarcastic and wouldn't answer your questions and so forth. And so judges got fooled into thinking briefly with very little exposure as a 13-year-old boy. And it docked all the questions that Turing was actually interested in, which is like, how do you make the machine actually intelligent? So that test itself is not that good.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“With respect to cognitive science And there's a kind of intelligence to thinking about intelligence. I like to think that I have some of that, but social intelligence, I'm just okay. There are people that are much better at that than I am.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't think there's one good test. In fact, I tried to organize a movement towards something called a Turing Olympics. And my hope is that Francois is actually going to take Francois Chalet is going to take over this. I think he's interested in, and I just don't have place in my busy life at this moment. But the notion is there to be many tests and not just one because intelligence is multifaceted. There can't really be a single measure of it because it isn't a single thing. Just the crudest level, the SIT has a verbal component and a math component because they're not identical. And Howard Gardner has talked about multiple intelligence, like kinesthetic intelligence and verbal intelligence and so forth. There are a lot of things that go into intelligence. And, you know, people can get good at one or the other. I mean, in some sense, like every expert has developed a very specific kind of intelligence, and then there are people that are generalists. And, you know, I think of myself as a generalist.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, there's a field called biomimicry, and people do that for material science all the time. We should be doing the analog of that for AI. And the analog for that for AI is to look at cognitive science or the cognitive sciences, which is psychology, maybe neuroscience, linguistics, and so forth. Look to those for insight.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I think the way that we accelerate that process is we borrow from biology. Not slavishly, but I think we look at how biology has solved problems and we say does that inspire any engineering solutions here?”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Not logically. So, all the stuff that evolution did, good engineer might be able to do. So, for example, evolution made quadrupeds, which distribute the load across a horizontal surface. A good engineer could come up with that idea. I mean, sometimes good engineers come up with ideas by looking at biology. There's lots of ways to get your ideas. Part of what I'm suggesting is we should look at biology a lot more. We should look at the biology of thought and understanding and the biology by which creatures intuitively reason about physics or other agents or like how dogs reason about people. Like they're actually pretty good at it. If we could understand, my college we joked, dognition, if we could understand cognition well and how it was implemented, that might help us with our AI.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I mean, those aren't cumulative, right? That's just random. And part of the point that I'm making is that evolution is cumulative. So if you have... Billion monkeys independently, you don't really get anywhere. But if you have a billion monkeys, I think Dawkins made this point originally, or probably other people, but Dawkins made it very nice and either a selfish gene or blind watchmaker. If there is some sort of fitness function, it can drive you towards something. I guess that's Dawkins' point. And my point, which is a variation on that, is that if the evolution is cumulative on the related points, then you can start going faster.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Different parameters. Evolution does a lot of that, which means that the speed over time picks up. So evolution can happen faster because you have bigger and bigger libraries. And what I think has Happened in attempts at evolutionary computation is that people start with libraries that are very, very minimal, like almost nothing. And then progress is slow and it's hard for someone to get a good PhD thesis out of it and they give up. If we had richer libraries to begin with, if you were evolving from systems that had an originate structure to begin with, then things might speed up.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Once that vertebrate plan evolved, it spread everywhere. So fish have it and dogs have it and we have it. We have adaptations of it and specializations of it. But and the same thing with a primate brain plan. So monkeys have it and apes have it and we have it. So there are additional innovations like color vision and those spread really rapidly. So takes evolution a long time to get a good idea and being anthropomorphic and not literal here. But once it has that idea, so to speak, which caches out into one set of genes are in the genome, those genes spread very rapidly. And they're like subroutines or libraries, I guess the word people might use nowadays or be more familiar with. They're libraries that get used over and over again. So once you have the library for building something with multiple digits, you can use it for a hand, but you can also use it for a foot. You just kind of reuse the library with slightly.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I'll come to that in a second. It's inefficient, except that once it gets a good idea, it runs with it. So it took I guess a billion years, if I've been roughly a billion years. To evolve to a vertebrate.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“If the cap is over here, I get a different outcome. If the timing is different, if I put this here after I move that, then I get a different outcome that relates to causality. So obviously these mechanisms, whatever they are, can certainly communicate with each other.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't think we know for people how they're represented, and machines just don't really do this yet. So I think it's an interesting open question, both for science and for engineering. Some of it has to be at least interrelated in the way that the interfaces of a software package have to be able to talk to one another. The systems that represent space and time can't be totally disjoint because a lot of the things that we reason about are relations between space and time and cause. So I put this on and I have expectations about what's going to happen with the bottle cap on top of the bottle. And those span space and time.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, there's one question about what is biology give its creatures? What has evolved in our brains? How is that represented in our brains? The question I thought about in the book, The Birth of the Mind. And then there's a question of what AI should have. And they don't have to be the same. But I would say that it's pretty interesting set of things that we are equipped with that allows us to do a lot of interesting things. So I would argue or guess based on That children are born with the notion of space, time, other agent, places. And also, this kind of mental algebra that I was describing before no certain of causation, if I didn't just say that. So at least those kinds of things. They're like frameworks for learning the other things.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“One of my earlier books was actually trying to understand the biology of this. The book was called The Birth of the Mind. How is it the genes even build innate knowledge? And from the perspective of the conversation we're having today, there's actually two questions one is what innate knowledge or mechanisms or what have you people or other animals might be endowed with. I always like showing this video of a baby Ibex climbing down a mountain. That baby Ibex a few hours after its birth knows how to climb down a mountain. That means that it knows not consciously something about its own body and physics and 3D geometry and all of this kind of stuff.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Can I just say how much I like that question? You phrased it just right, and almost nobody ever does, which is what is the innate knowledge in what's learned along the way. So many people dichotomize it and they think it's nature versus nurture when it is obviously has to be nature and nurture. They have to work together. You can't learn this stuff along the way unless you have some innate stuff. But just because you have the innate stuff doesn't mean you don't learn anything. And so many people get that wrong, including in the field. Like, people think if I work in machine learning, the learning side, I must not be allowed to work on the innate side where that would be cheating. Exactly. People have said that to me.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“The intelligent agents, there's two intelligent agents. I've learned a lot by watching my two intelligent agents. I think that what's fundamentally interesting, well, one of the many things that's fundamentally interesting about them is the way that they set their own problems to solve. So two kids are a year and a half apart. They're both five and six and a half. They play together all the time and they're constantly creating new challenges. Like that's what they do is they make up games and they're like, well, what if this or what if that? Or what if I had this superpower? Or what if, you know, you could walk through this wall? So they're doing these what-if scenarios all the time. And that's how they learn something about the world and grow their minds. And machines don't really do that.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“That's true too. What I would say to you is your brain uses like 20 watts and it does a lot of things that deep learning doesn't do or the symbol manipulation doesn't do that AI just hasn't figured out how to do. So it's an existence proof that you don't need server resources that are Google scale in order to have an intelligence. I built with a lot of help from my wife two intelligences that are 20 watts each and far exceed anything that anybody else has built at a silicon.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Exactly. There's diminishing returns on more money, but nobody's going to argue if you want to give them more money, right? Except maybe the people who signed the giving pledge. And some of them have a problem. They've promised to give away more money than they're able to. But the rest of us, you know, if you want to give me more money, fine.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“But I'll put a proviso on that. More compute is always better. Nobody's going to argue with more compute. It's like having more money.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So there are some successes for classically. I think deep learning has been more successful. But my usual line about this, and I didn't invent it, but I like it a lot, is just because you can build a better ladder doesn't mean you can build a ladder to the moon. The bitter lesson is if you have a perceptual classification problem throwing a lot of data at it is better than anything else. But that has not given us any material progress in natural language understanding, common sense reasoning like a robot would need to navigate a home. Problems like that, there's no actual progress there.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“It's very successful. Everybody on the planet uses it now, like multiple times a day That's a measure of success, right? So, I mean, I don't think classical AI was wildly successful, but there are cases like that that excused all the time. Nobody even notices them because they're so pervasive.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Sure. I mean, I want to make a larger point. But on the narrower point, classical AI, Used, for example, in doing navigation instructions”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“But most effective at what? So they have been most effective for perceptual classification problems and for some reinforcement learning problems. He works on reinforcement learning.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think the people at AI2, Paul Allen's AI Institute are trying to do that. They're trying to build data sets that, well, they're not doing it for quite the reason that you say, but they're trying to build data sets that”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“With modern amounts of data and computed and maybe some advance in compute for that kind of compute might be great. Perspective on it is not that we want to resuscitate that stuff per se, but we want to borrow lessons from it and bring together with other things that we've learned”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“They don't do that with deep learning. They don't say nothing that existed in 2010 and there were many, many efforts in deep learning was really worth anything, right? I mean, really, there's no model from 2010 in deep learning. predecessors that deep learning that has any commercial value whatsoever at this point they're all failures um but that doesn't mean that there wasn't anything there i have a friend um i was getting to know him and he said he said i i had a company too i was talking about i had a new company he said i had a company too and it failed and i said well what did you do and he said deep learning and the problem was he did it in 1986 or something like that and we didn't have the tools then or 1990 we didn't have the tools then not the algorithms you know his algorithms weren't that different from modern algorithms but he didn't have the gpus to run it fast enough he didn't have the data and so it failed um it could be that you know symbol manipulation per se”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“There was people view that project as a failure. I think that they confuse the failure of a specific instance that was conceived 30 years ago for the failure of an approach, which they don't do for deep learning. So in 2010, people had the same attitude towards deep learning. They're like, this stuff doesn't really work. And, you know, all these other algorithms work better and so forth. And then certain key technical advances were made. But mostly it was the advent of graphics processing units that changed that. It wasn't even anything foundational in the technique. So there was some new tricks, but mostly it was just more compute and more data, things like ImageNet that didn't exist before, that allowed deep learning. And it could be to work. It could be that, you know, Psych just needs a few more things or something like Psych. But the widespread view is that that just doesn't work. And people are reasoning from a single example.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think we could. I think it hasn't been done right. The closest, and this is the asterisk, is the CYC psych system tried to do this. A lot of logicians worked for Doug Lennon for 30 years on this project. I think they stuck too closely to logic, didn't represent enough about probabilities, tried to hand code it. There are various issues. And it hasn't been that successful. That is the closest existing system to trying to encode this.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, most of it is not. Well, I'll give you an asterisk on this in a second, but most of it is not ever been encoded in machine interpretable form. And so, I mean, if you say accessible, there's two meanings of that. One is like, could you build it into a machine? Yes. The other is like, is there some database that we could go download and stick into our first thing? No.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source