YouSaid · the spoken record

Gary Marcus

lines on the record: 109
first: 2019-10-03
most recent: 2019-10-03
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Most of it we would recognize if somebody said it, if it was true or not, but we wouldn't think to say that it's true or not.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Virtual home, where, and we talk about this in the book. Find the exact example there, but people were asked to do things like describe an exercise routine. And the things that the people describe are very low level and don't really capture what's going on. So go to the room with a television and the weights, turn on the television, or press the remote to turn on the television, lift weight, put weight down or whatever. It's like very micro level and it's not telling you what an exercise routine is really about, which is like, I want to fit a certain number of exercises in a certain time period. I want to emphasize these muscles. You want some kind of abstract description. The fact that you happen to press the remote control in this room when you watch this television isn't really the essence of the exercise routine. But if you just ask people like, what did they do? Then they give you this fine grain. And so it takes a level of expertise about”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“They're going to tell you more exotic things. And they're all well and good, but they're not getting to the root of the problem. So untutored humans aren't very good at knowing and why should they be, what kind of knowledge the computer system developers actually need. I don't think that that's an irremediable problem. I think it's historically been a problem. People have had crowdsourcing efforts and they don't work that well. There's one at MIT. We're recording this at MIT.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Some domains, it's not that hard, but it's often really hard. Partly because a lot of the Things that are important, people wouldn't bother to tell you. So if you pay someone on Amazon Mechanical Turk to tell you stuff about bottles, they probably won't even bother to tell you some of the basic level stuff that's just so obvious to a human being and yet so hard to capture in machines.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And very explicitly so. So you'd have somebody interview an expert and then try to turn that stuff into rules. And at some level, I'm arguing for rules. But the difference is those guys did in the 80s was almost entirely rules, almost entirely handwritten with no machine learning. What a lot of people are doing now is almost entirely one species of machine learning with no rules. And what I'm counseling is actually a hybrid. I'm saying that both of these things have their advantage. So if you're talking about perceptual classification, how do I recognize a bottle? Deep learning is the best tool we've got right now. If you're talking about making inferences about what a bottle does, something closer to the expert systems is probably still the best available alternative and probably we want something that is better able to handle quantitative and statistical information than those classical systems typically were. So we need new technologies that are going to draw some of the strengths of both the expert systems.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Sure. So, I mean, first, I just want to clarify I'm not endorsing expert systems per se. You've been kind of contrasting them. There is a contrast, but that's not the thing that I'm endorsing. Expert systems tried to capture things like medical knowledge with a large set of rules. So if the patient has this symptom and this other symptom, then it is like...”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Were transmitted and you just machine learned it would be insane or to build a web browser by taking logs of keystrokes and images, screenshots, and then trying to learn the relation between them. Nobody would ever, no rational person would ever try to build a browser that way. They would use symbol manipulation, the stuff that I think AI needs to avail itself of in addition to deep learning.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“He'd thrown at him But, I mean, you can't have it both ways. You can't be like, I don't know what to throw out, but I am going to throw out the symbols. I mean, and not just the symbols, but the variables and the operations are variables. Don't forget, the operations over variables, the stuff that I'm endorsing and which John McCarthy did when he founded AI, that stuff is the stuff that we build most computers out of. There are people now who say, we don't need computer programmers anymore, not quite looking at the statistics of how much computer programmers actually get paid right now. We need lots of computer programmers, and most of them, you know, they do a little bit of machine learning, but they still do a lot of code, right? Code where it's like, you know, if the value of x is greater than the value of y, then do this kind of thing, like conditionals and comparing operations over variables, like there's this fantasy you can machine learn anything. There's some things you would never want to machine learn. I would not use a phone operating system that was machine learned. Like you made a bunch of phone calls and you recorded which packets”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, you know, Hinton said that to Axios, and I had a friend who interviewed him and tried to pin him down on what exactly we need to throw out, and he was very evasive.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Form of that problem remains that what they learn is really correlations between different input and output nodes. They're complex correlations with multiple nodes involved and so forth. But ultimately they're correlative. They're not structured over these operations over variables. Now someday people may do a new form of deep learning that incorporates that stuff and I think it will help a lot. And there's some tentative work on things like differentiable programming right now that fall into that category. But the sort of classic stuff like people use for ImageNet doesn't have it and you have people like Hinton going around saying symbol manipulation like what Marcus, what I advocate is like the gasoline engine. It's obsolete. We should just use this cool electric power that we've got with a deep learning. And that's really destructive because we really do need to have the gasoline engine stuff that represents, I mean, I don't think it's a good analogy, but we really do need to have the stuff that represents symbols.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“In that day, that they would never ever make this generalization. And it's not that the networks were stupid. It's that they see the world in a different way than we do. They were basically concerned what is the probability that the rightmost output node is going to be 1? And as far as they were concerned, in everything they'd ever been trained on, it was a zero. That node had never been turned on. And so they figured, why turn it on now? Whereas a person would look at the same problem and say, well, it's obvious. We're just doing the thing that corresponds. The Latin for it is mutatis mutatis. Well, change what needs to be changed. And we do this. This is what algebra is. I can do f of x equals y plus 2, and I can do it for a couple of values. I can tell you if y is three, then x is 5, and if y is 4, x is 6. And now I can do it with some totally different number, like a million. They can say, well, obviously it's a million and two because you have an algebraic operation that you're applying to a variable. And deep learning systems kind of emulate that, but they don't actually do it. The particular example you could fudge a solution to that particular problem.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“This is an astute question, but I think the answer is at least partly no. One of the kinds of classical neural network architectures that we call an auto associator, it just tries to take an input, goes through a set of hidden layers, and comes out with an output, and it's supposed to learn essentially the identity function that your input is the same as your output. So you think of this binary numbers, you've got the one, the two, the four, the eight, the 16, and so forth. And so if you want to input 24, you turn on the 16, you turn on the 8. It's like binary 1, 1, and bunch of zeros. So I did some experiments in 1998 with the precursors of contemporary deep learning. And what I showed was you could train these networks on all the even numbers, and they would never generalize to the odd number. A lot of people thought that I was, I don't know, an idiot or faking the experiment or wasn't true or whatever, but it is true that with this class of networks that we had.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Knowledge. It is an example of having a machine learn something, but it's a machine that learns a particular kind of thing, which is object classification. It's not a particularly good algorithm for learning about the abstractions that govern our world. There may be such a thing. Part of what we counsel in the book is maybe people should be working on devising such things.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I was with you with almost everything you said except the phrase deep learning. What I think you really want there is a new form of machine learning. So let's remember deep learning is a particular way of doing machine learning. Most often it's done with supervised data for perceptual categories. There are other things you can do with deep learning, some of them quite technical, but the standard use of deep learning is I have a lot of examples and I have labels for them. So here are pictures. This one's the Eiffel Tower. This one's the Sears Tower. This one's the Empire State Building. This one's a cat. This one's a pig and so forth. You just get millions of examples, millions of labels. And deep learning is extremely good at that. It's better than any other solution that anybody has devised, but it is not good at representing abstract knowledge. It's not good at representing things like bottles contained liquid and half tops to them and so forth. It's not very good at learning or representing that kind of.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Based on years of watching this stuff and making predictions 20 years ago that still hold, even though there's a lot more computation and so forth, is that we actually have to do a different kind of hard work, which is more like building a design specification for what we want the system to do, doing hard engineering work to figure out how we do things like what Jan did for convolution in order to figure out how to encode complex knowledge into the systems. The current systems don't have that much knowledge other than convolution, which is, again, this, you know, objects being in different places and having the same perception, I guess I'll say. People don't want to do that work. They don't see how to naturally fit one with the other.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Use the next frame basically in order to tell it what's going on. And he thinks that's the royal road, and he's willing to put in the work in devising that algorithm. Then he wants the machine to do the rest. And again, I understand the impulse. My intuition is”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And nobody wants to do that hard work, even Ernie didn't want to do that hard work. Everybody would rather just feed their system in with a bunch of videos with a bunch of containers and have the systems infer how containers work. It would be like so much less effort. Let the machine do the work. And so I understand the impulse. I understand why people want to do that. I just don't think that it works. I've never seen anybody build a system that in a robust way can actually watch videos and predict exactly which containers would leak and which ones wouldn't or something. I know someone's going to go out and do that since I said it and I look forward to seeing it. But getting these things to work robustly is really, really hard. So Jan Lakun, who was my colleague at NYU for many years, thinks that the hard work should go into defining an unsupervised learning algorithm that will watch videos.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, and that's part of why people love it, right? But I always think of this quote from Bertrand Russell, which is it has all the advantages of theft over honest toil. It's really hard to program into a machine a notion of causality or even how a bottle works or what containers are. Ernie Davis and I wrote a, I don't know, 45-page academic paper trying just to understand what a container is, which I don't think anybody ever read the paper, but it's a very detailed analysis of all the things, not even all some of the things you need to do in order to understand a container. It would be a whole lot nicer. I'm a co-author in the paper. I made it a little bit better, but Ernie did the hard work for that particular paper. And it took him like three months to get the logical statements correct. And maybe that's not the right way to do it. It's a way to do it. But on that way of doing it, it's really hard work to do something as simple as understanding containers.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Kind of reify things and make them more abstract. And so what you'd really wind up with, if you don't program that in advance, is a system that kind of realizes that this is the same thing as this. But then I take your little clock there and I move it over and it doesn't realize that the same thing applies to the clock.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“A skeptic that that kind of emergence per se can work. So I think that deep learning might play a role in systems that do what I want systems to do, but it won't do it by itself. I've never seen a deep learning system really extract an abstract concept. What they do, principal reasons for that stemming from how backpropagation works, how the architectures are set up. One example is deep learning people actually all build in something like build in something called convolution, which Jan Lacun is famous for, which is an abstraction. They don't have their systems learn this. So the abstraction is an object that looks the same if it appears in different places. And what Lacun figured out and why essentially why he was a co-winner of the Turing Award was that if you programmed this innately, then your system would be a whole lot more efficient. In principle, this should be learnable, but people don't have systems that...”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Objects are material things in the physical world. So, like, I can make some inferences if I know that vessels need to Not have holes in them, then I can infer that in order to carry their contents, then I can infer that a bottle shouldn't have a hole in order to carry its contents. So you can do hierarchical inference and so forth. And we say that's great, but it's only a tiny piece of what you need for common sense. We give lots of examples that don't fit into that. So another one that we talk about is a cheese grater. You've got holes in a cheese grater. You've got a handle on top. You can build a model in the game engine sense of a model so that you could have a little cartoon character flying around through the holes of the crater. But we don't have a system yet. Taxonomy doesn't help us that much that really understands why the handle is on top and what you do with the handle or why all of those circles are sharp or how you'd hold the cheese with respect to the grader in order to make it actually work.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“A lot of people aren't actually going to read my book. But if they did read the book, one of the things that might come as a surprise to them is that we actually say common sense is really hard and really complicated. So my critics know that I like common sense, but that chapter actually starts by us beating up not on deep learning, but kind of on our own home team, as it will. So Ernie and I are first and foremost people that believe in at least some of what good old-fashioned AI tried to do. So we believe in symbols and logic and programming, things like that are important. And we go through why even those tools that we hold fairly dear aren't really enough. So we talk about why common sense is actually many things. And some of them fit really well with those classical sets of tools. So things like taxonomy. So I know that a bottle is an object or it's a vessel, let's say, and I know a vessel is an object.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So you know that bottles are correlated in your data with bottle caps, but you don't understand that there's a thread on the bottle cap that fits with the thread on the bottle and that that tightens in. If I tighten enough that there's a seal and the water can come out, like there's no machine that understands that. And having a good cognitive model of that kind of everyday phenomena is what we call common sense. And if you had that, then a lot of these other things start to fall into at least a little bit better place. Right now, you're like learning correlations between pixels when you play a video game or something like that. And it doesn't work very well. It works when the video game is just the way that you studied it and then you alter the video game in small.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“It's a very good question. And I'm going to be evasive because I think that they go together a lot. So some of them might be solved independently of others. But I think a good solution to AI starts by having real, what I would call cognitive models of what's going on. So right now we have an approach that's dominant where you take statistical approximations of things, but you don't really understand them.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I'm not saying humans are infinitely general or that humans are perfect. I just said a minute ago. It's a low bar, but it's just a low bar. But right now, like the bar is here and we're there and eventually we'll get way past it.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“What I mean by in general is that you could transfer the knowledge you learn in one domain to another. So if you learn about bank robberies in movies and there's chase scenes, then you can understand that amazing scene in Breaking Bad when Walter White has a car chase scene with only one person. He's the only one in it. And you can reflect on how that car chase scene is like all the other car chase scenes you've ever seen and totally different and why that's cool”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I mean, that would be like a first step in the right direction, but obviously that's not what it really means. You're kidding.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, if you want to make an argument that humans are constrained in what they can understand. I have no issue with that. I think that's right. But it's still not the same thing at all as saying here's a system that can play Go. It's been trained on 5 million games. And then I say, can it play on a rectangular board rather than a square board? And you say, well, if I retrain it from scratch on another 5 million games, it can, that's really, really narrow, and that's where we are. We don't have even a system that could play go and then without further retraining, play on a rectangular board, which any good human could do with very little problem. So that's what I mean by narrow. And so it's just wordplay to say...”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't think that makes sense. We have lots of narrow intelligences for a specific problem. But the fact is, like anybody can walk into, let's say, a Hollywood movie and reason about the content of almost anything that goes on there. So you can reason about what happens in a bank robbery or what happens when someone is infertile and wants to, you know, go to IVF to try to have a child or you can, you know, the list is essentially endless. And, you know, not everybody understands every scene in the movie, but there's a huge range of things that pretty much any ordinary adult can understand.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Strengths of brute force computation combined with kind of subtlety and understanding medicine that a good doctor or scientist has.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm saying I prefer general intelligence. I mean, human level intelligence is a real thing, and you could try to make a machine that matches people or something like that. I'm saying that per se shouldn't be the objective, but rather that we should learn from humans the things they do well and incorporate that into our AI, just as we incorporate the things that machines do well, that people do terribly. So, I mean, it's great that AI systems can do all this brute force computation that people can. And one of the reasons I work on this stuff is because I would like to see machines solve problems that people can't that combine the strength in order to be solved would combine the strengths of machines to do all this computation with the ability, let's say, of people to read. So I'd like machines that can read the entire medical literature in a day. 7,000 new papers or whatever is the numbers comes out every day. There's no way for any doctor or whatever to read them all. Machine that could read would be a brilliant thing. And that would be”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“It'd be great if you could just be like, I'm going to wipe this sector. You know, I'm done with that. I didn't have fun last night. I don't want to think about it anymore. Bye-bye. Gone. But we can't.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think you could design a better memory system. You could argue about utility functions and how you want to think about that. But objectively, it would be really nice to do some of the following things to get rid of memories that are no longer useful. Objectively, that would just be good. And we're not that good at it. So when you park in the same lot every day, you confuse where you park today with where you parked yesterday, with where you parked the day before and so forth. So you blur together a series of memories. There's just no way that that's optimal. I mean, I've heard all kinds of wacky arguments people trying to defend that. But in the end of the day, I don't think any of them hold water.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And then we do some nice stuff with that. So, my free associations are different from yours, and you're kind of amused by them, and that's great, and hence poetry. So there are lots of ways in which we take a lousy situation and make it good. Another example would be our memories are terrible. So we play games like concentration where you flip over two cards, try to find a pair. Can you imagine a computer playing that? Computers, like, this is the dullest game in the world. I know where all the cards are. I see it once. I know where it is. What are you even talking about? We make a fun game out of having this terrible memory.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I've heard a lot of arguments like this. I've never found them that convincing. I think that there's a lot of making lemonade out of lemons. So we, for example, do a lot of free association where one idea just leads to the next and they're not really that well connected. And we enjoy that and we make poetry out of it and we make kind of movies with free associations and it's fun and whatever. I don't think that's really a virtue of the system. I think that the limitations in human reasoning actually get us in a lot of trouble. Like for example, politically we can't see eye to eye because we have the motivational reasoning I was talking about and something related called confirmation bias. We have all of these problems that actually make for a rougher society because we can't get along because we can't interpret the data in shared ways.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Absolutely. I'll say two things about that. One is I was on a panel with Danny Conneman, the Nobel Prize winner last night, and we were talking about this stuff. And I think we converged on is that humans are a low bar to exceed. They may be outside of our skill right now, but as AI programmers, but eventually AI will exceed it. So we're not talking about human-level AI. We're talking about general intelligence that can do all kinds of different things and do it without some of the flaws that human beings have. The other thing I'll say is I wrote a whole book actually about flaws of humans. It's actually a nice bookend to the counterpoint to the current book. So I wrote a book called Kluge, which was about the limits of the human mind. Current book is kind of about those few things that humans do a lot better than machines.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I like the term general intelligence, so I don't think that the ultimate AI, if there is such a thing, is going to look just like humans. I think it's going to do some things that humans do better than current machines, like reason flexibly and understand language and so forth. But it doesn't mean they have to be identical to humans. So, for example, humans have terrible memory and they suffer from what some people call motivated reasoning. So they like arguments that seem to support them and they dismiss arguments that they don't like. There's no reason that a machine should ever do that.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“We can't really predict is the full scope of where AI will be in a certain period. I mean, I think it's safe to say that although I'm very skeptical about current AI, that it's possible to do much better. There's no in-principled argument that says AI is an insolvable problem, that there's magic inside our brains that will never be captured. I mean, I've heard people make those kind of arguments. I don't think they're very good. So, AI is going to come. In probably 500 years of planning to get there. And then once it's here, it really will change everything.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, here's some things that we can safely predict, I suppose. We can predict that AI will be faster than it is now. It will be cheaper than it is now. It will be better in the sense of being more general and applicable in more places. It will be pervasive I mean, these are easy predictions. I'm sort of modeling them in my head on Jeff Bezos' famous predictions. He says, I can't predict the future, not in every way. I'm paraphrasing. But I can predict that people will never want to pay more money for their stuff. They're never going to want it to take longer to get there. So you can't predict everything, but you can predict some things. Sure, of course, it's going to be faster and better.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Phil Can I first justify the arrogance before you try to push me beyond it? There are examples like people figured out how electricity worked. They had no idea that that was going to lead to cell phones. Things can move awfully fast once new technologies are perfected. Even when they made transistors, they weren't really thinking that cell phones would lead to social networking.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“It's really, really hard. Well, it's really, really hard for machines, for linguists, people trying to understand it. It's not that hard for children. And that's part of what's driven my whole career, right? I was a student of Stephen Pinker's and we were trying to figure out why kids could learn language when machines couldn't.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“The table, your brother thinks that maybe your mom is wrong to think that you think that I think, right? We can make it in sentences of infinite length or we can stack up adjectives. This is a very silly example, a very, very silly example, a very, very, very, very, very, very silly example and so forth. There are good arguments that there's an infinite range of sentences. In any case, it's vast by any reasonable measure. And for example, almost anything in the physical world we can talk about in the language world. And interestingly, many of the sentences that we understand we can only understand if we have a very rich model of the physical world. So I don't ultimately want to adjudicate the debate that I think you just set up, but I find it interesting. Maybe the physical world is even more complicated. than language. I think that's fair.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“There's something to what you say in some ways in which I disagree. One interesting thing about language is that it abstracts away. This bottle, I don't know if it will be in the field of view, is on this table. And I use the word on here, and I can use the word on here. Maybe not here, but there's that one word encompasses in analog space, a sort of infinite number of possibilities. So there is a way in which language filters down the variation of the world. And there's other ways. So, you know, we have a grammar. More or less, you have to follow the rules of that grammar. You can break them a little bit, but by and large, we follow the rules of the grammar. And so that's a constraint on language. So there are ways in which language is a constrained system. On the other hand, there are many arguments that say there's an infinite number of possible sentences, and you can establish that by just, you know, stacking them up. So I think there's water on the table. You think that I think there's water on the table. Your mother thinks that you think that I think the water is on the table.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“About the exponential as you look at the combinations of moves, but fundamentally the Go Board has 361 squares. That's it. Those intersections are the only places that you can place your stone. Whereas when you're reading, the next sentence could be anything. It's completely up to the writer what they're going to do next.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, you know, you read media accounts and it's like, ooh, AI, it must, you know, it's magical or can solve any problem. Well, no, some problems are really accessible, like Jess and Go. And other problems like reading are completely outside the current technology. And it's not like you can take the technology that drives AlphaGo and apply it to reading and get anywhere. DeepMind has tried that a bit. They have all kinds of resources. They built AlphaGo. I wrote a piece recently they lost, and you can argue about the word lost, but they spent $530 million more than they made last year. So they're making huge investments. They have a large budget. And they have applied the same kinds of techniques to reading or to language. It's just much less productive there because it's a fundamentally different kind of problem. Chess and Go and so forth are closed in problems. The rules haven't changed in 2500 years. There's only so many moves you can make. You can talk.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I finally mentioned that in the context of this conversation because Kasparov and I are working on an article that's going to be called AI is not magic. And neither one of us thinks that it's magic. And part of the point of this article is that AI is actually a grab bag of different techniques. And some of them each have their own unique strengths and weaknesses.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think a game like chess particularly where it's, you know, you have perfect information, it's two-player closed end, and there's more computation for the computer. It's no surprise the machine wins. I mean, I'm not sad when a computer calculates a cube root faster than me. I know I can't win that game. I'm not going to try.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“There was no point in my playing deep blue. I mean, it's a waste of deep blues computation. I played Casparov because we both gave lectures at this same event and he was playing 30 people. I forgot to mention that. Not only did he crush me, but he crushed 29 other people at the same time.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I'm frustrated with my game against him last year because I played him. I had two excuses. I'll give you my excuses up front that it won't mitigate the outcome. I was jet lagged and I hadn't played in 25 or 30 years. But the outcome is he completely destroyed me and it wasn't even close.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source
“That is very accessible. There's some things that are going to be obvious and some not. So I don't think anybody really can do this well yet, but I think it's not inconceivable to imagine machines in the not so distant future being able to understand that if people lose in a game that they don't like that's not such a hard thing to program and it's pretty consistent across people. Most people don't enjoy losing and so that makes it relatively easy to code. On the other hand, if you wanted to capture everything about frustration, well, people get frustrated for a lot of different reasons. They might get sexually frustrated. They might get frustrated. They can get their promotion at work. All kinds of different things. And the more you expand the scope, the harder it is for anything like the existing techniques to really do that.”
2019-10-03 · Lex Fridman Podcast · Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI · IDENTIFIED FROM THE TRANSCRIPT · source