YouSaid · the spoken record

Michael Littman

lines on the record: 118
first: 2020-12-13
most recent: 2020-12-13
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“That they got it to work, that they actually were able to leverage a whole bunch of different ideas, integrate them into one giant system, just the software engineering aspect of it is mind-blowing. I've never been a part of a program as complicated as the program that they built for that. And just the, you know, like Jerry Tissaro is a neural net whisperer, like, you know, David Silver is a kind of neural net whisperer too. He was able to coax these networks and these new way out there architectures to do these, you know, solve these problems that, as you said, when we were learning from AI. No one had an idea how to make it work. It was remarkable that”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, how is selfplay being used now? And why is it, does it feel like a more general powerful concept, sort of the idea of, well, the machine just going to teach itself to be smart?”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“If not in the title, it was definitely a term that he used. There's another term that we got from that work is rollout. So I don't know if you, do you ever hear the term rollout? That's a backamen term that has now applied generally in computers, well, at least in AI.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, because Tesaro's paper was something like training up an expert back end player through self play. So I think it was in the title of his paper.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“It's fine, but you know what? The thing you have to watch out for is you'll walk into a coffee shop once we can do that again.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“It's more compartmentalized. Don't be so worried. Like, it's like I get that you can be worried, but don't be so worried because we compartmentalize really well. And so it won't bleed into other parts of your life. You won't start, I don't know. Wearing red lipstick or whatever. Like, it's fine. It's fine.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“That wasn't the intent of the experiment. Just like social media, it wasn't intended as an experiment to see what we can take as a society, but it turned out that way.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Throw Taylor See, but I like them now because, again, I have no musical taste. Like now that I've heard Justin Bieber enough, I'm like, I really like his songs. Taylor Swift, not only do I like her songs, but my daughter's convinced that she's a genius. And so now I basically signed on to that.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“You were on fire. It started a war. Provocative So people Yeah, I don't get that because Emacs is clearly so much better. I don't understand. But you know, why do I say that? Because I spent a block of time in the 80s making my fingers know the Emacs keys. And now like, that's part of the thought process for me. Like I need to express. And if you take that, if you take my Emacs key bindings away, I become... I can't express myself.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And doesn't that prove that computers are super powerful and basically going to take over the world? It's like, no, Stamman is a hell of a hacker, right? So he was able to make the code do these amazing things. He couldn't have done it without the computer, but the computer couldn't have done it without him. And so I think people discount the role of people like Jerry, who”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Right, it's so easy. We're so drawn to the idea that it's the technology that is where the power is coming from, that I think we lose sight of the fact that sometimes you need a really good, just like, I mean, no one would think, hey, here's this great piece of software. Here's like, I don't know, GNU Emacs or whatever”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Learning with neural nets. And over and over and over again, we were failing. We couldn't get the good results that Jerry Tessaro got. I now believe that Jerry is a neural net whisperer. He has a particular...”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And then when he took the step from that to actually doing it as a full on reinforcement learning problem where you didn't need a trainer, you could just let it play. That was remarkable, right? And so I think as humans often do, as we've done in the recent past as well, people extrapolate. It's like, oh, well, if you can do that, which is obviously very hard, then obviously you could do all these other problems that we want to solve that we know are also really hard. And it turned out very few of them ended up being practical, partly because I think neural nets, certainly at the time, were struggling to be consistent and reliable. And so training them in a reinforcement learning setting was a bit of a mess. I had, I don't know, generation after generation of master students who wanted to do value function approximation, basically reinforcement learning.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, I found the TD Gammon result really just remarkable. So I had known about some of Jerry's stuff before he did TD Gammon. He did a system, just more vanilla, well, not entirely vanilla, but more classical back proppy kind of network for playing back admin where he was training it on expert moves. So it was kind of supervised. But the way that it worked was not to mimic the actions, but to learn internally an evaluation function. So to learn, well, if the expert chose this over this, that must mean that the expert values this more than this. And so let me adjust my weights to make it so that the network evaluates this as being better than this. So it could learn from From human preferences, it could learn its own preferences.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“At that point, I got to meet Rich Sutton. So everything was sort of downhill from there. And that was really the pinnacle of everything. But then I felt like I was kind of on the inside. So then as interesting results were happening, I could check in with Rich or with Jerry Tessaro, who had a huge impact on kind of early thinking in temporal difference learning and reinforcement learning and show that you could solve problems that we didn't know how to solve any other way. And so that was really cool. So, was good things were happening? I would hear about it from either the people who were doing it or the people who were talking to the people who were doing it. And so I was able to track things pretty well through the 90s”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“But we have some positive theoretical results for these things. You can come back at me. With, yeah, but they're really weak, and yeah, they're really weak. And you can even say that sorting algorithms, like if you do the optimal sorting algorithm, it's not really the one that you want. And that might be true as well.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Which is like, you're right, of course, but no, no, like, so what makes worst case so great, right? If you have a worst case analysis so great, is that you get modularity. You can take that thing and plug it into another thing and still have some understanding of what's going to happen when you click them together, right? If it just works well in practice, in other words, with respect to some distribution that you care about, when you go plug it into another thing, that distribution can shift and can change, and your thing may not work well anymore. And you want it to, and you wish it does, and you hope that it will, but it might not. And then, ah.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, and it sort of is. It seems they were firing off a lot of extra stuff supporting it. But nonetheless, the idea is really good. And as far as we know, it is. Very reasonable way of trying to create adaptive behavior, behavior that gets better at something over time.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Right. And TD separate difference in particular is about making predictions over time. And you can try to use it for making decisions, right? Because if you can predict how good a future action, an action outcomes will be in the future, you can choose one that has better. But the theory didn't really support changing your behavior. Like the predictions had to be of a consistent process if you really wanted it to work. And one of the things that was really cool about Q learning, another algorithm free enforcement learning, is it was off policy, which meant that you could actually be learning about the environment and what the value of different actions would be while actually figuring out how to behave optimally.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And so I was, you know, my mind was blown. And so Rich came and he gave a talk at Belcore. And he talked about what he was super excited, which was they had just figured out at the time, Q learning. So Watkins had visited the Rich Sutton's lab at UMass or Andy Barto's lab that Rich was a part of. And he was really excited about this because it resolved a whole bunch of problems that he didn't know how to resolve in the earlier paper. And so...”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“The idea, I got that they were using that he was using ideas that I was familiar with in the context of neural nets and backprop. But with this idea of making predictions over time, I'm like, this is so interesting, but I don't really get all the details I said to Dave. And Dave said, oh, well, why don't we have him come and give a talk? And I was like, wait, what? You can do that? Like, these are real people. I thought they were just words. I thought it was just like ideas that somehow magically seeped into paper. He's like, no, I know Rich. Like we'll just have him come down and he'll give a talk.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, the Perceptrons paper and Hinton, along with his student, Dave Ackley, and I think there was other authors as well, showed that, no, no, no, with Boltz machines, we can actually learn nonlinear concepts. And so everything's back on the table again. And that kind of started that second wave of neural networks. So Dave Ackley, he became my mentor at Bellcore. talked a lot about learning and life and computation and how all these things fit together. Now Dave and I have a podcast together. So I get to kind of enjoy that sort of his perspective once again, even all these years later. And so I said, so I said, I was really interested in learning, but in the concept of behavior. And he's like, oh, well, that's reinforcement learning here. And he gave me Rich Sutton's 1984 TD paper. So I read that paper. I honestly didn't get all of it.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And I was in a group with Dave Ackley, who was the first author of the Boltzmann machine paper. So the very first neural net paper that could handle XOR, right? So XOR sort of killed neural nets, the very first, the zeroth order. First winter.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Ah, that's right. So that's a good question. Right. So, this is, I think it wasn't actually talked about as behavior in the paper that I was reading. I think that it just talked about learning. And to me, learning is about learning to behave, but really neural nets at that point were about learning, like supervised learning. So learning to produce outputs from inputs. So I kind of tried to invent reinforcement learning. When I graduated, I joined a research group at Belcor, which had spun out of Bell Abs recently at that time because of the divestiture of long distance and local phone service in the 1980s, 1984”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Like really thick. There's a lot going on in there. And it talked about the reinforcement learning idea a little bit. I'm like, oh, that sounds really cool because behavior is what is really interesting to me about psychology anyway. So making programs that, I mean, programs are things that behave, people are things that behave. Like I want to make learning that learns to behave”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Sure, Gary's very feisty, and with his co author, they're kind of doing these kind of takedowns where they say, okay, well, yeah, it does all these amazing things, but here's a shortcoming. Here's a shortcoming, here's a shortcoming. And so the Pinkerprints paper is kind of like that generation's version of Marcus and Davis, right? Where they're trained as cognitive scientists, but they're looking skeptically at the results in the artificial intelligence neural net kind of world and saying, yeah, it can do this and this and this, but it can't do that and it can't do that and it can't do that. Maybe in principle or maybe just in practice at this point. But the fact of the matter is you've narrowed your focus too far to be impressed. You're impressed with the things within that circle, but you need to broaden that circle a little bit. You need to look at a wider set of problems. And so I was in this seminar in college that was basically a close reading of the Pinker Prince paper, which was”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, yeah. And so it was in a class by Richard Gehrig. He was kind of my favorite psych professor in college. And I took three different classes with him. And yeah, so they were talking specifically. The class, I think, was kind of a... Was a big paper that was written by Stephen Pinker and Prince. I'm blanking on Prince's first name, but Pinker and Prince, they wrote kind of a, they were at that time kind of like, I'm blanking on the names of the current people. The cognitive scientists who are complaining a lot about deep networks.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Was yeah, yeah, yeah. So I was a, I've always been a bit of a cognitive psychology groupie. So, I study computer science, but I like to hang around where the cognitive scientists are because I don't know, brains, man. They're like. They're wacky. Cool.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I grew up outside Philly, yeah. Yeah. So local schools were like Penn and Drexel and Temple. Everyone in my family went to temple at least at one point in their lives except for me. So yeah, Philly Philly family.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So I went to Yale. Princeton would have been way more convenient, and it was just a beautiful campus, and it was close enough to home. And I was really excited about Princeton. And I visited, I said, so computer science major, like, well, we have computer engineering. I'm like, oh, I don't like that word engineering. I like computer science. I really want to do like you're saying hardware and software? They're like, yeah, I'm like, I just want to do software. I couldn't care less about hardware.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Communing, possibly listening to the radio, listening to Billy Joel. That was the one album I had on vinyl at that time. And then I got it on cassette tape, and that was really helpful because then I could play it. I didn't have to go down to my parents' Wi-Fi or hi-fi, sorry. And at age 15, I remember kind of walking out and like, okay, I'm ready to talk to people again. Like I've learned what I need to learn here. And so yeah, so that was my home computer. And so I went to college and I was like, oh, I'm totally going to study computer science. And I opted the college I chose specifically had a computer science major. The one that I really wanted the college I really wanted to go to didn't.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I had a TS80 Model 1 before they were called Model 1s because there was nothing else. I got my computer in 1979. So I would have been Bar Mitzvah, but instead of having a big party that my parents threw on my behalf, they just got me a computer because that's what I really, really, re wanted. I saw them in the mall in Radio Shack and I thought. What? How are they doing that? I would try to stump them. I would give them math problems. Like one plus and then in parentheses, two plus one. And I would always get it right. I'm like, how do you know so much? Like, I've had to go to algebra class for the last few years to learn this stuff, and you just seem to know. So I was smitten and got a computer. And I think ages 13 to 15. I have no memory of those years. I think I just was in my room with the computer.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I've had the privilege, the pleasure of being of having almost a front row seat to a lot of this stuff. And it's been really, really fun and interesting. So when I was in college in the 80s, early 80s, the neural net thing was starting to happen. And I was taking a lot of psychology classes and a lot of computer science classes as a college student. And I thought, you know, something that can play tic-tac-toe and just like learn to get better at it, that ought to be a really easy thing. So I spent almost all of my what would have been vacations during college, like hacking on my home computer, trying to teach it how to play tic-tac-toe.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that's really interesting. And I guess I've become a believer in the human design, which I feel like I don't completely understand. Like, how do you make something? As robust as us. Like, we're so flawed in so many ways. And yet, and yet, you know, we dominate the planet and we do seem to manage to get ourselves out of scrapes eventually, not necessarily the most elegant possible way, but somehow we get to the next step. And I don't know how I'd make a machine do that. Generally speaking, like if I train one of my reinforcement learning agents to play a video game and it works really hard on that first stage over and over and over again and it makes it through it. It succeeds on that first level. And then the new level comes and it's just like, okay, I'm back to the drawing board. And somehow humanity, we keep leveling up and then somehow managing to put together the skills necessary to achieve success, some semblance of success in that next level too. And I hope we can keep doing that.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I believe, well, so this is my kid's age, right? And certainly my daughter's age, and she's very tapped in to social stuff, but she's also, she's trying to find that balance, right? Of participating in it and in getting the positives of it, but without letting it eat her alive. And I think sometimes she ventures. I hope she doesn't watch this Sometimes I think she ventures a little too far and is consumed by it, and other times she gets a little distance. And if there's enough people like her out there, they're going to navigate this choppy waters.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“How do we keep the good of this kind of technology without letting it eat us alive? And if they're successful, we move on to the next phase, the next level of the game. If they're not successful, then we're going to wreck each other. We're going to destroy society.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. So, all right, I do believe in the power of social media to screw us up royally. I do believe in the power of social media to benefit us too. I do think that we're in a Yeah, it's sort of almost got dropped on top of us, and now we're trying to, as a culture, figure out how to cope with it. There's a sense in which, I don't know, there's some arguments that say that, for example, I guess college aid students now, late college age students now, people who are in middle school when social media started to really take off, maybe really damaged. This may have really hurt their development in a way that we don't have all the implications of quite yet. That's the generation who, and I hate to make it somebody else's responsibility, but like they're the ones who can fix it.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Man, if we had a super intelligence that was in line with Wikipedia's values, It's a lot better than a lot of other things I could imagine. I trust Wikipedia more than I trust Facebook or YouTube as far as trying to do the right thing from a rational perspective. Now, that's not where you were going, and I understand that, but it does strike me that there's sort of smarter and less smart ways of. Exposing ourselves to each other on the internet”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So it's like, okay, that's maybe hard to argue against, but clearly 747s do get assembled. They get assembled by us. Basically, the idea being that there's a process by which we will get to the point of making technology that has that kind of awareness. And in that process, we're going to learn a lot about that process. And we'll have more ability to control it or to shape it or to... Build in on our own image, it's not something that is going to spring into existence like that 747, and we're just going to have to contend with it completely unprepared.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“That feels like it is against the laws of physics because these systems need help, right? They need to surpass the. The difficulty wall of complexity that happens in arranging something in the form that will happen in. Like, I believe in evolution. Like, I believe that there's an argument, right? So there's another argument, just to look at it from a different perspective, that people say, well, I don't believe in evolution. How could evolution, it's sort of like a random set of parts assemble themselves into a 747, and that could just never happen.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“But part of his story is also he's not going to put a date on it. It could be in a thousand years. It could be in a hundred years. It could be in two years. It's just that as long as we keep making this kind of progress, it's ultimately has to become a concern. I kind of am on board with that, but the thing that the piece that I feel like is missing from that way of extrapolating from the moment that we're in is that I believe that in the process of actually developing technology that can really get around in the world and really process and do things in the world in a sophisticated way, we're going to learn a lot about what that means, which that we don't know now because we don't know how to do this right now. If you believe that you can just turn on a deep learning network and it eventually give it enough compute and it'll eventually get there, well, sure, that seems really scary because we won't be in the loop at all. We won't be helping to design or target these kinds of systems. I don't see that.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Really natural to apply that same idea to AI. You see these systems that are doing some pretty remarkable computational tricks, demonstrations, and then to take that idea and just push it all the way to the limit and think, okay, where does this go? Where is this going to take us next? And if you're a deep believer in the power of ideas, then it's really natural to believe that those ideas could be taken to the extreme and kill us. So, I think his strength is also his undoing because that doesn't mean it's true, like it doesn't mean that that has to happen, but it's natural for him to think that.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think he said, but then he came to Providence, Rhode Island, which is where I live, and said to the governors of all the states, you know, you're worried about entirely the wrong thing. You need to be worried about AI. You need to be very, very worried about AI. journalists kind of reacted to that. They wanted to get people's people's take. And I was like, okay, my belief is that one of the things that makes Elon Musk so successful and so remarkable as an individual. Is that he believes in the power of ideas? He believes that you can have, you can, you know, if you have a really good idea for getting into space, you can get into space. If you have a really good idea for a company or for how to change the way that people drive, you just have to do it and it can happen”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“It'll be way more powerful than us, and we will be toast. So there's some, I don't know, very smart people who have signed on to that story. And it's a compelling story. I once, now I can really get myself in trouble. I once wrote an op-ed about this specifically responding to some quotes from Elon Musk, who has been on this very podcast more than once.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, sure, sure, sure. So, as I understand it, for example, I read Bostrom's book and a bunch of other reading material about this sort of general way of thinking about the world. And I think the story goes something like this, that we will at some point create computers that are smart enough that they can help design the next version of themselves, which itself will be smarter than the previous version of themselves. And eventually bootstrapped up to being smarter than us, at which point we are essentially at the mercy of this sort of more powerful intellect, which in principle, we don't have any control over what its goals are. And so if its goals are at all out of sync with our goals, like for example, the continued existence of humanity, we won't be able to stop it.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“One is super intelligence argument and the existential threat of AI is one where I feel pretty confident in my feeling about that one. Like I'm willing to hear other arguments, but like I am not particularly moved by the idea that if we're not careful, we will accidentally create a superintelligence that will destroy human life.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So I try very hard to see things from multiple perspectives, but There's this great Calvin and Harps cartoon where do you know, okay, so Calvin's dad is always kind of a bit of a foil and he was he talked Calvin into Calvin had done something wrong. The dad talks him into like seeing it from another perspective and Calvin this breaks Calvin because he's like oh my gosh now I can see the opposite sides of things and so it becomes like a cubist cartoon where there is no front and back everything's just exposed and it really freaks him out and finally he settles back down. It's like, oh, good, no, I can make that go away. But like I'm that I'm that. I live in that world where I'm trying to see everything from every perspective all the time. So there are some things that I've formed opinions about that I would be harder, I think, to disavow me of.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So I wasn't planning to dance. They had me in the studio and they gave me the jacket. And it's like, well, you can't, if you have the jacket and the glove, like there's not much you can do.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, it's a great song. So the lyrics are great because, first of all, it rhymes. Not all songs rhyme. I've done Rolling Stone songs, which turn out to have no rhyme scheme whatsoever. They're just sort of yelling and having good time, which makes it not fun from a parody perspective because, like you can say anything. But the lines rhymed and there was a lot of internal rhymes as well. And so figuring out how to sing with internal rhymes a proof of the halting problem was really challenging. And I really enjoyed that process.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source
“No, that was really fun. I wrote the lyrics really quickly, and then I gave it over to the production team. They recruited an a cappella group to sing. It went really smoothly. It's great having a team because then you can just focus on the part that you really love, which in my case is writing the lyrics. For me, the most challenging one, not challenging in a bad way, but challenging in a really fun way, was I did one of the parody songs I did is about the halting problem in computer science, the fact that you can't create a program that can tell for any other arbitrary program whether it's actually going to get stuck in an infinite loop or whether it's going to eventually stop. And so I did it to an 80 song because that's I hadn't started my new thing of learning current songs. And it was Billy Joel's The Piano Man.”
2020-12-13 · Lex Fridman Podcast · #144 – Michael Littman: Reinforcement Learning and the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source