YouSaid · the spoken record

Dileep George

lines on the record: 139
first: 2020-08-14
most recent: 2020-08-14
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Yeah, I think so because it's basically being able to understand the machinery of the world such that you can pursue whatever goals you want, right?”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Interesting because I mean, I'm a doubly undergrad, so I got lucky in that way. But I think it does have some of the right ingredients because you learn about circuits, you learn about how you can construct circuits to approach do functions. You learn about microprocessors. You learn information theory. You learn signal processing. You learn continuous math. So in that way, it's a good step. If you want to go to computer science or neuroscience, it's a good step.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, this was a question in one of the Redwood Neuroscience Institute workshops that Jeff Hawkins organized almost 10 years ago. This question was put to a panel, right? What should be the undergrad major? You should take if you want to understand the brain. And the majority of opinion in that one was electrical engineering.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Being an experimental neuroscientist might not be the way to go about it. A better way to pursue it might be through computer science, electrical engineering, machine learning, and AI. And of course, you have to study up the neuroscience, but that you can do on your own. If you are more attracted by finding something intriguing about discovering something intriguing about the brain, then of course it is better to be an experimentalist. So find that motivation. What are you intrigued by? And of course, find your strengths too. Some people are very good experimentalists and they enjoy doing that.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Sure. I think every piece of advice should be taken with a pinch of salt, of course, because each person is different. Their motivations are different. But I can definitely say if your goal is to understand the brain from the angle of wanting to build one, then...”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“It's a good analogy to have. Of course, all analogies have their merits. So people in AI often use airplanes as an example of, hey, we didn't learn anything from birds. Look. But the funny thing is that the saying is airplanes don't flap wings, right? This is what they say. The funny thing and the ironic thing is that you don't need to flap to fly is something Wright Brothers found by observing birds. In their notebook, in some of these books, they show that notebook drawings, they make detailed notes about buzzards, just soaring over thermals. And they basically say, look, flapping is not the important propulsion is not the important problem to solve. Solve here. We want to solve control. And once you solve control, propulsion will fall into place. All of these are people. They realize this by observing birds.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And the third one History, it's the name of the book, I think, is Bishop's Boys. It's about Wright Brothers and their path and how there are multiple books on this topic and all of them are great. It's fascinating how flight was treated as an unsolvable problem. And also what aspects did people emphasize? People thought, oh, it is all about just powerful engines. Just need to have powerful lightweight engines. And so some people thought of it as how far can we just throw the thing, just throw it. Catapult. So it's very fascinating. And even after they made the invention, people not believe. Leaving it.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, yeah, so I would recommend probabilistic reasoning in intelligent systems. Another book I liked was one from Doug Hofstadter. This was a long time ago. He has a book, he had a book, I think, call it, it was called The Mind's Eye. It was probably Hufstarter and Daniel Dennett together”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So that was in his later book. So, this was the first book, Probabilistic Reasoning in Intelligence Systems. He mentions causality, but he hadn't really sunk his teeth into how do you actually formalize it. And the second book, causality, the one in 2000, that one is really hard. So I wouldn't recommend that.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So I definitely liked Judy Pearl's book, Probabilistic Reasoning and Intelligent Systems. It's a very deep technical book. But what I liked is that so there are many places where you can learn about probabilistic graphical models from. But throughout this book, JDAPool kind of sprinkles his philosophical observations and he thinks about connects to how the brain thinks and attentions and resources, all those things. So that whole thing makes it more interesting to read.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, and that is part of our drives. And that's why I'm not too worried about AI having motivations to kill all humans and those kinds of things. Why? Just wait. So why do you need to do that?”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“You can think of it like this. So suppose you invent a way to freeze people for a long time. It's not dying. So you can be frozen and woken up thousands of years from now. So it's no fear of death”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, it is a part of human cognition, but there is no reason for that mortality to come to the equation for a artificial system, because we can copy the artificial system. The problem with humans is that I can't clone you. Even if I clone you as a hardware, your experience that was stored in your brain, your episodic memory, all those will not be captured in the new clone. But that is not the same with an AI system, right?”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Things vicariously just in your brain. And similarly, you can experience another person's thoughts by having a model of how that person works and running, putting yourself in some other person's shoes. So that is being vicarious. Now, it's the same modeling apparatus that you're using to model the external world or some other person's thoughts. turn it to yourself. You can upload if that same modeling thing is applied to your own modeling apparatus, then that is what gives rise to consciousness, I think.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Vicadius, you know. So Vicadius is the company name. And so what does Vicarious mean? At the first level, it is about modeling the world. And it is internalizing the external actions. So you interact with the world and learn a lot about the world. And now after having learned a lot about the world, you can run Those things in your mind without actually having to act in the world.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“It is not outside the Ralam, but it doesn't on a day to day basis inform what we do, but it's more, so in many ways the company name is connected to this idea of consciousness.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So the surgery part of it, the biology part of it, the long term repercussions part of it. Again, I don't know what else will, we often find after a long time in biology that, okay, that idea was wrong. So people used to cut off the gland called the thymus or something. And then they found out, oh no, that actually causes cancer.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Then it is brain adapting to, and of course, your external system is design such that it is adaptable, just like we design computers or mouse, keyboard, all of them to be interacting with humans. So, of course, that feedback system is designed to be human compatible. But now it is not trying to record from all of the brain and now two systems trying to adapt to each other. It's the brain adapting into one way.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Exactly. That's the way I think that I find that to be a problem more promising way. It's basically saying, you know, okay, attach electrodes to some part of the cortex and make sure maybe if it is done from birth, the brain will adapt it such that that part is not damaged. It was not used for anything. These electrodes are attached there, right? And now you train that part of the brain to do this high bandwidth communication between something else, right? And if you do it like that,”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it's very, very hard because you also do not know what will happen to the brain with that. In the sense of how does the brain adapt to something like that?”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Correct, surgically implanted electrons. So I think of it as a very, very promising field, especially when it is helping people overcome some limitations. Now, at some point, of course, it will advance the level of being able to communicate.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Good steps to take, and they have enormous applications, people losing limbs, being able to control prosthetics, quadruplegics being able to control something. And therapeutics, and I also know about another company working in the space called Paradromics, based on a different electrode array, but trying to attack some of the same problems. So I think it's a very”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So I think BCI is a cool research area. And in fact, when I got interested in brains initially, so I was enrolled at Stanford. And when I got interested in brains, it was through a brain computer interface talk that Krishna Shanoy gave. That's when I even started thinking about the problem. So it is definitely a fascinating research area and it is the applications are enormous, right? There is a science fiction scenario of brains directly communicating. Let's keep that aside for the time being. Even just the intermediate milestones they are pursuing, which are very reasonable as far as I can see, being able to control an external limb using a direct connections from the brain and being able to write things into the brain. So those are all”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Knowledge base. Yes, it is possible that you can put many of the structures into neural networks and we will find ways of combining properties of neural networks and graphical models. So, I mean, it's already started happening. Graph neural networks are kind of emerged between them And there will be more of that thing. But to me, it is the direction, looking at biology and the history of evolutionary history of intelligence, it is pretty clear that, okay, what need is more structure in the models and modeling of the world and supporting dynamic inference?”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Putting back into the cortex and then putting it back into the cortex, of course, affects what you're going to see next in your current situation.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, this is again hypothesis, right? We're making this. So when you come to a new situation, your Cortex is doing inference over in the new situation. And then, of course, hippocampus is connected to different parts of the cortex. And you have this deja vu situation. Okay, I have seen this thing before. And then in the hippocampus, you can have an index of, okay, this is when it happened as a timeline. And then you can use the hippocampus to drive the similar timelines to say now rather than being driven by my current input simuli, I am going back in time and rewinding my experience from there.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Statistical causal structural model that you built over time. So it's basically the idea is that the hippocampus is just storing or sequencing in a set of pointers that happens over time. And then, whenever you want to reconstitute that memory and evaluate the different aspects of it, whether it was good, bad, do I need to encounter the situation again, you need the cortex to reinstantiate, to replay that memory.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And so you need to be able to index that similarity using your other giant, the model of the world that you have learned. Although the situation came from the episode, you need to be able to index the other one. So the episodic memory being implemented as an indexing over the other model that you're building.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, at least it's very clear that you need to have two kinds of memory. That's very, very clear, which is there are things that happen as statistical patterns in the world. But then there is the one timeline of things that happen only once in your life. And this day is not going to happen ever again. And that needs to be stored as a just a stream of string, right? This is my experience. And then the question is about how do you take that experience and connect it to the statistical part of it? How do you now say that, okay, I experienced this thing, now I want to be careful about similar situations.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So, yeah, we have another piece of work which came out recently on how do you form episodic memories and form abstractions from them. And we haven't figured out all the connections of that to the overall cognitive architecture.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“And also causality, being able to do counterfactual reasoning, being able to do intervention, which is actions in the world. So all those things require different kinds of models to be built. I don't think transformers captures that family. It is very good at statistical modeling of text. And it will become better and better with more data, bigger models. But that is only going to get so far. Finally, so I had this. And come up with hypothesis and revise the hypothesis based on evidence from experiments, all those things. Those are the things that we want the system to do when we have AGI, not solve with simple puzzles.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, so Transformer is still a feat for word neural network. It has a very interesting architecture which is good for text modeling and probably some aspects of video modeling, but it is still a feat for architecture.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Unless somebody has written on all the details about how everything works in the world to the absurd amounts like, okay, it is easier to walk forward than backward, that you have to open the door to go out of the thing. Doctors wear underwear. Unless all these things somebody has written down somewhere or somehow the program found it to be useful for compression from some other text, the information is not there.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“But that is not a model of the world. It's a model of the text world and it will have interesting properties and it will be useful. But just scaling it up is not going to give us AGI or natural language understanding or meaning.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, they will look more impressive than GPT3. So if you take that to the extreme, then a Markov chain or just first order, if you go to, I'm taking it the other extreme. If you read Shannon's book, he has a model of English text which is based on first Markov chains, second order Markov chains, third order Markov chains and saying that, okay, third Markov chains look better than faster Markov chains. So does that mean a faster Markov chain has a model of the world? Yes, it does. So yes, in that level, when you go high order models or more sophisticated structure in the model like the transformer networks have, yes, they have a model of the text world”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Think will happen less than GPT 3 because it is trained on even more data and it can do even more longer distance coherence. But it will still have the fundamental limitations that it doesn't have a world model and it can't run simulations in its head to find whether something is true in the world or not.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So I haven't directly evaluated it yet. From what I have seen on Twitter and other people evaluating it, it looks very intriguing. I am very intrigued by some of the properties it is displaying. And of course the text generation part of that was already evident in GPT2 that it can generate coherent text over long distances. But of course the weaknesses are also pretty visible in saying that okay it is not really carrying a world state around. And sometimes you get sentences like I went up the hill to reach the valley or the thing completely incompatible statements or when you're traveling from one place to the other it doesn't take into account the time of travel, things like that. So those things”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Correct. Correct. And it will be in some simple world initially, but it will be about, okay, can the system connect the language and ground it in the right way and run the right simulations to come up with the answer?”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“One of the problems that we are working on, and the way we are approaching that is basically saying, okay, you need to, so the takeaway is that language is simulation control and your perceptual plus motor system is building a simulation of the world. And so that's basically the way we are approaching it. And the first thing that we built was a controllable perceptual system. And we built a schema networks, which was a controllable dynamic system. Then we built a concept learning system that puts all these things together into programs, substractions that you can run and simulate. And now we are taking the step of connecting it to language. It will be very simple examples initially. It will not be the GPTE3-like examples, but it will be grounded simulation-based language.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Can do a lot of it. You can still do a lot of it by quick associations without having to go into the depth. Most of the time you will be right. You can just do quick associations, but I can easily create tricky situations for you where that quick association is wrong and you have to actually run the simulation.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Exactly, exactly. So here I post a question in natural language. The answer to that question was you got the answer from actually simulating the scene. Now I can go more and more detail about, okay, was Sally standing on something while doing this? Could she have been standing on a light bulb to do this? I could ask more and more questions about this and I can ask make you simulate the scene in more and more detail. Where is all that knowledge that you're accessing stored? It is not in your language system. It is not, it was not just by reading text you got that knowledge. It is stored from the everyday experiences that you have had from and by the by the age of five you have pretty much all of this. And it is stored in your visual system, motor system, in a way such that you can be accessed through language.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So I have this classic example I give. So suppose I give you a few sentences and then ask you a question following that sentence. This is a natural language processing problem, right? So here it goes. I'm telling you Sally pounded a nail on the ceiling. Okay. That's a sentence. Now I am asking you a question. What's the nail horizontal or vertical? Okay, how did you answer that?”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Everything is open, right? No problem is solved, solved. I think of perception as kind of the first thing that you have to build. But the last thing that you will be actually solved, because if you do not build perception system in the right way, you cannot build concept system in the right way. So you have to build a perception system. However wrong that might be, you have to still build that and learn concept from there and then keep iterating. And finally, perception will get solved fully when perception cognition language, all those things work together finally.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“The lower level kind of makes it feel like okay, that's unimportant, like it's more like I would say the concepts in the visual and motor system and the concept learning system, which if you cut off the language part, just what we learn by interacting with the world and abstractions from that, that is a prerequisite for any real language understanding.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So I want to make a distinction between concepts that are just learned from text. By just feeding brute force text, you can start extracting things like, okay, cow is likely to be on grass. So those kinds of things, you can extract purely from text. But that's kind of a symbol association thing rather than a concept as an abstraction of something that happens in the real world in a grounded way that I can simulate it in my mind and connect it back to the real world.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“So think of it like this suppose you wanted to tell a new person that you met, you don't know the language or that person uses, you want to communicate to that person to achieve some task. So I want to say, hey, you need to pick up all the red cups from the kitchen counter and put it here. How do you communicate that? You can show pictures. You can basically say, look, this is the starting state. The things are here. This is the ending state. And what does the person need to understand from that? The person need to understand what conceptually happened in those pictures from the input to the output, right? So we are looking at pre-verbal conceptual understanding without language. How do you have a set of concepts that you can manipulate in your head?”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“We have made a pass through multiple of the steps that, as I mentioned earlier, we were looking at perception from the angle of cognition. It was not just perception for perception's sake. How do you connect it to cognition? How do you learn concepts? And how do you learn abstract reasoning similar to some of the things Francois talked about? So we have taken one pass through it, basically saying, What is the basic cognitive architecture that you need to have which has a perceptual system, which has a system that learns dynamics of the world, and then has something like a routine program learning system on top of it to learn concepts. So we have built the version 0.1 of that system. This was another science robotics paper.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Correct. On the inference set, on the recognition side, I am much more amenable to being constrained because it's much easier to do experiments because the stimulus, how many steps did it get to take the answer? I can trace it back. I can understand the speed of that computation, et cetera, much more readily on the inference side.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“Could be it's just that you don't know enough to, on the learning side, you don't know enough to say that is definitely not the way the brain does it.”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source
“It doesn't have to be differentiable. Yeah, but you have to have a model that you start with. You have data comes in and you have to have a way of adjusting the model such that it better fits the data. So that is all of learning. And some of them can be using backprop to do that. Some of it can be using very local graph changes to do that. Many of these learning algorithms have similar update properties locally in terms of what the neurons need to do locally”
2020-08-14 · Lex Fridman Podcast · #115 – Dileep George: Brain-Inspired AI · IDENTIFIED FROM THE TRANSCRIPT · source