YouSaid · the spoken record

Jay McClelland

lines on the record: 125
first: 2021-09-20
most recent: 2021-09-20
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Dave was a standout undergraduate. He went at a younger age than most people do to college at the University of South Dakota and majored in mathematics. And I don't know how he got interested in psychology, but he applied to the mathematical psychology program at Stanford and was accepted as a PhD student to study mathematical psychology at Stanford. So mathematical psychology is the use of mathematics to model Mental processes”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“So Dave came from a little tiny town in western South Dakota And his mother was the librarian and his father was the editor of the newspaper. And I know one of his brothers pretty well. They grew up, there were four brothers, and they grew up together, and their father encouraged them to compete with each other a lot. They competed in sports and they competed in mind games, you know, I don't know, things like Sudoku and chess and various things like that.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“And these systems are doing that now, right? So they are in terms of the parallel idea that we were talking about before, they are doing this massively parallel computation in the sense that Of the neurons in each of those layers is thought of as computing its little bit of Something about the input simultaneously with all the other ones in the same layer. Get to the point of abstracting that away and thinking, oh, it's just one whole vector that's being computed, one activation pattern that's computed in a single step. And that abstraction is useful, but it's still that parallel. And distributed processing right. Each one of these guys is just contributing a tiny bit to that whole thing”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Millions in each layer, but we're getting sort of similar in a certain way, right? And then we think, okay, at the bottom level, there's an array of things that are like the photoreceptors in the eye. They respond to the amount of light of a certain wavelength at a certain location on the pixel array. So that's like the biological eye. And then there's several further stages going up, layers of these neuron-like units And you go from that raw input, array of pixels to a classification. Actually, built a system that could do the same kind of thing that you and I do when we open our eyes and we look around and we see there's a cup, there's a cell phone, there's a water bottle.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, yes, I agree with you. There's still this sort of sense. Not sort of knowing how we kind of get all the way there, I think. And this very much remains at the core of the questions that everybody's asking about the capabilities of deep learning and all these kinds of things. But if I could just play this out a little bit. Convolutional neural network or a CNN, which many people may have heard of, is Set of, you could think of it biologically as. A set of Collections of neurons. Each one neurons in it, but there's many layers, right? Some of these things are hundreds or even a thousand layers deep, but others are closer to the biological brain and maybe they're like 20 layers deep or something like that. So we have within each layer we have Thousands of neurons or tens of thousands maybe, well, in the brain, we probably have”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“So the idea is that each, you know, our brains have, oh, look, a hundred or hundreds, almost a billion of these little neurons, right? And they're all capable of doing their work at the same time. So it's like instead of just a single central processor that's engaged in chug one step after another. Billion of these little computational units working at the same time.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Would that be fair to say? Well, yeah, the parallel, the word parallel in this. Comes from the idea that each neuron is An independent computational unit, right? It gathers data from other neurons. Integrates it in a certain way, and then it produces a result. And it's a very simple little computational unit, but it's autonomous in the sense that You know, it does its thing, right? It's in a biological medium where it's getting nutrients and various chemicals from that medium. But it's, you know, you can think of it as almost like a little computer in and of itself.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“And so Hinton was one of the first people who came and joined this group of postdoctoral scholars that was funded by this wonderful grant that they got. Another one who's also well known in Neural Network Circus Circles is Paul Smolensky. He was another one of that group. Anyway. Jeff and Jim Anderson organized a conference at UCSD where we were. And it was called parallel models of associative memory and it brought all the people together who had been thinking about these kinds of ideas in 1979 or 1980. And this began to kind of... really resonate with some of Rummel Hart's Own thinking, some of his reasons for wanting something other than the kinds of computation he'd been doing so far. So let me talk about Rummel Hart now for a minute with that context.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“In terms of a neural network, it will help me answer the questions about the mind that I'm trying to answer. And that really excited me. I think that a lot of people were becoming excited about that. And one of those people. Was Jim Anderson, who I had mentioned, another one was Steve Grossberg, who had been writing about neural networks since the 60s. And Jeff Hinton was yet another. And his PhD dissertation showed up in an applicant pool. A postdoctoral training program that Dave and Don, the two men I mentioned before, Rummelhart and Norman, were administering. And Rummelhart got really excited about Hinton's PhD dissertation.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“But by then, a professor in a psychology department who had used Linear algebra to create neural network models of perception and categorization and memory. And I just blew me out of the water that one could, you know, create a model. That was simulating neurons, not just kind of Engaged in a stepwise algorithmic process that was construed abstractly. But it was. Simulating, remembering, and recalling and recognizing the prior occurrence of a stimulus or something like that. So for me, this was. A bridge between the mind and the brain. And I just like and I. I remember I was walking across campus one day in 1977 and I almost felt like St. Paul on the road to Damascus. I said to myself, you know, if I think about the mind,”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“A series of chapters exploring interesting questions about cognition, but in a completely sort of abstract, you know, non-biological kind of way. And I'm saying, gee, this is amazing. I'm coming to this community where people can get together and feel like they've collectively exploring. Know ideas. And it was a book that had a lot of, I don't know, lightness to it. And, you know, Don Norman, who was the more senior figure to Rummelheart at that time, who led that project, you know, always created this spirit of playful exploration of ideas. And so I'm like, wow, this is great. I was also, you know, still trying to get from. The neurons to the The cognition I realized at one point I got this opportunity to go to a conference where I heard a talk by a man named James Anderson who was an engineer.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm going to start sort of with my own. Process in the mid 70s and then into the late 70s when I met Jeff Hinson, and he came to San Diego. And we were all together. My time in graduate schools, I've already described to you, I had this sort of feeling of, okay, I'm really interested in human cognition, but Disembodied sort of way of thinking about it that I'm getting from the current mode of thought about it isn't working fully for me. And when Got my assistant professorship. I went to UCSD Was in 1974. Something amazing had just happened. Dave Rummelhart had written a book together with another man named Don Norman. And the book was called Explorations in Cognition. And it was”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, ultimately, um... That is a very important part of the whole process of understanding this sort of. Emergence of mind from brain kind of thing”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“The stages of our mental abilities, we seem to have a certain kind of mindset at a certain age. And then at another age where we look at that four-year-old and say, oh my God, how could they have thought that way? So Piaget was known for this kind of stage theory of child development, right? And you look at it closely and suddenly those stages aren't so discreet and the transitions, but the difference between the four-year-old and the seven-year-old is profound. That's another thing that's always interested me is how we Something happens over the course of several years of experience where at some point we reach the point where Something like an insider, a transition, or a new stage of development occurs. Know these kinds of things can be understood. In complex systems research. And so, um, Evolutionary biology, developmental biology”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Know there's so many fascinating aspects to these kinds of things. We think of evolution as continuous, right? We think, oh, yes, okay, over 500 million years, there could have been this. Relatively continuous changes. And, um, But that's not Anthropologists, evolutionary biologists found from the fossil record. They found Hundreds of years of millions of years of stasis. And then, you know, suddenly a change occurs. Well, suddenly on that scale is. Million years or something Or even 10 million years, but. But the concept of punctuated equilibrium was a very important concept in evolutionary biology. Also feels somehow right about.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. And, you know, so Chomsky, um, Argued. That some. Genetic fluke occurred 100,000 years ago. And just happened that some Human, some hominin. Predecessor of current humans had this. One genetic tweak that resulted in language. And language then provided. This special thing that separates us from all other animals. Think there's a lot of truth to the value and importance of language. But I think it comes along with the evolution of a lot of other related things related to sociality and mutual engagement with others. Establishment of I don't know. Rich mechanisms for Organizing an understanding of the world. which language then plugs into.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“In the field, I think people have felt for a long time. In the study of vision, the continuity between humans and non-human animals has been second nature for a lot longer. I had this conversation with somebody who's a vision scientist and you're saying, oh, we don't have any problem with this. You know, the monkeys visual system and the human visual system are extremely similar. To certain levels, of course, they diverge after a while, but the first visual pathway from the eye to the brain and the first few layers of cortex or cortical areas, I guess one would say, are extremely similar”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. So that's absolutely right. And a slightly different career path that would have been equally interesting to me would have been to actually study the process of embryological development. Flowing on into brain development and the The exquisite sort of laying down of pathways and so on that occurs in the brain. And I know the slightest bit about that. It's not my field, but there are... Fascinating aspects to this process that eventually result in the You know, the complexity of various brains, at least, you know, one thing where”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Oh, you know, somebody comes. Oh, there's a certain part of the brain that's still different. They don't, you know, there's no hippocampus in the monkey brain. It's only in the human brain. Huxley had to do a surgery in front of many, many people in the late 19th century to show to them that there's actually a hippocampus in the chimpanzee sprain, you know. So their continuity of the species is another element that contributes to this sort of That we are ourselves a total product of nature. And that, to me, is the magic and the mystery how. How nature could actually... Know give rise to Organisms that have the capabilities that we have.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“He would dream about the complexity of the eye and the arguments that people had given about how ridiculous it was to imagine that that could have ever emerged from some sort of, you know, Unguided process, right? That it hadn't been the product of design. And so he didn't publish for a long time, in part because he was scared of his own ideas. He didn't think they could possibly be true. But then, you know, by the time. 20th century rolls around we all You know, we understand that many people understand or believe that evolution produced the entire range of animals that there are. And, you know, Descartes' idea starts to seem a little wonky after a while, right? Like, well, wait a minute. There's the apes and the chimpanzees and the bonobos and, you know, like, they're pretty smart in some ways.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, obvious and not obvious at the same time. And I think about Darwin in this context too, because Darwin. Knew very early on that none of the ideas that anybody had ever offered. Gave him a sense of understanding how evolution could have worked. But he wanted to figure out how it could have worked. That was his goal. He spent a lot of time working on this idea and coming, you know, reading about things that gave him hints and thinking they were interesting but not knowing why and drawing more and more pictures of different birds that differ slightly from each other and so on. And then he figured it out. But after he figured it out, he had nightmares about it.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“When I was an undergraduate, I learned about the physiological mechanisms. And so when I'm studying cognitive psychology as a first year PhD student, I'm saying, wait a minute, the whole thing is biological.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Were these little hydraulic tubes that if you touch something that would be pressure and it would send a signal to pressure to the other parts of the system and that would cause action? So he had a mechanistic theory of animal behavior. And he thought that the human had this animal body. But that some divine something else had to have come down and been placed in him to give him the ability to think. Right The physical world includes the body in action, but it doesn't include thought according to Descartes, right? And so the study of physiology at that time was the study of sensory systems and motor systems and things that you could directly measure when you stimulated neurons and stuff like that. The study of cognition was something that was tied in with abstract computer algorithms and things like that.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“I used to think, or I used to talk about the idea of awakening from the Cartesian dream. So Descartes, Thought about these things right he was walking in the gardens of Versailles one day and he stepped on a stone. And a statue moved. And he walked a little further, he stepped down another stone and another statue moved. And he like. Why did the statue move when I stepped on the stone? And he went and talked to the gardeners and he found out that they had a hydraulic system. That allowed Physical contact with the stone to cause water to flow in various directions, which caused water to flow under the statue and move the statue. And he used this as the beginnings of a theory about how animals And he had this notion that these little fibers that people had identified that weren't carrying the blood.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source
“Fundamental thing I think about with neural networks is how they allow us to link With Mysteries of thought. When I was first entering the field myself in the late 60s, early 70s Cognitive psychology had just become a field. There was a book published in 1600. Um, And the author said You know, the study of the nervous system was only of peripheral interest. It wasn't going to tell us anything about the mind. Didn't agree with that. I always felt, oh, look, I'm a physical being. From dust to dust, you know, ashes to ashes. And somehow I emerged from that.”
2021-09-20 · Lex Fridman Podcast · #222 – Jay McClelland: Neural Networks and the Emergence of Cognition · IDENTIFIED FROM THE TRANSCRIPT · source