YouSaid · the spoken record
John Hopfield
- lines on the record
- 76
- first
- 2020-02-29
- most recent
- 2020-02-29
- sittings or episodes
- 1
- sources
- podcast
Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections
“Which goes under the general name of associative memory. And a large part of intelligent behavior is actually just large associated memories at work as far as I can see”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“Okay, what you mean by associative memory is... Ah, you have a memory of Each of your friends your friend has all kinds of properties from what they look like what their voice sounds like to where they went to college, where you met them go on and on what science papers they've written. If I start talking about a Cognitive scientists has got a very bad back. It doesn't take very long for you to say, oh, he's talking about Jeff Hinton. I never mentioned the name Or anything very particular. Somehow a few facts are associated with a particular person enables you to get hold of the rest of the facts. Or not the rest of them, another subset of them. And it's this ability to Link things together. Link experiences together.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“At the same time the simple ones will conquer big aspects. And I think one of the most biggest surprises to me was... How well learning systems was your manifestly non biological. How important they can be useful they can be in AI.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“If biology uses them, it's going to take us to more generations of things for people to actually dig in and see how they are used and what they mean.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“The scale of 10 to the 10. I can't remember the number either. And all the transistors are somewhat similar, and most physical systems are That many parts, all of which are similar, have collective properties. Sound waves and air, earthquakes, what have you collective properties, weather? There are no collective properties used in artificial neural networks, in AI.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“It's going to be a long time before that kind of thing. Which can take place in the In large complex networks of things is actually used in the computation. Look, the How many transistors are there on your laptop these days?”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“Sentences, it might be a couple more. And going back to my brain waves, as it were, From the AI point of view, they would say, ah, maybe these are an abenomen and not important at all. The first car I had, a real wreck of a 1936 dodge Of 45 miles an hour and the wheels with Jimmy. Good speedometer there. Now, nobody designed the car that way. The car is malfunctioning to have that. But in biology, if it were useful to know when are you going more than 45 miles an hour, you just capture that and you wouldn't worry about where it came from.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it's going to go on. Generation after generation, the way it has where what you might call the AI computer science community says, Let's take the following. This is our model of neurobiology at the moment. Let's pretend it's good enough and do everything we can with it. And it does interesting things, and after the while it sort of grinds into the sand. Ah something else needed for neurobiology and some other grand thing comes in. and enables you to go a lot further. We'll go into the sand again, and I think it will convey generations of this evolution. I don't know how many of them, and each one is going to get you further into what a brain does. In some sense past the touring chest longer and more broad aspects. And how many of these are going to have to be before you say? I've made something, I've made a human. I don't know”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“Suppose you ask a neurosur When has somebody dead? probably go back to saying well I can look at the brain rhythms Tell you this is a brain which is never going to function again. This other one is one which if we treat it well is still recoverable and then just do that by some electrodes looking at simple electrical patterns. Don't look in any detail at all at what individual neurons are doing. His are utterly absent from anything which goes on at Google.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“answer that in two ways i think You look at real systems. Feedback is an essential aspect of how these real systems compute. On the other hand, if I have a mathematical system with feedback, I know I can unlayer this and do it. But I have an exponential expansion in the amount of stuff I have to build if I'm going to solve the problem that way”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“There's a question of what do you mean by understand? When I taught freshman physics, I used to say I wanted to give physics to understand this subject, to understand Newton's laws. I didn't want them simply to memorize a set of examples to which they knew the equations to write down to generate the answers. I had this nebulous idea of understanding. So that if you looked at a situation, you could say Oh I expect the ball to make that trajectory I expect some intuitive notion of understanding and I don't know how to express that very well, and I've never known how to express it well. And you run smack up against it when you look at these simple neural nets, feed forward neural nets which Do amazing things, and yet you know contain nothing of the essence of what I would have felt was understanding.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“Both of my parents were physicists, and the real thing I got out of that was a feeling that The world is an understandable place And if you do enough experiments, then think about what they mean and structure things that you can do the mathematics of the relevant to the experiments, you ought to be able to understand how things work.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, of course, I see the world always through a lens of physics. I grew up in physics. And the way I pick problems is very characteristic of physics and of an intellectual background which is not psychology, which is not chemistry and so on and so on.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“They are, and of course There are times of immense cell multiplication, there are also times of the greatest cell death in the brain. during infancy.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“You can do there's the developmental neurobiology which understands how the And the structure evolves. A combination of what the genetics is like and the real, the fact that you're building a system in three dimensions.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“I find things most interesting that I begin to see how How to get into the edges of them and tease them apart a little bit and see how they work. And since I can't see the evolutionary process going on, I am in awe of it. But I find it just a black hole as far as trying to understand what to do. And so in a certain sense, I'm in awe of it, but I couldn't be interested in working on it.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“Very difficult to understand how to do computationally. On the other hand, it can't do simple floating point arithmetic, so it's awfully stupid.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“But when you talk to a computer scientist about neural networks, it's all math. The fact that biology actually came about from evolution and the fact that Biology is about a system which you can build in. three dimensions. You look at computer chips. Computer chips are basically two dimensional structures 2.1 dimensions, but they really have difficulty doing three-dimensional wiring. Biology is neocortex is actually also sheet-like, and it sits on top of white matter, which is about 10 times the volume of the gray matter, and contains all what you might call the wires. There's a huge. The effect of computer structure on what is easy and what is hard is immense. And biology does, they make some things easy that are”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“Adaptation is Everything when you get down to But there are differences between adaptation where your learning goes on only over generation, but over evolutionary time, your learning goes on at the time scale of one individual who must learn from the environment during that individual's lifetime. And biology has both kinds of learning in it. And the thing which makes neurobiology hard is that The mathematical system as it were built on this other kind of evolutionary system.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“And companies have difficulty. A new product competing with an old product. And, um... IBM built its first PC. You probably read the book. They made a little isolated internal unit to make the PC, and for the first time in IBM's history, they didn't insist that you built it out of IBM components. But they understood that they could get into this market for them. Which is a very different thing by completely changing their culture Biology finds other markets and In a more adaptive way.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“And there's evolutionary pressure to improve. Look, there isn't in computers too, but improvement has to do with closing some companies and opening some others. Evolutionary process looks a little different.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, look, in the biological molecule level You have a piece of DNA which would encode for a particular protein. You could duplicate that piece of DNA and now One part of it can code for that protein, but the other one could itself change a little bit and thus start coding for a molecule which is slightly different. Now if that molecule was slightly different had it a function which helped any old chemical reaction was as important to the cell. would go ahead and let that try and evolution would slowly improve that function. And so you have the possibility of Duplicating and then having things drift apart, one of them retained the old function, the other one do something new for you.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“While they fire together, you can be sure other cells are going to notice it. So you can make a computational feature out of this in an evolving brain. artificial neural networks don't even have action potentials, let alone have the possibility for synchronizing them.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“And the engineers made a simple-minded mistake. When you walk at step step and it's back and forth motion, but when you walk, it's also right foot, left foot, side to side motion. And it's a side-to-side motion for which the bridge was strong enough, but it wasn't. It wasn't stiff enough. As a result, you could feel the motion and you'd fall into stuff with it. And people were very uncomfortable with it. They closed the bridge for two years while they built stiffening for it. Nerve cells produce action potentials. If you have a bunch of cells which are loosely coupled together producing action potentials at the same rate, there'll be some circumstances under which these things can lock together Other circumstances which they wound”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“They can. Look, let me take one of the things that I used to do research on. If you take things which Oscillate have rhythms which are sort of close to each other under some circumstances these things will have a phase transition and suddenly the rhythm will everybody will fall into step. There was a marvelous physical example of that. In the Millennium Bridge across the Thames River until about 2001, and pedestrians walking across pedestrians don't walk, synchronize, they don't walk in lockstep, but they all walk at about the same frequency, and the bridge could sway at that frequency, and the slight sway made pedestrians tend a little bit lock into a while the bridge was oscillating back and forth and the pedestrians were walking in stepped to it. And you could see it in the movies made out of the bridge.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source
“One of the things that very much intrigues me is the fact that Neurons have all kinds of trees. Components, properties to them. evolutionary biology if you have some little quirk into how a molecule works or how a cell works and it can be made use of evolution will sharpen it up and make it into a useful feature rather than a glitch And so you expect in neurobiology for evolution to have captured all kinds of possibilities of getting neurons of how you get neurons to do things for you. And that aspect has been completely suppressed in artificial neural networks.”
2020-02-29 · Lex Fridman Podcast · #76 – John Hopfield: Physics View of the Mind and Neurobiology · IDENTIFIED FROM THE TRANSCRIPT · source