YouSaid · the spoken record

Adam Marblestone

lines on the record: 144
first: 2025-12-30
most recent: 2025-12-30
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“And then what we have in common with a mouse. And so we do know at some level that the differences between us and a chimpanzee or something, and that includes the social instincts and the more advanced differences in cortex and so on, it's a tiny number of genes that go into these additional amount of making the eight layer transformer instead of the six-layer transformer or tweaking that reward function.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't know. I mean, but I think you might be able to talk to biologists about this to some degree because you can say, well, We just have a ton in common. I mean, we have a lot in common with yeast from a gene's perspective. Yeast is still used as a model for some amount of drug development and stuff like that in biology. And so much of the genome is just going towards you have a cell at all. It can recycle waste. It can get energy. It can replicate.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“And how much genomic real estate do those genes take up versus the ones that specify visual cortex versus auditory cortex, you kind of are just reusing the same genes to do the same thing twice. Whereas despite a reflex hucking up, yes, you're right, they have to build a vision system and they have to build some auditory systems and touch systems and navigation type systems. So even feeding into the hippocampus and stuff like that, there's head direction cells. Even the fly brain, it has innate circuits that figure out its orientation and help it navigate in the world and it uses vision, figure out its optical flow of how it's flying. And how is its flight related to the wind direction? It has all these innate stuff that I think in the mammal brain, we would all put that and lump that into the steering subsystem. So there's a lot of work. So all the genes basically that go into specifying all the things a fly has to do, we're going to have stuff like that too, just all in the steering subsystem.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“This is what the cell types helps you get at it. I don't think it's exactly like, oh, this percent of the genome is doing this. But you could say, okay, in all these steering substances, subtypes, how many different genes are involved in sort of specifying which is which and how they wire.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“It's still complicated. And that's all the more reason why a lot of the genomic real estate in the genome and in terms of these different cell types and so on would go into wiring up the steering subsystem. And can we tell? Pre-wiring it. Can we tell?”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. Yeah, yeah. Well, I think so. I think you either have to have a special cell types or you have to somehow otherwise get special wiring rules that evolution can say this neuron needs to wire to this neuron without any learning. And the way that that is most likely to happen, I think, is that those cells express like different receptors and proteins that say, okay, when this one comes in contact with this one. Let's form a synapse. So it's genetic wiring. Yeah. And those need cell types to do it. Yeah.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Situation specific crap that the cortex doesn't know about spiders, it just knows about layers and learners, but you're saying in a certain”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, so this is where you get innately wired circuits, right? So in the learning algorithm part, in the learning subsystem, you specify the initial architecture, you specify a learning algorithm. All the juice is happening through plasticity of the synapses, changes of the synapses within that big network. But it's kind of like a relatively repeating architecture, how it's initialized. It's just like the amount of Python code needed to make an eight-layer transformer is not that different from one to make the three-layer transformer, right? You're just replicating. Whereas all this Python code for the reward function, if superior click list sees something that's skittering and you're feeling goosebumps on your skin or whatever, then trigger spider reflex. That's just a bunch of bespoke species specific.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“There are a lot more weird and diverse and bespoke cell types in the steering subsystem, basically, than there are in the learning subsystem, like the cortical cell types. There's enough to build. It seems like there's enough to build a learning algorithm up there and specifies a hyperparameters. And in the steering subsystem, there's like a gazillion thousands of really weird cells, which might be like the one for the spider flinch reflex and the one for I'm about to taste salt. So why?”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“How many different types of cells are there in different areas of cortex? Are they the same across different areas? And then you look at these subcortical regions, which are more like the steering subsystem or reward function generating regions. How many different types of cells do they have and which neurons types do they have? We don't know how they're all connected and exactly what they do or what the circuits are, what they mean. But you can just quantify how many different kinds of cells are there. with sequencing the RNA.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“And various other researchers who have been doing these single cell atlases. So, one of the things that neuroscience, technology, or sort of scaling up neuroscience technology, again, this is kind of like one of my obsessions, has done through Brain initiative, a big neuroscience funding program is they've basically gone through different areas, especially the mouse brain, and map where are the different cell types.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Gets to exploit that too, right? So it gets to build a reward function that actually has a bunch of generalization in it just by specifying these innate spider stuff and the thought assessors, as Steve calls them, that do the learning. So that's like a potentially a really compact solution to building up these more complex reward functions too that you need. So it doesn't have to anticipate everything about the future of the reward function just anticipate what variables are relevant, what are heuristics for finding what those variables are. And then, yeah, so then it has to have a very compact specification for the learning algorithm and basic architecture of the learning subsystem. And then it has to specify all this Python code of like all the stuff about the spider and all the stuff about friends and all the stuff about your mother and all the stuff about mating and social groups and joint eye contact. It has to specify all that stuff. And so is this really true? And so I think that there is some evidence for it. So Fae Chen and Evan McCauley.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, and it also gets to do this generalization thing with the thing I was describing where we were talking with about the spider, right, of where it learns that just the word spider triggers the spider reflex or whatever.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Maybe that's if we believe this is true that there's a learning subsystem per Steve Burns and a steering subsystem, the learning subsystem doesn't have a lot of pre-initialization or pre-training. It has a certain architecture. But then within lifetime it learns, then evolution didn't actually like amortize that much into that network. It amortized it instead into a set of innate behaviors in a set of these bootstrapping cost functions or ways of building up very particular reward signals.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“But it's also kind of strange how perception can work in just like milliseconds or whatever. It doesn't seem like it uses that much sampling. So it's also clearly also doing some kind of baking things into like approximate forward passes or something like that to do this. Yeah, so in the future, I don't know. I mean, I think. Is it already a trend to some degree that things that people were having to use test time compute for are getting like Used to train back the base model, right? Yeah. Yeah, so now it can do it in one pass. Right. Yeah. So, I mean, I think Maybe evolution did or didn't do that. I think evolution still has to pass everything through the genome to build the network. And the environment in which humans are living is very dynamic. And so”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Right. I mean, first of all, I think the probabilistic AI people would be like, of course, you need text time compute because this inference problem is really hard. And the only ways we know how to do it involve lots of test time compute. Otherwise, it's just this crappy approximation that's never going to have to do infinite data or something to like make this. So I think some of the probabilistic people will be like, no, it's like inherently probabilistic and amortizing it in this way just doesn't make sense. And so, and they might then also point to the brain and say, okay, well, the brain, the neurons are kind of stochastic and they're sampling and they're doing things. And so maybe the brain actually is doing more like the non-amortized inference, the real inference.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“You don't have to evaluate all these energy values or whatever and sample around to make them higher and lower. You just say Approximately that process would result in this being the top one or something like that.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Let's keep sampling That's actually pretty good, I think, of sort of, yeah. Yeah, this Bayesian inference in general is like of this very intractable thing. Right. The algorithms that we have for doing that tend to require taking a lot of samples, Monte Carlo methods, taking a lot of samples, and taking samples takes time. I mean, this is like the original Boltimon machines and stuff we're using techniques like this. And still, it's used with probabilistic programming, other types of methods. Given some model of the world and given some data, like how should I update my, what are the variables, missing variables in my internal model? And I guess”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Vision systems that basically require less training. They put inbuild into the assumptions of the design of the architecture that things like objects are bounded by surfaces and surfaces have certain types of shapes and relationships of how they occlude each other and stuff like that. So it may be possible to build more assumptions into the network. Evolution may have also put some changes of architecture. It's just, I think that also the cost functions and so on may be a key thing that it does.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“I think he put a model, I think it was a model of V1, of sort of specifically how the early visual cortex represents images and put that as like an input into a convNet and that improved some things. So it could be differences. The retina is also doing motion detection and certain things are kind of getting filtered out. So there may be some pre-processing of the sensory data. There may be some clever combinations of which modalities are predicting which or so on that lead to better representation. There may be much more clever things than that. Some people certainly do think that there's inductive biases built in the architecture that will shape the representations differently or that there are clever things that you can do. So stera, which is the same organization that employs Steve Baron's just launched this neuroscience project based on Doris So's work and she has some ideas about how you can build”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“The problem is these questions are all commingled. So if we don't know if it's doing a backprop like learning and we don't know if it's doing energy based models and we don't know how these areas are even connected in the first place, it's very hard to really get to the ground truth of this. But yeah, it's possible. I mean, I think that people have done some work. My friend Joel Depello actually did something some years ago where”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Certain kinds of foundation models. LLMs are maybe just predicting the next token, but vision models maybe are a trait in learning to fill in the blanks or reconstruct different pieces or combinations. But I think that it does it in an extremely flexible way. So it's, you know, if you train a model to just to fill in this blank at the center, okay, that's great. But what if you didn't train it to fill in this other blank over to the left, then it doesn't know how to do that. It's not part of its repertoire of predictions that are like immortized into the network. Whereas with a really powerful inference system, you could Choose at test time what is the subset of variables it needs to infer and which ones are clamped.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, that may be the way. So it's not clear to me. Some people think that. There's sort of a different way that it does probabilistic inference or different learning algorithm that isn't backprop, that might be other ways of learning energy-based models or other things like that that you can imagine, but that is involved in being able to do this and that the brain has that. But I think there's a version of it where what the brain does is like crappy versions of backprop to learn to predict through a few layers. And that, yeah, it's kind of like a multimodal foundation model. Right. Yeah. So maybe the cortex is just kind of like”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“He's an AI safety researcher. He's just synthesizing. This comes back to the academic incentives thing. I think that this is a little bit hard to say. What's the exact next experiment? How am I going to publish a paper on this? How am I going to train my grad student to do this? Very speculative. But there's a lot in the neuroscience literature, and Stephen has been able to pull this together. And I think that Steve has an answer to Ilya's question, essentially, which is how does the brain ultimately code for these higher level desires and link them up to the more primitive rewards.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“That's because you learned this. And the cortex inherently has the ability to generalize because it's just predicting based on these very abstract variables and all these integrated information that it has. Whereas the steering suspicion only can use whatever the superior cortless and a few other sensors can spit out.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“So now I'm activating your steering subsystem, your steering subsystem, Spider Hypothalamus, a subgroup of neurons of skittering insect are activating based on these very abstract concepts in the conversation.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Or the concept of spider is going to cause that to trigger. And this predictor can learn that. So whatever spider neurons are in my world model, Which could even be a book about spiders or somewhere a room where there are spiders or whatever that is”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Could I have predicted that flinching response? It's going to be a group of neurons that is essentially a classifier. Am I about to flinch? And I'm going to have classifiers for that for every important steering subsystem variable that evolution needs to take care of. Am I about to flinch? Am I talking to a friend? Should I laugh now? Is the friend high status? Whatever variables the hypothalamus brainstem contain, am I about to taste salt? So that's going to have all these variables. And for each one, it's going to have a predictor. It's going to train that predictor. Now the predictor that it trains, that can have some generalization. And the reason it can have some generalization is because it just has a totally different input. So it's input data might be things like the word spider, right? But the word spider can activate in all sorts of situations to lead to the world spider activating in your word model. So, you know, if you have a complex world model with really complex features, that inherently gives you some generalization. It's not just the thing skittering toward me. It's even the word spider.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Or I just flinched. I just flinched neurons and hypothalamus. So when you flinch, first of all, a negative contribution to the reward function, you didn't want that to happen, perhaps. But that's a reward function then that doesn't have any generalization in it. So I'm going to avoid that exact situation of the thankskittering toward me. And maybe I'm going to avoid some actions that lead to the thing skittering. So that's something, a generalization you can get, what Steve calls it, is downstream of the reward function. So I'm going to avoid the situation where the spider was skittering toward me. But you're also going to do something else. So there's going to be like a part of your amygdala say that is saying, okay, a few milliseconds, hundreds, hundreds of milliseconds or seconds earlier.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, it's because the learning subsystem. Is a powerful learning algorithm that does have generalization, that is capable of generalization. So the steering subsystem, these are the innate responses. So you're going to have some, let's say, built into your steering subsystem, these lower brain areas, hypothalamus brainstem, et cetera. And again, they include they have their own primitive sensory systems. So there may be an innate response. If I see something that's kind of moving fast toward my body that I didn't previously see was there and is kind of small and dark and high contrast, that might be an insect kind of skittering onto my body, I am going to like flinch, right? And so there are these innate responses. And so there's going to be some group of neurons, let's say in the hypothalamus, that is that I am flinching.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“So they'll be part of the amygdala or part of the cortex that is learning to predict those responses. And so, what are the neurons that matter in the cortex for social status or for friendship? Or they're the ones that predicts those innate heuristics for friendship, right? So you train a predictor in the cortex and you say which neurons are part of the predictor, those are the ones that are, now you've actually managed to wire it up.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Innately programmed responses and the innate programming of these series of reward functions, cost functions, bootstrapping functions that exist. So there are parts of the amygdala, for example, that are able to monitor what those parts do and predict what those parts do. So how do you find the neurons that are important for social status? Well, you have some innate heuristics of social status, for example, or you have some innate innate heuristics of friendliness that the steering subsystem can use. And the steering subsystem actually has its own sensory system, which is kind of crazy. So we think of vision as being something that the cortex does. But there's also a steering subsystem, subcortical visual system called the superior colliculus with innate ability to detect faces, for example. or threats. So there's a visual system that has innate heuristics in that the steering subsystem has its own responses.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“And this is important, right? Because if we're going to be able to seek status in the tribe or learn from knowledgeable people, as you said, or things like that, exchange knowledge and skills with friends, but not with enemies. I mean, we have to learn all this stuff. So it has to be able to robustly wire these learned features of the world, learn parts of the world model up to these innate reward functions, and then actually use that to then learn more, right? Because next time I'm not going to try to piss off Jan Lakun if he emails me that I got this wrong. And so I'm going to do further learning based on that. So in constructing the reward function, it has to use learned information. But how can evolution didn't know about Jan Lakun? So how can it do that? And so the basic idea that Steve Burns is proposing is that, well, part of the cortex or other areas like the amygdala that learn, what they're doing is they're modeling the steering subsystem, the steering subsystem is the part with these more innate.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“And so it's important that I have that instinctual response. But of course, evolution has never seen Jan Lacun or known about energy-based models or known what an important scientist or podcast is. And so somehow the brain has to encode this desire to not piss off really important people in the tribe or something like this in a very robust way without knowing in advance all the things that the learning subsystem of the brain, the part The cortex is going to learn this world model that's going to include things like Jan Lakun and podcasts. And evolution has to make sure that those neurons, whatever the young lacuna being upset with me, neurons, get properly wired up to the shame response or this part of the reward function.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“I am embarrassed for saying the wrong thing on your podcast because I'm imagining that young Lakun is listening and he says, that's not my theory. You describe energy-based models really badly. That's going to activate in me innate embarrassment and shame. And I'm going to want to go hide and whatever. And that's going to activate these innate reflexes. And that's important because I might otherwise get killed by Jan Lacun's marauding army of other.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, well, let's come back to this. And this is partly having to do with Steve Barren's theories, which I'm recently obsessed about. But on your podcast with Ilya, He said, Look, I'm not aware of any good theory of how evolution encodes high-level desires or intentions. I think this is very connected to all of these questions about the loss functions and the cost functions that the brain would use. And it's a really profound question, right? Like let's say that.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“There might be areas that predict things that the more innate part of the brain is going to do. Because remember, this whole thing is basically riding on top of the sort of a lizard brain and lizard body, if you will. And that thing is a thing that's worth predicting too. So you're not just predicting, do I see this or do I see that? But is this muscle about to tense? Am I about to have a reflex where I laugh? Is my heart rate about to go up? Am I about to activate this instinctive behavior?”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“What is the likelihood or unlikelihood of just any combination of variables? And if I clamp some of them, I say, well, definitely these variables are in these states, then I can compute with probabilistic sampling, for example, I can compute, okay, conditioned on these being set in this state, and these could be any arbitrary subset of variables in the model. Can I predict what any other subset is going to do and sample from any other subset given clamping this subset? And I could choose a totally different subset and sample from that subset. So it's omnidirectional inference. And so that could be there's some parts of cortex that might be like association areas of cortex that may predict vision from audition.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Can learn how to do that stuff in this emergent level in context learning, but natively is just predicting the next token. What if the cortex is just natively Made so that it can, you know, any area of cortex can predict any pattern in any subset of its inputs given any other missing subset. That is a little bit more like. Quote unquote probabilistic AI. I think a lot of the things I'm saying, by the way, are extremely similar to like what Jan Lacun would say. He's really interested in these energy-based models and something like that is like the joint distribution of all the variables.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“And no one knows. But I think one thought about it, one possibility about it is that it's just this incredibly general prediction engine. So any one area of cortex is just trying to predict Basically, can it learn to predict any subset of all the variables it sees from any other subset? So like omnidirectional inference or omnidirectional prediction. Whereas an LLM is just you see everything in the context window and then it computes a very particular conditional probability, which is given all the last thousands of things, what is the very probabilities for all the next token. But it would be weird for a large language model to say the quick brown fox blank blank. Lazy dog and filling in the middle. Versus do the next token if it's doing just forward.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“What's up with that? This is a huge question, and we don't know. I've seen models where what the cortex has typically this like six layered structure layers in a slightly different sense than layers of a neural net. It's like any one location in the cortex has six physical layers of tissue as you go in layers of the sheet. And then those areas then connect to each other. And that's more like the layers of a network. I've seen versions of that where what you're trying to explain is actually just how does it approximate backprop and what is the cost function for that? What is the network being asked you to do if you sort of are trying to say it's something like backprop? Is it doing backprop on next token prediction or is it doing backprop on classifying images or what is it doing?”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Lot of Python code basically generating a specific curriculum for what different parts of the brain need to learn because evolution has seen many times what was successful and unsuccessful and evolution could encode the knowledge of the learning curriculum. So in the machine learning framework maybe we can come back and we can talk about where do the loss functions of the brain come from can that can different loss functions lead to different efficiency of learning”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“There is, how is it initialized? Okay. So if we take the learning part of the system, it still may have some initialization of the weights. And then there are also cost functions. There's like, what is it being trained to do? What's the reward signal? What are the loss functions, supervision signals? My personal hunch within that framework is that the field has neglected The role of this very specific loss functions, very specific cost functions. Machine learning tends to mathematically simple loss functions. Predict the next token cross-entropy, these simple kind of computer scientists loss functions. I think evolution may have built a lot of complexity into the loss functions, actually many different loss functions were different areas turned on at different stages of development.”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, this might be the quadrillion dollar question or something like that. It's. Arguably, you could make an argument this is the most important question in science. I don't claim to know the answer. I also don't really think that the answer will necessarily come even from a lot of smart people thinking about it as much as they are. My overall meta level take is that we have to empower the field of neuroscience to just make neuroscience a more powerful field technologically and otherwise to actually be able to crack a question like this. Maybe the way that we would think about this now with modern AI, neural nets, deep learning is that there are sort of these certain key components of that. There's the architecture. There's maybe hyperparameters of the architecture, how many layers do you have or sort of properties of that architecture? There is the learning algorithm itself. How do you train it, you know, backprop gradient scent? Is it something else?”
2025-12-30 · Dwarkesh Podcast · Adam Marblestone — AI is missing something fundamental about the brain · IDENTIFIED FROM THE TRANSCRIPT · source