YouSaid · the spoken record

Pushmeet Kohli

lines on the record: 25
first: 2025-06-26
most recent: 2025-06-26
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Share about the particulars yet. But as you can imagine, there are so many exciting computational problems in a place like Google within AI and also outside that I'm sure there will be many, many really cool results coming in the future.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“In this white paper, we try to think about holistically when we look at the computational infrastructure of Google, what are the key parts in this infrastructure to demonstrate that alpha evolve can make discoveries across the stack, not only in one part of it, and that it can make discoveries that are highly valuable. And so we try to cover the entire spectrum. So we show that alpha evolve can improve the efficiency of the data center. It can contribute to hardware design and it can contribute to improving the efficiency of most important pieces of software that are being run inside Google. One intention here was to demonstrate that this is a really versatile tool that you can apply across the spectrum. And as Pushmit was saying, this is a tool that is already available inside Google and it is being used for many, many problems. There are quite a few exciting ones. I'm not ready to show.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“Would you have thought of writing it in exactly this way and then trying to understand okay, what exactly is it doing? That's a really interesting experience. But at the same time, it's one of the key strengths of the system, not only for scientific applications where you can look at the code and get some understanding out of it, but also for many of the practical applications. It's hugely valuable that the artifact you get out of Alpha Evolve is a piece of code. And then you deploy that piece of code. And so before you do that, experts, engineers who have worked on that system can visually inspect that piece of code, understand it, and make the final decision of whether it's going to be deployed. The code, you can look at it, understand it, and make the decision yourself.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“How you build it. And so we had a first hand experience collaborating with multiple mathematicians and it's been really fascinating to see where we would share with them the output from alpha evolve and they would be really fascinated looking at the code that it found and trying to understand, okay, what is it actually doing and then understanding, oh, okay, this is doing this, this is doing that, and now I can see why if you put it together, then it leads to a really good solution. Yeah, I can also confirm from my own personal experience that looking at the code or the algorithms that the system finds, it's often a really interesting experience because it's code that kind of like looks human-like, like it's something that you could have written.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“We have seen this in particular with mathematicians that we have collaborated with. And there are a few reasons for this. But one is that alpha evolve is an agent that doesn't just give you the solution. It searches for an algorithm that constructs that solution. And so depending on how you set up your problem definition, often it's actually the algorithm that's even more valuable than the solution itself because the algorithm, it tells you how to construct the solution. So that means you understand what are the ideas that go into building that solution. And maybe especially or definitely it's true in mathematics, that's what people really care about, to understand the nature of our universe and build up the understanding of fundamental ideas. And so it's actually often not interesting almost at all what the solution is, but what you care about is.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“I just wanted to highlight that even though we have been describing Alpha Evolve as this kind of autonomous agent that does things on its own, actually in practice using this agent often turns out to be surprisingly collaborative”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“Molecule, then you can have an evaluation function again in the form of a simulator or a predictive model that given a candidate molecule will make a meaningful prediction about, okay, is this actually going to work in practice? And then if you are in this regime, then again alpha evolve would be applicable. And we are only talking about the version of alpha evolve that we have built today. And these are problems that we can address today. But we don't think that the journey of alpha evolve finishes here. We have many ideas about how to make the system more powerful and more broadly applicable. And I'm fairly confident that we'll see many applications across many branches of science. And then this is only talking about alpha evolve. There are many other agents pushmit mentioned AI co-scientists and many others that I'm sure will keep transforming how science is.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“First of all, yes, I'm super excited working in this area of using AI to accelerate the sciences because in a way it's the most exciting application of AI that I can imagine. What could be more valuable or exciting to advancing the frontiers of human knowledge? So yes, that is definitely there. And then, of course, in different fields of science, the speed of progress or the advance you get from AI might be slightly different. So in Alpha Evolve, we've primarily focused on mathematics and computer science because these are the domains where it's the easiest to get this automated evaluation functions. You often get them basically for free. That's not to say that you cannot get them in other branches of science, but in math and computer science, it's just they're just most common. If you think about biology or chemistry, you want to design”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“Maybe just one thing to add to it is that I would also agree that we are maybe seeing the first sign of self improvement, but one also needs to be very specific about what we have shown so far, like as Pushmit mentioned, it's the speeding up the training of the next generation of the Gemini model. So the feedback loop is fairly long, at least currently, maybe on the order of months. But there is, you can call it self-improvement for sure. Maybe the big question that many people are curious about is how does this extrapolate into the future? And you can have different types of self-improvement. One is where you get maybe just a one-off benefit, like the model improves itself once and that's it. Another one is, okay, the model keeps improving itself continuously, but maybe the improvements get marginally smaller and smaller and smaller and you converge to some limit. Or maybe the improvements will keep accumulating up and up and up. And that's a big open question that we don't.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“Strict evaluation that exactly tells you how good a solution is on one end and then natural language evaluation, biolanguage model on the other end. There is a continual spectrum of simulators and auxiliary evaluation functions, which are maybe not perfect, but as long as they are correlated with the true signal, then we can build the algorithmic scaffolding of the evolutionary algorithm around this in such a way that we still make meaningful progress. And maybe it will take a few more iterations, but we can still go really, really far.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“Perspective is that we don't actually think this is a conceptual limitation. So today we have this was maybe the easiest way to get into this game of discovering new things by looking at problems that already come with this very, very precise evaluation functions. So that's just a natural first step to take. But I do believe that this assumption can be relaxed in very significant ways. And in particular, you already mentioned one example where maybe language models themselves will be able to evaluate whether proposed solutions look promising or not or whether they fail in some particular ways. And indeed, there is a parallel work from FeepMind as well called AI co-scientist, which demonstrates this very clearly that if you propose ideas in natural language, then you can get language models to provide meaningful critiques and identify the ones that work from the ones that don't. So I really do see a lot of hope on relaxing this assumption. And then even in between these two extremes of...”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“That's a really, really great question. And I think you can view it from two perspectives that I think will happen at the same time. So one is that, yes, currently the strict evaluation function plays a key role in Alpha Evolve. And one takeaway you can take from this thinking about the future is that it shows the really high value of having these evaluators available. Because in many cases it might be that you have a really important problem, but you don't actually have a very precise definition of what makes for a good solution. And one takeaway you can have from a system like this is that if you actually do build a very precise evaluation function, then this unlocks the possibility of having an agent like alpha evolve discover something that's way beyond what, let's say, humans have been able to discover or your best developers have been able to discover. So that's one takeaway. But the other takeaway that I'm maybe even more excited about from the research”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“Do you know how difficult this question is going to be? And especially in the sciences, that's something that often has a very surprising answer. Very trivial questions can turn out to be extremely, extremely difficult and vice versa. But the nice thing is that you have continual improvement if you run this system. And as long as you can run it, you can expect to get better and better results. And you just have to see where this gets you.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“I know it maybe sounds trivial that if you wait longer, you get better results. But in practice, that's actually a really difficult thing to build automated agents that are able to sustain this continual improvement without plateauing quite early. This is, I think, a nice feature. There was a second part to the question about predicting how many iterations you will need. So that is something that is actually not so easy because it's like asking.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“There are two parts to your question one is about how does scaling work and then how can you predict it so for the first part this is actually a really nice feature of alpha evolve that it can adapt to the difficulty of the problem if you ask alpha evolve to find a solution to a problem that's actually unexpectedly easy, then it will just do it very, very quickly, like almost immediately you will have the solution. But if you ask it a problem that's really, really difficult and by really, really difficult, I mean like really difficult, maybe an open question that has stood for decades in the sciences or you want the practical algorithm.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“That's right, like you would expect that in each iteration of evolution, what you are doing is you are looking at the previous iteration, looking at maybe the strongest solutions you have, and then trying to be creative about how can I combine ideas from those solutions or maybe bring in completely new ideas to come up with something even better. And so yes, each generation gets stronger and stronger”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“By the user that is able to actually filter out the things that work from the ones that don't. And then this is wrapped inside an evolutionary algorithm that makes sure that we kind of discover the whole space of algorithms in that region so that we don't commit to a very specific type of solution early on. But instead we maintain a diverse pool of potential solutions. Over time, maybe we combine ideas from different solutions that are already strong until we actually have an algorithm that's so strong that we are happy to deploy to a critical part of Google's infrastructure, let's say.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“You define the what by providing the evaluation function, and then alpha evolve fills in the how. So that's the job of our system. And you can do it in two fairly different ways. One is you tell alpha evolve, I have no idea how to solve this problem. Let's start completely from scratch and let's try to be creative and come up with something completely new. So that's one option you can take. Another option you can take is actually we have already worked on this problem for a really long time. Here is a very strong initial solution that we can provide to the system and you can start from here. And that's what we did for the application to discovering new algorithms for scheduling jobs in a data center. So Alpha Evolve takes this initial solution and then it on a high level it combines the creative power of large language models to propose creative new ways how to improve that solution, the strictness of the evaluation function provided.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“So that's a simulator that we already had. And I would say it's something that is quite natural to have in many domains because whenever you want to innovate on something, you need to have a way of telling, okay, is the innovation actually good or not? So it's a very natural object to have at least in principle.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“I can walk you through that. So the user of a system like Alpha Evolve, they basically specify what is the problem that they are trying to solve. So that's the most important thing. And you specify it by providing what is called an evaluation function. What this function does is whenever there is a proposed solution for solving the problem, you're able to tell how good this solution is. So you basically define what makes a good solution. For discovering an algorithm for scheduling jobs on a data center, this evaluation function could be something like a simulator of jobs in a data center that given an algorithm for doing the scheduling, it simulates how good this algorithm is. So that's what the user provides.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“And the other discover something new on these problems. I think that's as strong a demonstration as I can imagine of the fact that this is indeed something that is new because no one found it before. And also it is something that was not easy to discover because those results stood for such a long time and have been worked on by such strong people.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“Just to add to that, so I definitely agree the search space is just unbelievably vast. The solutions are maybe non-intuitive. And the third thing I want to emphasize is that I really believe that people who worked on this in the past were definitely not complacent. And in fact, the problems we chose to apply Alpha Evolve to in the first instance, both on the scientific side and the practical side, we deliberately chose problems which have been worked on for a very long time by the very best people. On the scientific side, since we are talking about matrix multiplication, this has been a known open problem for decades and then many people have been working on it. And similarly for the practical applications that we mentioned in our Alpha Evol release in key parts of Google's infrastructure, again, like these are things that have been heavily optimized inside Google because they are so important. And so by having a system like Alpha Evolve or”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“Multiplying matrices for the first time show that AI agents can discover better algorithms than what humans had been able to do before them. So this was the first system that gave weight to this idea that indeed with AI, we'll be able to go into this superhuman region of algorithms that we as humans had not been able to discover ourselves.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“We are not new to this space of algorithm discovery. As you might know, the mission of all of DeepMind is to build AI responsibly to benefit humanity and the way our particular team has been doing it for years now is to look for ways how AI can discover new algorithms. New algorithms are everywhere around us. So this is a very, very important question and can have very high impact when we can discover algorithms that solve important computational problems with higher efficiency than what we have been able to do so far. And kind of the first breakthrough we had in this space was in 2022 when we released a system called Alpha Tensor. And so that was a system that was an AI system using reinforcement learning that for a very specific but fundamental computational task.”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT
“So, in maybe one sentence, Alpha 5 is an AI coding agent that is able to discover new algorithms that are able to make new discoveries on open scientific problems, and at the same time, those algorithms can be so practical that they are already deployed in key parts of Google's own infrastructure”
2025-06-26 · No Priors · Meet AlphaEvolve: The Autonomous Agent That Discovers Algorithms Better Than Humans With Google DeepMind’s Pushmeet Kohli and Matej Balog · IDENTIFIED FROM THE TRANSCRIPT