YouSaid · the spoken record

David Lichtenstein

lines on the record: 53
first: 2025-08-21
most recent: 2025-08-21
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Product form factors that have product market fit and are growing like crazy. I bet when we fast forward five years and we look back on this period, there will be six to seven more of these crucial product form factors that will look obvious in hindsight, but no one's really solved today. And if you really want to take an asymmetrical upside bet, I would try to spend some time and figure out what those are now.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“First off, tiny teams with lots of compute. Is the correct recipe for building a frontier lab? That's what we're doing at Amazon with SF and my team. It's really important that you have the opportunity to run your research ideas in a particular environment. If you go somewhere, it already has 3,000 people. You're not really going to have a chance. There's so many senior people ahead that are all too ready to try their particular ideas. The second thing is I think people underestimate the co-design of like the product and the user interface and the model. I think that's going to be the most important game that people are going to play in the next couple years. And so going somewhere that actually is very strong product sense and a vision for how users are actually going to deeply embed this into their own lives is going to be really important. And one of the best ways to tell is, are you just building another chatbot? Are you just trying to fight one more entrant in the coding assistant space, right? Those just happen to be two of the earliest.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“The people who know how to do super meaningful work will definitely expand, but it will be still a little constrained by the fact that you cannot have too many people on any one of these projects at once.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Bag of weights, right? So, what I do in pre-trained day, what this other person does in supervised fine tuning, what this other person does in RL, and this other person does to make the model run fast, they all interact with each other in sometimes pretty unpredictable ways. So it has one of the worst diseconomies of scale with number of people of anything I've ever seen, except maybe even sports teams, right? Maybe that's the one other case where you don't want to have 100 mid-level people you want to have 10 of the best, right? And because of that, the number of people who are going to have a seat at the table at some of the best funded efforts in the world, I think is actually going to be somewhat capped.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Field's definitely going to expand. There's going to be more and more people who really learn the tricks that the field has developed so far and discover the next set of tricks and breakthroughs. But I think one of the dynamics that's going to keep the field smaller than other fields like software is that unlike regular software engineering, foundation model training breaks so many of the rules that we think we should have, right? In software, let's say our job here is to build Microsoft Word, right? I can say, hey, Alex, it's your job to make the save feature work. It's David's job to make sure that cloud storage works and someone else's job to make sure the UI looks good. You can factorize these problems pretty independently from each other. The issue with foundation model training is that every decision you take interferes with every other decision because there's only one deliverable at the end. The deliverable at the end is your frontier model. It's like one giant.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Say it's probably less than a thousand people. But again, I don't want to trivialize. I think junior talent is extremely important. And people who come from other domains, like physics or quant finance or have just been doing undergrad research, these people make a massive difference really, really, really fast. But you want to surround them with a couple of folks who have already learned all the lessons from previous training attempts in the past.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Is that it doesn't take that many years actually to find yourself at the frontier if you're a junior person. Like some of the best people in the field were people who just started three or four years ago. And by working with the right people, focusing on the right problems, there's like working really, really, really hard, they found themselves at the frontier. Like AI research is one of those areas where if you ask four or five questions, you already discovered a problem that nobody has the answer to. And then you can just focus on that and focus on how do I become the world expert in this particular subdomain. And so I find that really counterintuitive, that there's only very few people who really know what they're doing, and yet it's very easy in terms of number of years to become someone who knows what they're doing.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Two things I want to talk about one is honestly the founder plays a really important role. The founder has to want to really take care of the team and make sure that everybody is treated pro rata and equally, right? The second thing is it's very counterintuitive in AI right now because there's only a small number of people with a lot of experience. And because the next couple years is going to move so fast and a lot of the value, the market positioning, et cetera, is going to be decided in the next couple years. If you're sitting there responsible for one of these labs and you want to make sure that you have the best possible AI systems, you need to hire the people who know what they're doing. And so the market demand, the pricing for these people is actually totally rational just solely because of how few of them there are. But the counterintuitive thing.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Two things I saw coming. One, if you want to be at the frontier of intelligence, you have to be at the frontier of compute And if you're not on the frontier of compute, then you have to pivot and go do something that is totally different. And my whole career, all I want to do is build the smartest and most useful AI systems. So the idea of turning adept into an enterprise company that only sells small models or turns into a place that does forward deployed engineering to go help you deploy an agent on top of someone else's model, none of those things appeal to me. Like I want to figure out here are the four crucial remaining research problems left AGI. How do we nail them every single one of them is going to require like two digit billion dollar clusters to go run at? So how else am I going to be able to have in this whole team that I've put together who all are motivated by the same thing? How are we going to have the opportunity to go do that?”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“That's one researcher, right? That's one employee. So if that's the world that you live in, it's really important, I think, for us to partner with someone who's going to go fight all the way to the end. And that's why we came to Amazon.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Well, first off, humanity's demand for intelligence is way, way, way higher than the amount of supply. And so therefore, for us as a field to invest ridiculous amounts of money in building the world's biggest clusters and bringing the best talent together to drive those clusters is actually perfectly rational, right? Because if you can spend an extra X dollars to build a model that has plus 10 IQ points and can solve like a giant new concentric circle of useful tasks for humanity, that is a worthwhile trade that you should do any day of the week. And so I think it makes a lot of sense that all these companies are trying to put together critical mass on both talent and compute right now. And from my perspective, for why join Amazon, it's because Amazon knows how important it is to win on the agent side in particular. And that agents are a crucial bet for Amazon to really build one of the best frontier labs possible and to”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“As a product category wasn't even coined yet. So we were trying to find a good term. And we started, we played with things like large action models and action transformers. So our first product was called action transformer. And then only after that did agents really start picking up as being the term.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Multimodal input, you kind of have to throw away a lot of the optimizations you did in the text only world, and that gives time for other people to catch up. I think that was actually part of how Gemini was able to catch up was that they bet on certain interesting ideas on native multimodal that turned out well for them, right? But then after that with reasoning models, right, they gave another opportunity for people to catch up. That's why deep sequ was able to surprise the world, because they straight quantum tunnel to that instead of doing every stop along the way. And I think with the next turn being agents, especially agents without verifiable rewards, if at Amazon we can figure that recipe out earlier, faster, better than everybody else with all the scale that we have as a company, it basically brings us to the frontier at that point.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Right way to think about it is that every time you have a new upstart lab trying to join the frontier of the AI sort of game, you need to bet on something that can really leapfrog, right? What's interesting is every time there's a recipe change for how these models are trained, it creates a giant window of opportunity for someone new who's starting to come to the table with that new recipe instead of trying to catch up on all the old recipes. Because the old recipes are actually baggage for the incumbents. So to give some examples of this, at OpenAI, of course, we basically pioneered giant models, right? The whole LLM thing came out of GPT2 and then 3, of course. But those LLMs initially, they were text-only training recipes. And then we discovered RLHF, and then they started getting a lot of human data via RLHF. But then in the switch to multimodal,”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Honestly, I think we're sub one year. We have line of sight. We've built out teams for every step of that particular problem. And things are just starting to work. It's just really fun to go to work every day. And you realize that one of the teams has made a little very useful breakthrough that particular day. And the whole cycle that we're doing for this training loop seems to be going a little bit faster every day. Going back to GPT-5, people have said, you know, does this portend does slow down an AI progress? 100% I think the answer is no because when one S-curve peters out, right, the first one being pre-training, which I don't think is petered out, by the way either, but it's definitely at this point less easy to get gains than before. And then you've got RL with verifiable rewards. But then every time one of these S curves seems to slow down a little bit, there's another one coming up. And I think Agents is the next S curve and the specific training recipe we were talking about earlier is one of the main ways.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Overall in the field. And we've been having a lot of really good luck specifically by focusing extreme amounts of effort on reliability. So we're now used for things like, for example, doctor and nurse registrations. Or we have another customer called Navan, which is formerly trip actions, which uses us basically to automate a lot of back-end bookings for travel for their customers. We've got companies that basically have like 93-step QA workflows that they've automated with a single act script, et cetera. So I think the early progress has been really cool. Now what's up ahead is how do we do this extreme large scale self-play on a bajillion gems to get to something where there's a bit of a GPT for RL agents moment and we're running as fast as we can towards that right now.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, so a wide range of enterprises and developers are using Nova Act. And the reason why it's not something that you hear about is because we're not a consumer product, if anything, the whole Amazon agent strategy, including what I did before at Adept is sort of doing norm core agents, not the super sexy stuff that works one out of three times, but super reliable low level workflows that work 99 plus percent of the time. So that's the target since Nova Act came out, we've actually had a bunch of different enterprises end up deploying with us where they're seeing 95 plus percent reliability, which is, as I'm sure you've seen from the coverage of other agent products out there, is like a material step up from the average 60% level reliability that folks see with those systems. And I think that reliability bottleneck is why you don't see as much agent adoption.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Released Nova Act, which was a research preview that came out in March. But as you can imagine, we've added way more capability since then, and it's been really cool. The thing we always do is we first dog food with internal teams.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Again, as we were saying earlier, because Amazon sort of has an internal effort for almost every useful domain of knowledge work, there's been a lot of enthusiasm to pick up a lot of these systems. And we have this internal channel called, actually, I won't tell you what it's called, what codename-interest, which is related to the product that we've been building. And it's just been crazy to see teams from all over the world within Amazon, actually, because one of the main bottlenecks we've had is we didn't actually have availability outside of the US for quite a while. It was crazy just how many international Amazon teams wanted to start picking this up and then using it themselves on various operations tasks that they had.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“While we're really close to Peter Beale's group on the robotic side, which is awesome, some of the other areas we have this a big push for internal adoption of agents within Amazon. And so a lot of those conversations or engagements are happening.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Lexa Plus is just one of many customers that we have. And what's really interesting about being within Amazon is going back we were talking about earlier, web data is effectively running out and it's not useful for training agents. It's actually useful for training agents is lots and lots of environments and lots and lots of people doing reliable multi-step workflows. And so the interesting thing at Amazon is that Initial Alexa Plus, basically every Fortune 500 business's operations are represented in some way by some internal Amazon team, right? Like there's one medical, everything happening on supply chain and procurement on the retail side. There's all this developer facing stuff on AWS. And agents are going to require a lot of private data and private environments to be trained. And because we're an Amazon that's all now 1P. So they're just one of many different ways in which we can get reliable workflow data to train the smarter agent.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, and the early reception to Alexa Plus has been that it's a dramatically for Alexa, but still brittle. There's still moments where it's not reliable. And I'm wondering, is this the real gem? Is this the at-scale gym where Alexa Plus is how your system gets more reliable much faster? You have to have this in production and deployed to, I mean, Alexa has millions and millions of devices that it's on. Is that the strategy? Because I'm sure you've seen the early reactions to Alexa Plus are it's better but still not as reliable as people would like it to be.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“A good question. So, Alexa Plus has the ability to, for example, if your toilet breaks, it's like, oh man, I really need a plumber. Alexa, can you get me a plumber? Then what happens is Alexa Plus spins up a remote browser powered by our technology, basically, that then goes and uses thumbtack like a human to go get you a plumber to your house, which I think is really cool. It's the first production web agent that's been shipped, if I remember correctly.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Amazon's done a great job for what we're doing here is we're allowed to run pretty independently. And I think there's a recognition that some of the startup DNA right now is really valuable for maximum speed. If you believe AGI is two to five years away, some people are getting more bullish, some people are getting more bearish. Doesn't matter. That's not a lot of time in the grand scheme of things. You need to move really, really fast. So we've been given a lot of independence. We've also taken the tech stack that we've built and contributed a lot of that upstream to the Nova Foundation model as well.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Interface between humans and AIs is this like perpendicular one on one interaction where I'm delegating something or maybe giving me some news back or I'm asking you a question, et cetera. One of the real things we've always missed is this parallel interaction where both the user and the AI actually have a shared canvas that they're jointly collaborating on. I think if you really think about building a teammate for knowledge workers or even just the world's smartest personal assistant, you would want to live in a world where there's actually a shared collaborative canvas for the two of you.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Chatbots are definitely not the long term answer, or at least not chatbots in the way we think about it today. If you want to build systems that take actions for you. The best analogy I have for this is, so my dad is a very well-intentioned, smart guy, spent a lot of his career working in a factory, and he calls me all the time for tech support help. And he's like, David, something's wrong with my iPad. You got to help me with this. And we're just doing this over the phone. And I can't see what's on the screen for him. And so I'm trying to figure, oh, you know, like you have the settings menu open. Have you clicked on this thing yet? Oh, like what's going on with this toggle? Chat is such a low bandwidth interface. Like that is the chat experience for trying to get actions done with a very competent human on the other side trying to handle things for you. So one of the big missing pieces, in my opinion, right now in AI is our lack of creativity with product form factors, frankly, right? We're so used to thinking that the right”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“The end state of all this is a model plus a system that is like rock solid, reliable, like 99% reliable at all sorts of valuable knowledge tasks that are done on a computer. And this is going to be something that we think is going to be a service on AWS that's going to underpin effectively so many useful applications in the future.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“But we call them gyms actually because at OpenAI, we had a very useful early project called OpenAI Gym. And what it was was this was far, way before LLMs were a thing. And what OpenAI Jim did was a collection of video game tasks and robotics tasks. Like, can you balance a poll that's on a cart and can you train an RL algorithm that can keep that thing perfectly standard, et cetera? What we were inspired to do is now that These models are smart enough why have toy tasks like that? Why not put in the actual useful tasks that humans do on their computer into these gyms and have the models learn from these environments? And I don't see why this wouldn't also generalize to robotics.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Do you have a background in robotics, right? I've also done robotics work before. Here we also have Peter Abiel, who came from Clovarian and is a Berkeley professor that basically created, or his students ended up creating the majority of the RL algorithms that work well today. It's funny that you say gyms because we were trying to find an internal code name for the ever. We kicked around Equinox and various boot camp and all this stuff. And I'm not sure everybody had the same sense of humor.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“I think that what's interesting is this field ultimately you have to be able to do something like this in my opinion to be able to get beyond the fact that there's a limited amount of free floating data on the internet that you can train your models on. The thing we're doing at Amazon is because this came from what we did at Adept and Adept has been doing agents for so long, we just care about this problem way more than everybody else and I think have made a lot of progress towards this goal.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“The depreciation correctly, or did I correctly make this part in CAD, or did I successfully book the flight or did choose a consumer analogy? Every time it does this, it actually learns the consequences of its actions. And we believe that this is one of the big missing pieces left for actual AGI. And we're really”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Making moves that no human had ever seen before and contributed to like the state of the art of that whole field. What we're doing is rather than doing more behavioral cloning or watching YouTube videos, what we're doing is we're creating a giant set of RL gems. And each one of these gyms, for example, is an environment that a knowledge worker might be working in to get something useful done. So here's a version of something that's like Salesforce. Here's a version of something that's like an ERP. Here's a CAD program. Here's an electronic medical record system. Here's accounting software. Here's every interesting domain of possible knowledge work is now a simulator. And now instead of training an LLM just to do tech stuff, we have the model actually propose a goal in every single one of these different simulators. Try solving that problem, figure out if it's successfully solved it or not, and then get reward and feedback based on, you know, oh, did I?”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Have it spend 99% of its time watching YouTube videos of tennis and then 1% of its time actually playing tennis, you would have something that's far more balanced between these two things. So what we're doing in our lab here at Amazon is we're actually doing large-scale self-play. And so if you remember the concept of self-play, what it was was a technique that really deep mind made popular in the mid-2010s when they beat humans at play and go. For playing Go, what they did was they spun up a bajillion simulated Go environments, right? And then they had the model play itself over and over and over again. Every time they found a strategy that was better at beating a previous version of itself, it would effectively get positive reward via reinforcement learning to go do more of that strategy in the future. And if you spent a lot of compute on this in the GO simulator, it actually discovered superhuman strategies for how to play Go and then ended up, you know, when they played the world champion.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Words, I should go say this particular next word. The issue with this is this is great for chat. This is great for creative use cases, right, where you want some of the chaos and randomness from hallucinations. But if you want it to be an actual successful decision-making agent, these models need to learn the true causal mechanism, right? It's not just cloning human behavior. It's actually learning if I do X, the consequence of it is Y. And so the question is, how do we train agents to be able to learn the consequences of its actions? And the answer obviously cannot be just doing more behavioral cloning and copying text, right? It has to be something that looks like actual trial and error in the real world. And so that's basically the research roadmap for what we're doing in my group at Amazon. My friend Andre Carpathi has a really good analogy here, which is, you know, imagine you have to train an agent to go play tennis, right? You wouldn't”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“So we started with a topic of how these models are increasingly converging in capability. So while that's true for LLMs, I don't think that's been true to date for agents. And it's because the way that you should train an agent and the way that you train an LLM actually quite different from each other. So LLMs, as we all know, the bulk of their training happens from doing next token prediction, right? I've got a giant corpus of every article on the internet. Let me try to predict the next word. And if I get the next word right, then I get a positive reward. And if I get it wrong, then I'm penalized, right? But in reality, what's actually happening, this is, you know, in the field we call it behavioral cloning or imitation learning. It's the same thing as cargo culting, right? The LLM never learns why the next word is the right answer. All it learns is that when I see something that is similar to the previous set of”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Do you agree, though, that there's an inherent limitation in large language models and decision making and executing things? When I see how LLMs, even still, you know, the frontier ones still hallucinate, still make things up, confidently lie, it's terrifying to think of putting that technology in a construct where now I'm asking it to go do something in the real world, interact with my bank account, ship code, work in a science lab. When ChatGPT can't spell right, that doesn't feel like the future that we're going to get. And so I'm wondering, are LLMs it or is there more to be done here?”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“You today, and you talk to it about this problem, it's going to go and find all the scientific research and write you a perfectly formatted piece of markdown of what the receptor does and maybe some things you want to try. But that's not an agent. An agent in my book is a model and a system that actually literally you can hook up to your wet lab and it's going to go and use every piece of scientific machinery you have in that lab, read all the literature, pose the right optimal next experiment, run that experiment, see the results, react to that, try again, et cetera until it's actually achieved the goal for you. And the degree to which that gives you leverage is so, so, so much higher than what the field is currently able to do right now.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“So I feel for all the people who have been told over and over again that agents are the future and then they go try the thing and it just doesn't work at all. So let me try to give an example of what the actual promise of agents is relative to how they're pitched to us today. Right now, the way that they're pitched to us today is for the most part it's just a chat bot with extra steps, right? It's like, you know, at company X doesn't want to put a human customer service rep in front of me. So now I have to go talk to a chatbot maybe behind the scenes that clicks a button or you've played with a product that does computer use or something like that that is supposed to help me with something on my browser, but in reality it takes four times as long and one out of three times it screws up. This is kind of the current landscape of agents. Let's take a concrete example of I want to do a particular drug discovery task where I know there's a receptor that I need to be able to find something that ends up binding to this receptor. If you pull up ChatGPT,”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Think that if you look at it from the perspective of computing, so far the building blocks of computing have been can I rent a server somewhere in the cloud? Can I rent some storage? Can I write some code to go hook all these things up and deliver something useful to a person? The building block of computing is changing, right? At this point, the code's written by an AI down the line, the actual intelligence and decision making is going to be done by an AI. And so then what happens to your building blocks, right? So in that world, it's super important for Amazon to specifically be good at the agent's problem.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“So it's safe to say that for Amazon AGI means more than shopping for me, which is the cynical joke I was going to make about what AGI means for Amazon. I'd be curious to know when you joined and you were talking to the management team and Andy Jassy and still to this day how you guys talk about the strategic value of AGI as you define it for Amazon broadly. Because Amazon is a lot of things. It's really a constellation of companies that do a lot of different things. But this idea kind of cuts across all of that, right?”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“C12, etc. Like, we don't have to live in that box if that's what AGI does for me. I think it's way more interesting to look at the box of the space of all useful knowledge worker tasks, how many of them are doable on your machine, how can these agents do them for you.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Well, the way that I look at it is self-improvement is interesting, but to what end, right? Like, why do we as humans care if the AGI self-improving itself? I don't really care personally. I think it's cool from a scientist perspective. I think what's more interesting to me is how do I go build the most useful form of this super generalist technology and then be able to put that in everybody's hands? And I think the thing that gives people tremendous leverage is if I can teach this agent that we're training to handle like any useful tasks that I need to get done on my computer because so much of our life these days is in the digital world, right? So I think that's like, it's very tractable. Going back to our discussion about benchmarking, right? The fact that the field cares so much about, you know, MMLU, MMLU Pro, Humanities Last Exam.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“The OpenAI definition for AGI we had was a system that outperforms humans at economically valuable tasks. And while I think that was an interesting, almost doomer North Star back in 2018, Think we've gone so much past that as a field. What gets me excited every day is not how do I replace humans at economically valuable tasks. It's how do I ultimately build towards like a universal teammate for every knowledge worker and just like what keeps me going is the sheer amount of leverage we can give to humans on their time if we had AI systems that you could ultimately end up delegating a large chunk of the execution of what you do every day too. And so my definition for AGI, which I think is very attractable and is very much focused on helping people, is the first most important milestone that would lead me to say we're basically there is a model that can help a human do anything they want to do on a computer.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Okay, that brings me to this topic. I was going to ask it later, but yeah, AGI, you're running the AGI research lab at Amazon. I have a lot of questions about what AGI means to Amazon specifically, but I'm curious first for you, what did AGI mean to you when you were at OpenAI, helping get GPT off the ground? And what does it mean to you now? Has that definition changed at all for you?”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Way more than just code. Those just happen to be the first two use cases that we all know work really well for these models. There's so many more useful applications and actually useful base model capabilities that people haven't even started figuring out how to measure well yet. And I think the better question to ask now if you want to do something interesting in the field is what should I actually run at? Why am I trying to spend more time making this thing slightly better at creative writing? Why am I trying to spend my time trying to make this model X percent better at the International Math Olympiad when there's so much more left to do? And when I think about what keeps me and the people that really are focused on this agent's vision that we have going is looking to solve way more breadth of problems than what people have done so far.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Was a project called Lambda or MENA at Google in 2020 that basically was ChatGPT before ChatGPT but only available to Google employees. Even back then, we started seeing employees start developing personal attachments to these AI systems. Humans are so good at anthropomorphizing anything, right? And so I wasn't surprised to see that people form bonds with certain model checkpoints. But I think that when you talk about benchmarking, the thing that stands out to me is benchmarking is really all about at this point, people are just studying for the exam, right? We know what the benchmarks are in advance. Everybody wants to post higher numbers. It's like the megapixel wars, right? From the early digital camera era. Like they just clearly don't matter anymore. They have very loose correlation with how good of a photo does this thing actually take. And I think the question and the lack of creativity in the field that I'm seeing boils down to AGI is way more than just chat.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Is that maybe what is really starting to matter is how people actually use these things and the feelings and the attachments that they have to them. So open AI bringing back 4.0 because people had a literal attachment to it as a thing that they felt. And people on Reddit saying it's like my best friend's been taking it away. And so it really doesn't matter that it's better at coding or that it's better at writing, it's your friend now. And that's freaky, but I'm curious when you saw that and you saw the reaction to GPT-5. Did you predict that? Did you see that we were moving that way, or is this something new for everyone?”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, we have too much to cover. We can't get into multiple realities. But to your point about everything converging, it does feel like benchmarks are starting to not matter as much anymore and that the actual improvements and the models Like you said, are commodifying everyone's getting to the same point and GPT 5 will be the best on Ella Marina for a few months until, you know, Gemini 3 comes out or whatever and so on and so on. And if that's the case, I think what this release has also shown.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“In part, there's an explanation that one of my old colleagues at OpenAI, who's now professor at MIT, came up with called the Platonic Representation Hypothesis. Have you heard of this hypothesis? So, the platonic representation hypothesis is this idea similar to Plato's cave, which is really what it's named after, that there is one reality, but we as humans, for example, only see a particular rendering of that reality. Like in Plato's cave, it is the shadows that you see on the wall of the cave, right? And so that's the same thing for LLMs. LLMs see slices of this reality by the training data that it sees. every incremental YouTube video of, for example, someone going for a nature walk in the woods somewhere is all ultimately generated by the actual reality that we live in. And as you train these LLMs on more and more and more data, the LLMs become smarter and smarter, they all converge to representing this one shared reality that we all have. And so if you believe this hypothesis, what you should also believe then is that all LLMs will converge to the same model of the world. And I think that's actually happening in practice from”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT
“I think it really signifies a high level of maturity at this point. The labs have all figured out how to reliably tape out increasingly better models. One of the things that I always harp on is that your job as a frontier model lab is not actually to train models. Your job as a frontier model lab is to build a factory that repeatedly turns out increasingly better models. That's actually a very different philosophy for how to make progress. I build a better model path. All you do is you think about, let me make this tweak, let me make this tweak. Let me try to glom on people to get a better release. If you care about it from the perspective of a model factory, what you're actually trying to do is you're trying to figure out how you can build all the systems and processes and infrastructure to make these things smarter. But with the GPT-5 release, I think the part that I found most interesting about it is that a lot of the Frontier models these days are converging in capabilities.”
2025-08-21 · Decoder with Nilay Patel · Amazon is betting on agents to win the AI race · IDENTIFIED FROM THE TRANSCRIPT