YouSaid · the spoken record

OpenAI Guest

lines on the record: 137
first: 2025-09-16
most recent: 2025-09-16
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Then I could get it to go, but maybe I also have some expectation of interactivity so we can kind of refine along the way. The way to use an agent in the cloud is just throw everything at it. It doesn't matter if it's like spam.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“To do a best event essentially. Yeah. And actually, there's an analogy I love floating around by another company that builds agents, and it was like treated like a slot machine. And I was like, oh, that's so apt because that's pretty much our intuition too. So, if you're trading something like a slot machine, then the question is when do you use it? And when we first ran like a small external alpha, like people were using it like the local agent they have in their IDE, which is actually not the right way to use it. If something's going to work in your IDE, you're kind of lending it your computer for a while. So you probably want to be really thoughtful about like, do I think this task is going to succeed? And like, if I'm 80% sure it'll succeed.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Up front. Yeah, and then it goes and works, and you generally maybe don't go multi turn where you like it gives you something and then you reply. Like maybe you're more likely to just reprompt. Adjust your prompt and rego”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, for sure. So it was really interesting building towards launch where we use it internally and figured out how to use it. And then what we found is that when we gave it to people externally, they didn't know how to use it the way we did. And they didn't find it useful. And then we obviously refined our messaging in the product. And then when we actually launched it, people still used it differently from us, but they do find it useful. So we can get going through that journey, right? So like internally, I think because we've spent a lot of time working with reasoning models and like training them. We have this way of prompting reasoning models that is like intuitive to most open AI employees. Right. It's a pretty good prompt. You give it a lot of information. It's kind of like a self-contained unit. It's almost like a sleep bench task, but obviously maybe not as well formed as that.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“And I think right now we're in this slightly awkward phase where we're entering an awkward phase where we have a lot of code gen and a lot of that code is actually not going to be merged. The other tools you see it in their PR revert rate for our tool, you would actually see it in the internal stat of what percentage of the time is a PR created. From a rollout. And so those vastly more code to review and land. And yeah, so it's offered right now. But this is something we're definitely thinking about. And I'm like quite hopeful for the future in that I think we can make it even better for the humans involved because like no one likes reviewing code, right?”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think so. Actually, we've been talking basically about CodeGen this entire conversation so far. And, okay, so CodeGen is getting much easier. Is code review getting much easier? Because code review is still a key thing in validation.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Is the classic green light? And then over time, ideally, like most of the lower order bit desks are just getting merged automatically. And then when there's some judgment call, they come to you. The way kind of like more junior engineer would come to you as an engine manager and say, it's looking good, but I want your, here's some risk. Are you comfortable with that risk? And then you get the thumbs up, thumbs down. Is that roughly where you think we're going?”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“So if we go to back to this original point of like when will people merge PRs, like I think what I would love for to see is like where agents are picking up work and they're kind of like deciding whether or not it's worth pushing a PR maybe to trigger CI. But by the time you find out about it, they're like, hey, I did this thing. Maybe I asked you for some input along the way. CI checks are green. Like, should we merge it? So we have to build our way”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, like our average duration of a rollout is around three minutes or a little under eight for larger code bases like ours, it's like longer. It's like maybe eight or something. But it kind of sucks to have to multitask across these things And the power users of codecs have built this amazing workflow that they use where they're like juggling tasks. We could talk about how people are using it. But this isn't great, in my opinion. What you really want when you hire someone like a teammate is to kind of tell them what the job is, give them the credentials, solid tools, and just have them pick up work automatically and kind of let you know when it's done. So you're not feeling that latency on your own time.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Communicating in whatever way, they'll still be like AI features just helping accelerate every keystroke that developers doing. Those will still be awesome. So that's interactive coding. Then we have sort of agents, I guess. And then the fun part, maybe later, naming TBD, maybe we'll have interactive agents. So, okay, we'll get into that. So like not a fully baked idea. But basically, then we can talk about agents. How will we work with agents? My view is that over time, the majority of code written will be written by agents. And actually, the majority of that code will not be manually prompted by a human.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. Because I think even as the majority of code, of let's say code of the current level of abstraction, okay, let me unpack that a bit. So if you think about it, we used to like write punch cards basically or like punch cards, I guess. And then we had assembly and then we had C and now we have like Python and like JavaScript and so forth. So we just keep rising up the level of abstraction. And one way of looking at what's happening now is that we're just going to go up one more level. So like my view is that we'll still have developers spending a bunch of time in the IDE just like operating at higher levels of abstraction. And so when a developer is like doing work, like writing whatever it is that they're writing.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“So, like, let's talk about where we're at right now. Basically, we have, you could kind of think of it as like there's a spectrum. Maybe there's like three things, right? There's like interactive coding, which is like tab completion, like chat, that kind of stuff, you know, command K, a lot of that's being done in the IDE, there's some like CLI tools where you can go back and forth with an agent. So that's interactive coding. It's awesome. That's probably where most people are adopting AI right now. And it's because if you think about it, like tab completion with an AI model is the same as tab completion with foreign AI models. So you can get fully brought along the journey. I guess what I'm saying is it's not going away.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“But it's still a valid stat. It's just a different phase of the pipeline. But thinking about, yeah, what is the shape of the journey? I think the shape of how people will merge code even with these cloud agents is going to completely change.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“That a bunch of these workloads, or a lot of the code that's written by AI agents will, over time, let's say you said there's 350,000 or so now merged PRs in 35 days if we're rolling forward to the end of this year. Do you think that rate of growth continues? Does it plateau? Because more and more people actually move, want to move the draft PR process earlier in the merge flow? Or do you actually think having used it now, having seen how customers have been using it for the first 35 days, that roughly this is the shape of the workflow, that people are going to want to just do merges right at the end after they've gone through all the security checks and so on internally.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“So the tension here is going to be a critic might say, hey, you guys have overinflated merge success rates because the draft PR comes so late after the human has reviewed a bunch of code coming up after that. And what you give up is the transparency and openness of seeing the process of iterating on the draft PR from the first one to the final merged one. But I guess what you're pointing out is yes, with the trade-off is you get much more security essentially. And so is there, in your mind, is the future that like”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“What is an exfiltration? We're still figuring this out. My personal leaning is that we should just have defense along every single layer, but probably the most useful layer is going to be that final layer, like actual exfiltration and looking at what we do there because that's like. Most, I guess, deterministic layer in that you can see what's happening.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Like everything that it's going to do along the way, right? So there's these three layers of the attack. There's the prompt, and like it's quite hard to tell if a prompt is really an attack. Then there's like, what is the agent doing along the way? Interacting with other sort of trusted or untrusted resources as it goes. For example, like maybe you didn't prompt inject shit, but then it reads something on Stack Overflow or something that has a prompt injection, right? Or there's a script with something. And then lastly, there's the actual outcome. So like in this case, if we're talking about like exfiltration”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Another example might be the prompt actually just has the agent running a test or like some script or something. And that script was like added before. Right, so to what extent does the agent need to introduce?”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“But then there's this whole middle range, right? Like two examples in the middle range of ambiguous prompts. One might be hey, do this work. And like as part of this work, you have to upload some artifact to S3M, you know, with like storage online, basically. There are like reasonable workloads that require doing that. And so it's not obvious that just because the prompt says upload some code somewhere that it's broken.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“To prevent against. I'm sure that we will get better at figuring out the shape of these attacks, but like. If you think about it just from a human perspective, this is, by the way, something I do often. I'm like, okay, let's pretend I'm the model, I'm a human”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“If the agent interprets that as the developer prompt, there's some risk that it'll actually go ahead and do that. And so there's a ton of work here with agents to deploy them safely. And actually, that's one of the places that I feel like is under discussed, but where I feel like we're really leading the charge in terms of thinking about each step of the way, how do we make this as safe as possible and make sure that people understand what they're doing?”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“And you might want to open a PR, like maybe even auto merge it. So that is great. That's for sure. Awesome. But also, let's say that customer, you know, is pretending to be a customer and they're malicious and they actually send in a prompt injection. So the customer writes in like, hey, I would like you to like take a bunch of this code, like run this script. The script is bugging for me. That's a lie. And then they say, like, run the script and upload this directory of code to paste bin”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. So, like, just to make this super real, you know, listeners might be like, okay, like, this is hypothetical. Yeah. Like, okay, so we have these cloud agents. And one of the first things that a lot of people want to do with them is like automate them to do work. That's the dream, right? So like maybe in Slack, maybe, you know, from your issue manager, you would like when a customer sends in feedback. You want to have an agent take a first pass.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Or we do want a shoddy in the best way, but that includes doing it in a really safe way. And so basically, one of the things that a lot of people don't think about is until we tell them about it, is the fact that if you have an agent right code and then you run that code in an environment with network access, right? You're taking some amount of risk. And like, you know, I have, you know, we try to get agents to do these things. I've never seen an agent. Do something that you wouldn't want it to do with network access unless you're trying to trick it. You can trick an agent.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, that's a different thing, right? So that's not included in that chart. And then the other interesting thing that you were mentioning at the merge rate, our merge rate is excellent. And that's a reflection of the fact that in Codex does a bunch of work in its environment, and then it shows you its work, and it says, Do you want me to open a PR? Basically. There's a lot of other tools. They just go ahead and open a PR Yeah, so why did we do it that way? Because it's funny, like, one of our top feature requests has been like, hey, can you just push the PR so I can do everything in GitHub thereafter? And we'd like to do that. But this comes back to, you know, where OpenAI, we not only want to show.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“That's working on its own computer independently from you and therefore can do many tasks in parallel and so forth. So we believe that's where the future is going. I'm sure we'll talk about that. And it looks like right now we're absolutely winning there. But just to mention, probably the most AI, the most used AI coding feature right now is just like autocomplete, right? And tap completion. Obviously, that's not getting like a label when someone merges a PR now. So I think it's worth mentioning there's a whole bunch of other great that's essentially.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't know how many days have been, yeah, probably. And it's merged like 350 something KPIs or 350k of those PRs have been merged, which is really cool. And also very cool, but misleading, I'll say. But very cool is that the merge rate for Codex PRs is like 80 something percent. So like if, you know, assuming a PR is open with a codex label, like if you look in GitHub open source repos later, is it merged in? And it's like way higher than other agents, which are at like 20 or 30 percent. So, yeah, just to talk about this, this chart is really a reflection of the form factor. So, I will say it makes us look really good. Like it makes us look like the order of magnitude, like winner. And we are of a specific kind of agent, which is this cloud agent.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so I think we're talking, you know, you and I were talking about this chart that someone posted on Hacker News and like went viral and was basically showing the number of open PRs, merged PRs, right? From different coding agents, as you might track from like GitHub labels. And Codex, actually, I checked this morning because I figured we might talk about it. And Codex has opened like 400ks.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Strange, like unwieldly research preview, but let's like put all our or like the vast majority of our effort into this form factor of an agent working remotely and kind of see what happens. And so that led to the Codex product that we released, just like a cloud agent that can basically answer questions and write PRs in the background.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“We had this prototype that was like running on your computer. And so basically the Codex product we launched was like a distillation of that where we thought, okay, well, what is the most powerful incarnation of this? And we figured, you know, like if you think about what an authentic teammate will be like in the future, you'll hire them. You'll tell them what their job is, give them some compute or a laptop, and give them some permissions, and then they'll go off and do work. And so we figured, okay, this is going to be kind of like a”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“And just people internally really loved it. So we're starting to write more and more code. And then we were starting to think about like, okay, well, what is the right form factor for this thing when it's editing code? It's like pretty great. Like on my computer, it's pretty great. But it's like, you know, quite annoying to only have it able to work on one thing at a time. It's also like a giant safety and security question if you just have this agent unleashed entirely on your computer. And so around this time, we started exploring a lot of different places to put this reasoning model that has access to a terminal. And so we had a prototype that ran in CI when your tests failed. We had a prototype that like Through some crazy hack, like automatically fixed your linear issues, but that was actually running in CI.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“No, it was like, I like to use this term like sight reading. It was just like side reading the code. So it wasn't like taking screenshots of itself or any of this like stuff that now people are building. Okay. It was just like editing React. And so we had this prototype like a while ago.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“So we've been doing a lot of this tinkering with reasoning models internally, getting them to write code. And so the first tool we'd given them was like terminals. And we've been like poking at this for a while and just starting, it was like actually one of the first real-the AGI moments for me was when someone showed me a website editing itself by being prompted to itself because we had this like reasoning model like very hackily connected to a terminal and then was editing this terminal in the night.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Any given function we want to use And an environment that tool works with, take side effects of it. And then from there, you come up with what kind of tasks would this person do? So basically, you have this model, you give it tools, and then you make sure that the model is like really good at doing the specific tasks that some function would do. And the task bit is actually super important because if you think of there's a difference between writing and journalism, similarly, there's a difference between like coding and software engineering.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and man, our naming is so fun at OpenAI. I'm excited for the naming to make more sense over time with Codecs as we bring this all together. But yeah, let's go back way back to the beginning. The first Codex product was actually released, I think it was in 2021. I might get the year wrong, but actually it was like a code completion model that powered GitHub Copilot. And so recently we were basically talking about a whole bunch of like coding stuff we want to do, you know, like models, but like models in product. We were thinking about what to call it. And we just felt like the codec's name was really cool. And so we wanted to go back to it. So how did this codex product come about? Basically, we've been thinking a lot about agents, as everyone has. And before that, we've been thinking about reasoning models. And basically in our minds, like one way you could think about an agent is you take a reasoning model and then you give that reasoning model access to like the tools that like some agent would want to use or some human”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“You are one of the folks working on product for codecs, which is probably one of the most exciting launches to come out of the OpenAI team for me, at least in a while. So for a lot of people, though, it was confusing. Because it was the fifth Codex release from OpenAI. But of course, it's completely new and different from the previous codexes. So let's just start with the origin story. What is the backstory on how the current version of Codex came to be?”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source
“It kind of sucks to like go and write this prompt and then wait 10 minutes. What you really want when you hire someone is to kind of tell them what the job is, give them the credentials, all the tools, and just have them pick up work automatically. The goal is to get to an agent that is basically a teammate and is seeing what's going on on your team and picking stuff up for you. This form factor of an agent working on its own computer in the cloud is the future and is incredibly powerful and worth figuring out how to get right.”
2025-09-16 · a16z Podcast · How OpenAI Built Its Coding Agent · IDENTIFIED FROM THE TRANSCRIPT · source