YouSaid · the spoken record

Isa Fulford

lines on the record: 65
first: 2025-08-08
most recent: 2025-08-08
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Yeah, I think with GPT 5, the thing that's the word that's like been in my mind throughout all of this is like usable. And I think the thing that we're excited about is getting this out to everyone. We're excited to get our best reasoning models out to free users now. And I think just getting our smartest model yet to like everyone. And I'm just excited to see what people are going to actually use it for.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I feel like with every research release we do, and when people figure out what happened there, they're like, oh, that's so simple. Like, oh, I should like that. Obviously, that would have worked. But I think it's like knowing to try that obvious or like at the time not obvious thing that is obvious in hindsight.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I think also I've been surprised by how often the thing that is the most simple, like easy to explain is the thing that works the best. And so sometimes it seems very obvious, but it's quite hard to get the details of something right. But I think usually good researcher tastes just like pretty. Simplifying the problem to the dumbest thing, or the most simple thing you can do.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I think taste is quite important, especially now that it is like, like I said, like our models are getting smarter, it's easier to use them as tools. So I think having the right direction matters a lot now and like having the right intuitions and like with the right questions you want to ask So, I would say maybe it matters more now than before.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I guess if you tie it to the mission, it's like we're trying to make the most capable thing and we're also trying to have make it useful to as many people as possible and accessible to as many people as possible. So like in that framing, I think it makes a lot of sense”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I guess one thing that I think is unique about OpenAI is that you're both very much a consumer company by revenue, et cetera, products, but also an enterprise company. How does that internally, like what would you guys consider yourself? Or is that even just the wrong paradigm to think about?”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it's different for different teams, but my team collaborates so closely with the engineering team and the product team and design team in a way that I think sometimes research can be quite separate from the rest of the company. But for us, it's so integrated. We all sit together, you know, sometimes the researchers will help with like implementing something. I'm not sure that engineers are always happy about it, but we'll try, like get out of the front end code. And vice versa, like they'll help us with things that we're doing for model training runs and things like that. So I think. Some of the product teams are quite integrated. I think it's for post training. It's a pretty common pattern, which I think just lets you move really quickly.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, we definitely reward agency, and I think that's like what's been true. And I think, especially in the research side, the teams are quite small. Like when Issa was working on deep research, it was like two people still. So I think we still do that on the research side. Like most research teams are quite small and nimble for that reason.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Few thousand for sure. Yeah, when I joined, it was also Hundred before ChatGPT. So it's obviously very different in how all of your friends have heard of what you work on. But I think culturally, obviously the company is much bigger. I still think we've maintained it. It still feels very much like a startup. I think some people who come from a startup are surprised that like, oh, I'm working even harder than when I was working. I was the startup that I founded. I think ideas can still come from anywhere. And if you just like take initiative and want to make something happen, you can. And this doesn't really matter how senior you are or anything like that. I think we've been able to maintain that culture, which I think is pretty special.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“When I first joined OpenAI, the applied team was 10 engineers or something. It just like we didn't really have this product RM. We had just launched the API. It was just a completely different world. And I think AI is in most people's mind now after ChatGPT, but I think pre-ChatGPT, like people didn't really know what AI was or really like thought about it as much. It's kind of cool working in a place that like my parents know what I do now and like it's like that's really cool. And I think the company obviously is just a lot bigger, but I think with that we can just take a lot more bets. I think when I first joined OpenAI there were obviously way less people. Like it was much, much smaller. It was around like 200ish people and I think we're close to.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Okay, like you're stalking us. Do you want to interview her? But yeah, I think it was pretty clear to me, but just how much I was using GPT-3, which wasn't even, compares what we have now just appeals in comparison. But I was like, from then I was hooked and just trying to figure out a way to work here.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I think for me, it was also before I started working at OpenAI using, I think I first learned about OpenAI in an AI class or something, or some kind of computer science class, and they were saying like, oh, they trained on the whole internet. It's like, oh, that's so crazy. Like, what is this company? And then started using GPT-3 in the, I think I was a power user of the open AI playground. And at a certain point, had early access to these different open AI features like embeddings and things like that and just became this like big open AI fan, which is like a little embarrassing, but you know, it's fine because it got me.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Honestly, I kind of have this moment before I joined Open AI. Like, I think with the scaling laws paper with GBT3, I was just kind of hit me that if this exponential is true, there's not really much else I want to spend my life working on. And like I want to be part of this story, like I think there's going to be so many interesting things unlocked with this. And I think this is probably the next step level in terms of like technology that it kind of made me realize like, oh, I should probably go start reading about deep learning and figure out how I can get into one of these labs.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“But I think it kind of clicked into me that like maybe there was actually something interesting happening here. We gave early access to about 50 people, most of those people being like people I lived with at the time. And there are two of my roommates just used it all the time. They just like would never stop using it. They would just have these long conversations and they would ask it like quite technical things because they're also AI researchers. And so I was just like, oh, this is like kind of interesting. Like I don't know. And at the time we're kind of thinking like, okay, we kind of have this chatbot. Should we make this like a really specific like meeting bot type of thing? Do we like make it a coding helper? But it was interesting to see my two roommates just use it for anything and everything and just like literally be chatting with it like the whole workday as they were using it. So I was like, oh, this is kind of interesting. But then it was also interesting to see like the majority of the people that I gave access to on that 50 person list like didn't really use it that much. But I was like, oh, it's there's clearly like something here, but it's like not quite maybe for everyone yet. But there's something here.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Honestly, with WebGBT, the main thing we were just excited about was trying to ground these language models. We had so many issues with like hallucinations and the model just saying random things and like the fact of we didn't really do mid-training then so like the fact of like how do we make sure the model is actually up to date like most factually up to date so then that's kind of how we thought about like oh let's give it a browsing tool I think that makes sense And then yeah, like I said, that kind of went on from like, oh, actually, I want to keep asking questions. So what does a chatbot would look like? But at this point, I think there had been a few chat bots by a few other companies. And I feel like a chatbot is also like a very common AI thing to think of. But they're quite unpopular at the time. So we weren't really even sure that this is actually something useful for people to work on or like people to use or will people be excited about this. Is this really like a research innovation that we are remaking the Turing test here?”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I think with your pre-training runs, these are the big runs. These are the massive ones. Like what we're building all these giant clusters for. So you can kind of think of mid training as literally for like middle. Like we do it before, after pre-training, but before post-trading, you can kind of think of a way to extend the models intelligence without having to do a whole new pre-training run. So this is mostly just focused on data and off of the pre-training models. So this is a way for us to do things like updating the knowledge cutoff of these models, right? So when you pre-train it, you're kind of like, okay, shoot, now we're kind of stuck in this date and we can't never update it again. It doesn't quite make sense to put all that data into post training. And so mid training is just a smaller pre-training run to help expand like the model's intelligence and up-to-dateness.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I think one cool thing is for, for example, for initial deep research, there's not really any data sets that exist for browsing in the same way that you have a math data set that already exists. So we had to create all this data. But once you have good browsing models or good computer use models, you can bootstrap them to help you make synthetic data. So I think that's a pretty promising area.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I think one thing is so free trading, it's based on like what data is available, right? And so I think when we've done these free trade, there's not much data out there to begin with with people using computers. Like computer usage is not really a thing that there's lots of data out there. And this is something we actually have to seek out now that this is a capability that we want. So I think that's actually probably a big one. Just for general improvements of computer usage.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Having oversight during training is also like an interesting area. I think there's just like new things that we have to develop to, you know, push these agents even further. So yeah, I think that's part of it. And then also like every time we have a smarter base model or something like this, it improves every model that's built on top of that. So I think that will also help, especially with like multimodal capabilities, as Tina said, with computer use because it's like just literally looking at screenshots of a web page and it's like it's a little interesting because the way that humans like focus on specific things, it's like it's a lot to expect a model to just like take a whole image and be able to like know everything about the image when like when we're looking at something we'll like focus on a specific thing”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I think a big part of it is the things that we train on we're often really good at, and then sometimes the things outside of that, it can be a bit sometimes it's good at those things, sometimes it's not good at those things. So I think, yeah, creating more data across like a broader range of things that we want it to be good at. I think also what's interesting with agents is we have this when something is doing something on your behalf and it has access to your you know your private data and the things that you use it's kind of more scary the different things it could do to achieve its final goal You know in theory if you asked it to buy you something that and like make sure that I like”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I think we hear this with GPT 5 internally when people are testing and they're like, Oh, I thought I asked a really hard question. I feel like I'm a little bit insulted that it got two seconds and it doesn't even want to think at all.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I do think we have a specific number. One thing that's interesting is I think sometimes people just biased to thinking that the longer answer is more like thorough or is done more work for it, which I don't necessarily think is the case. For example, always gives you a really long report. But sometimes for me, I don't want to read this whole long report. I actually don't like that. And so agent, like it will only give you a long report if you ask for it. But I think sometimes people, since now that you're still always getting a really long report, they're like, wait, I've been waiting. Where's my long report? But sometimes it's really hard to find a specific piece of information and would have also taken a human long time because it's in like page 10 of the results, whereas where it finds this information. So I think it's interesting also how you can condition people's expectations with a product so that when you change or like with deep research it always thinks for a really long time which again I don't necessarily think is a feature but I think now people are like really used to the amount of time that”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Because, yeah, I was going to say, is there any sort of rule of thumb? I'm sure it's constantly shifting where as long as you're 10 times faster than it would take the human to do, they're willing to wait for it? Or is that just constantly shifting sand?”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“It's interesting because I built the retrieval on ChatGPT and was on the browsing team before this. Tina was also on the browsing team. We were always making these trade-offs and optimizations for latency. And so we were thinking, how can you best fill the context with information you've retrieved so that the answer is pretty good in a few seconds? And so I think with deep research, I was just very excited to remove latency as a constraint. And since we were going for these, we're going for these tasks that are really hard for humans to do when we take humans many hours to do. I think we felt like, you know, if you asked an analyst to do this and it would take them 10 hours or two days seems reasonable that someone would be willing to wait like five minutes in your product. So I think that was the we just kind of made that bet and luckily it seems like it's the case. But I do also think that, you know, initially people were like, oh, this is amazing. It's doing all this work that would have taken me so”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't know if you would agree with this, but it felt like a revelation to me at least at the beginning of the year that people were willing to wait. Because you kind of think about, oh, we want it faster, like the value prop of this tool is that it gives me the answer fast, right? That was sort of very 2024. Clearly, this paradigm has shifted. People are willing to wait for high quality, high value answers in work. How do you think about the trade-off between how long something take, how long you take to get something back to the user versus what you're actually the value that you're providing? And like, what do you think is the ideal frontier for something like that?”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“That's incredible. On the shopping piece, I now do not make a single large ticket purchase without having ChatGPT put all the options in a table for me along the dimensions I care about. It's incredible. But I want to push on the async piece because”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Also, being better at creating and editing artifacts like docs or slides and spreadsheets, because I think so much of like the work that's useful that people do in their jobs is basically just research and making something. But then also I'm personally like love all the consumer use cases, like making it better at like shopping or planning a trip and those kinds of things are like also really fun. And so that also involves like taking an action, which is interesting because it's It's kind of the last step often of a task. And it's maybe a task that would take less time for a human. And it's like actually very hard research question to get it to do something or book something or use a calendar picker. But yeah, once you have the end-to-end flow working really well, it can basically do anything.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I guess my very general definition would just be something that does work, useful work for me on my behalf with, I would say, asynchronously. So like you'd kind of leave it and then come back and either get a result or like a question about what it's doing. And then in terms of, I guess, roadmap for agents, I mean, longer term, you want it to be able to do anything that, you know, a chief of staff or assistant or something like that would do for you. But I think in the more immediate term, there are a lot of new capabilities that we launched in ChatGPT agent that we just want to improve. So one of the main capabilities is deep research. So just being really good at synthesizing information from the internet, but also I think we can improve capabilities on synthesizing information from all of the services that you use and like private data that you have. And then”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I think a lot of it is not just about the model capability, but it's actually like how you set it up in a way to do things. Like I'm sure that you could build something that's like monitoring your Humio or like Datadog, whatever, like with these current models, it's just like setting up the harness to make that possible. And same for a genetic task. I think a lot of things that will be quite useful will be when the agent like proactively does something for you, which I don't think is impossible today. It's just not set up that way. But eventually as it proactively does things for you, then we might get feedback on whether that was useful and we can make it even better at like triggering.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I think GBT5 is great because, like, yeah, within a couple minutes, maybe you get a full fledged app, but then what would it look like if you actually gave it like an hour, like a day, a week? What can it actually get done? And I think that's, there's going to be a lot of interesting stuff. We're interested to see what will happen there.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I don't know about the exact thing of DevOps, but I do feel like with the models getting much smarter, one other thing that came to my mind when you asked me the question is like longer running tasks and like things like that, I think like.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Maybe just to build on that question in terms of what it can't do today, but what you would sort of direct future research toward, if you look at coding, something like end-to-end DevOps, for example, that feels like the logical next set of capabilities, do you guys think we'll get there in, I don't know what you'll name it, but 5.5 or GPT-6. How far are we from?”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, as I said, you could ask the agent to do anything, but it's not capable enough to do everything you want it to do yet. We take a conservative approach, especially with asking the user for confirmation before doing any kind of action that's irreversible. So like sending an email or ordering something, booking something. So I think I can imagine quite a number of tasks where you'd want to take like bulk actions, which you might not be able to do right now because it would ask you every single time. But I think as people get more comfortable using these things and as they get better and you trust them more, you might allow it to do things for you without checking in with you as much.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“With 3.5 when we first released it, the most common use case for me then also was still just for coding. But now even though 4 was better at coding, I feel like the jump between four and five in terms of like breadth of ability to do things is just way different and way more. And you can just handle a lot more complex things than like before, like with the context length being much longer as well. I think the jump to four to five to me is like much bigger.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, at least one thing for me and my usage of it is sometimes I'm wondering if I have hard enough questions to ask it to actually like highlight the difference. Because when it gets to a point where it's just answering what you need so well, it's like almost harder to tell the difference in some areas. But with writing, yeah, I've been using it for a few weeks and it's just kind of blown me away in a way that models previously haven't.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I guess people adapt to things rather quickly, in my opinion, with technology, and it is really easy. And I think because of form factor is so easy, even with like new tools like deep research and chat GPT agent, it's like presented in such a easy way that people already know how to interface with. I think as long as that's true, even with the model's getting much smarter than us, like I think it'll be, it's still going to be quite approachable to people.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, it seems like people addressed really quickly, don't you think? I feel like ChatGT got released and everyone was like, wow, that's so cool. But then you just kind of take it for granted that you literally have this like wizard in your pocket. You can like ask it whatever random thought you have. And it just pops out like a good essay and you're like, oh, okay, cool. That's what's happening.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Compared to other things I'm better at. But it's so great to have this tool to help me like craft whenever I use it literally for as simple things as like Slack messages to figure out like how to phrase this well and help me give me some iterations how to say something to the team.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“That's one of my favorite improvements in GBT5. Writing, I honestly find it's very tender and touching, especially for a lot of the creative writing that we want to do. We were thinking through like a bunch of different samples for the live stream and like every time I was like, oh, that's like actually that like hits like and it's like spooky and I'm just like, oh, this feels like someone, like someone should have written this But I think it's really cool because you can actually really use it for helping you with things. My example I did in the live stream was like writing, helping me write the eulogy, something that's kind of hard to write, especially if it's writing isn't really something a lot of people are good at. Like I'm personally a very, very bad writer. I think it's”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, like there's some generalization from training on one website to another, but if you want to get really, really good at something, the best thing to do is just train on that exact thing. Yeah, I think we're definitely just constrained by how things that we can represent in a way that we can train on. ChatGPT agent, for example, has such a general tool. It has a browser and a terminal. And between those two things, you can basically do most of the tasks that a human does on a computer. In theory, you can ask it to do anything that you can do on your computer. It's obviously not good enough to do that yet. But with the tools it has in theory, you can push it really, really fast. So now we just have to make it really good at all those things by, you know, training on training on way more things.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think in my opinion, I do think there is a lot of value in getting really good tasks and getting really good tasks requires really good RL environments. I think the more complicated and the more realistic, the more simulated we can make them, I think the better we'll get. And I think we're kind of saying that like tasks matter more at this point given the fact that we have such a strong algorithm. So I think the data creating data and figuring out like the best tasks to train on is like one of the big questions we have.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Maybe on the data topic, we've been talking a lot about RL environments. It's a popular space for startups who all want to work with you guys. And I was curious just to get your thoughts on this since you've been data pilled, but what are the bottlenecks that you see for the next stage? Is that, I mean, maybe tying it to RL environments, is there sort of a lack of good, realistic RL environments that that's sort of the next frontier, which maybe creates an opportunity for these startups that once you Sort of are able to really work within an environment that takes a long time to build. These are not sort of built in a day or two that you can actually automate labor to the full extent of compute, you know, the way that you would need computer use to do.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“My opinion, I'm very data pilled. Like, I think data is very important. I think deep research was so good because Issa put so much thought and careful attention to the data curation that they did and thinking about all the different use cases she wanted to have represented. So I'm on team data.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, it's the same thing with everyone was talking about agents, but we didn't really have a way of actually training useful agents. I mean, I think everyone was talking, there were all these agent demos, but nothing that actually really works. But I think when we saw the reinforcement learning algorithm working really well on math and physics problems and coding problems, it became pretty clear, like just from reading through the chain of thought, like, okay, this thing's actually thinking and reasoning and backtracking and to build something that's able to navigate the real world, it also needs to have that ability. So we realized, okay, like this is a thing that's going to actually let us get to useful agents. And so I think it's interesting at OpenAI because you have people pushing foundational algorithms, getting really good at math, getting a gold medal and the IMO. And then on post-training, we'll often take those methods and try and figure out how to make things that are most useful and usable to all of our users.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think we've kind of seen this with the progression of even the models that we've had in ChatGPT. Like, as the model gets smarter, it's better at instruction following. It's better at tool use. And more things get unlocked as we just continue to make smarter models. So I think a good chunk of our team also does focus on just getting general intelligence up because I think the wins that we get from there are, like Isa's saying, pretty great whenever we get a new base model and it's just seeing like, oh, wow, suddenly this clicks. It works. And I think we kind of saw that moment with like operator because we had been working on computer usage, but I think it was hard to find like get the model to actually without the multimodal capabilities to really support it, like you couldn't have something like operator when it launched.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Like all different kinds of users. So, yeah, I mean, I think if you choose a capability, that's quite general. Like online research, you just have to make sure that you represent a distribution of tasks across loads of different domains if you want to get good at all of them. But then, yeah, sometimes it is, it's hard to decide to focus on one specific thing because there are just so many different verticals that you could go could choose from. But I think in some cases maybe like coding will be really important. So then, you know, a specific team will focus on coding. But I think in general, because the capabilities are so general, usually like the next model improvement just kind of improves performance on a pretty broad range.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm going to be so happy to try to hill climb that. Yeah. I like what you said about starting with the capabilities first. How do you prioritize which you actually are shooting for? Let's say there's this dimension of maybe deeper into everyday use versus getting much deeper into the expert use cases. How do you think about that trade-off? What does that trade-off mean practically speaking? And what do you guys prioritize when? I mean, I think it's pretty unique OpenAI to be able to work on something that's so generally useful. I mean, it's like everything they tell you not to do at a startup. It's just like your user is anyone like for deep research. We wanted it to be good across every single domain someone might want to do research in and I think you only have the like privilege of doing that if you work at a company that has like huge distribution.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“I think we make this joke a lot internally that if you want to nerd sipe someone into working on something, you just need to make a good eval and then so happy to try to hill climb that.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“You go to editing spreadsheets. And then if evals for those things don't exist, we try to make evals that are representative measures of that capability in a way that's actually going to be useful for users. And then a lot of those are internal. We'll collect them maybe from human experts or try and synthetically create examples. Or we'll actually look at usage data. And then for us, we'll just try and hill climb on those.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, actually, I think Greg made this comment about how he was comparing the last model to this model and the benchmark went from 98 to 99. He's like, clearly we've saturated the benchmarks, at least on that front, which I think is instruction following. What benchmarks do you pay attention to? Like, how do you guys think about evals, right? Because given you're already saturating what's out there to a large extent or doing very well along those dimensions, what actually gets you to push the frontier is that before the, I mean, so usage would be kind of post the model release, but before you get there, what are you guys looking to internally to help guide you? Is it a lot of internal evals that you've created? Is it early access to startups, seeing what they think? Maybe it's a combo of all the above, but how do you weigh all those things? Yeah, I mean, I think on our team, we really work backwards from the capabilities we want the models to have. So maybe we want it to be good at creating slide decks or something.”
2025-08-08 · a16z Podcast · GPT-5 and Agents Breakdown – w/ OpenAI Researchers Isa Fulford & Christina Kim · IDENTIFIED FROM THE TRANSCRIPT · source