YouSaid · the spoken record

Kevin Weil

lines on the record: 102
first: 2025-04-10
most recent: 2025-04-10
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Not that many, actually. I don't know. 25. Maybe it's a little more than that. But my personal belief is that you want to be pretty PM light as an organization just in general. I say this with love because I am a PM, but too many PMs causes problems. You know, we'll like fill the world with decks and ideas versus execution. So I think that's a good thing when you have a PM that has, that is working with maybe slightly too many engineers because it means they're not going to get in and micromanage. You're going to leave a lot of influence and responsibility with the engineers to make decisions. It means you want to have really product focused engineers, which we're fortunate to have. We have an amazingly product focused high agency engineering team. But when you have something like that, you have a team that feels super empowered.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Products you're trying to solve or the problems you're trying to solve, building evals for them, using those evals to go gather data and fine-tune models to get them to be better at these use cases that you're looking to solve, it's a huge amount of back and forth to do it well. And I think the best products are going to be engroduct design and research working together as a single team to build novel things. So that's actually how we're trying to operate with basically anything that we build. It's a new muscle for us because we're kind of new as a product company, but it's one that people are really excited about because we've seen every time we do it, we build something awesome. And so, you know, now every product starts like that.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“And it was just a way that we were going to let people play with and iterate on the models. And so we were primarily a research company, a world class research company. And as ChatGPT has grown and as we've built our B2B products and our APIs and other things, now we're more of a product company than we were. I still think OpenAI should never be a pure product company. We need to be both a world-class research company and a world-class product company. And the two need to really work together. And that's the thing that I think we've been getting much better at over the last six months. If you treat those things separately and the researchers go do amazing things and build models and then they get to some state and then the product and engineering teams go take them and do something with them. We're effectively just an API consumer of our own models. The best products, though, are going to be, it was like I was talking about with Deep Research. It's a lot of iterative feedback. It's understanding the”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“It's an area where we're evolving a lot. I'm really excited about it, frankly. I think if you go back a couple years when ChatGBT was just getting started. Obviously, I wasn't at OpenAI. We were more of a pure research company at the time. ChatGPT, if you remember, was a low key research preview. For many years. Yeah, it wasn't a thing that the team launched thinking it was going to be this massive product.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“The way, I think there are like it's not that it's only chat either, if you have high volume use cases where they're more prescribed and you don't actually need the full generality, there are many use cases where it's better to have something that's less flexible, more prescribed, faster at a specific task. And those are great too. And, you know, you can build all sorts of those. But you still want chat as like this baseline for anything that falls out of whatever vertical you happen to be building for. It's like a catch-all for every possible thing you'd ever want to express to a model.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Of LLMs. So to me, it's like an interface that's exactly fit to the power of these things. And that doesn't mean that it always has to be just like, I don't necessarily always want to type, but you do want that very open-ended, flexible communication medium. It may be that we're speaking and the model speaking back to me, but you still want that very sort of lowest common denominator, no restrictions way of interacting.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I don't know if maybe this is one of those things I believe that most people don't believe. But I actually think chat is an amazing interface because it's so versatile. People tend to go, oh, chat, yeah, well, that's just like, you know, we'll figure out something better. And I kind of think this is incredibly universal because it is the way we talk. I can talk to you verbally like we're talking now. I can, you know, we can see each other and interact. We can talk on WhatsApp and be texting each other. But all of these things is this sort of like unstructured method of communication. And that's how we operate. If I had to rigid interface that I was allowed to use when we spoke, I would be able to speak to you about far fewer things and it would actually get in the way of us having maximum communication bandwidth. So there's something magical. And by the way, in the past, it never worked because models, there wasn't a model that was good at understanding all of the complexity and nuances of human speech. And that's the magic.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, we learned from that, actually. Because when we first launched it, we kind of gave you the subheadings of what the model was thinking about, but not much more. And then DeepSeek launched, and it was a lot. And we kind of went, you know, I don't know if everyone wants that. There's some novelty effect to seeing what the models really thinking about. We felt that too when we were looking at it internally. It's interesting to see the model's chain of thought. But it's not, you know, I think at the scale of like 400 million people, you don't want to see the model kind of like babble a bunch of things. And so what we ended up doing was summarizing it in interesting ways. So instead of just getting the subheadings, you're kind of getting like one or two sentences about how it's thinking about it. And you can learn from that. So we kind of tried to find a middle ground that we thought was an experience that would be meaningful for most people. But showing everybody like three paragraphs is probably not the right answer.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“That's all trying to attack the same problem, and then you have a model that's looking at all their outputs and integrating it and then giving you a single answer at the end. I mean, sounds a little bit like brainstorming, right? Like I certainly have better ideas when I get in a room and brainstorm with other people because they think differently than me. And so anyways, there's just like all these situations where you can actually kind of reason about it. Like a group of humans or an individual human, and it sort of works, which I don't know, maybe I shouldn't have been surprised, but I was.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“It's a long experience. It's a long time to wait, but it's not long enough to go do something else. And so you actually need, you know, so you can think, if you ask me something that I needed to think for 20 seconds to answer, what would I do? I wouldn't just like go mute and not say anything and kind of, you know, shut down for 20 seconds and then come back. So we shouldn't do that. We shouldn't just have a slider sitting there. That's annoying. But I also wouldn't just start babbling every single thought that I had. So we probably shouldn't just like expose the whole chain of thought as the model's thinking. But, you know, I might go like, huh, that's a good question. All right. I might approach it like that and then think, you know, you're sort of like maybe giving a little updates. And that's actually what we ended up shipping. You have similar things where you can find situations where you get better thinking sometimes out of a group of models.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it could be one of these two, but that means there's an A here. So that one has to be this. Backtrack kind of step by step build up from where you are. Same way you answer any difficult logistical problem, any scientific problem. So this reasoning breakthrough was big, but it was also the first time that a model needed to sit and think. And that's a weird paradigm for consumer product. You don't normally have something where you might need to hang out for 25 seconds after you ask a question. And so we were trying to figure out what's the UI for this because it's also not like with deep research where the model is going to go and think for 25 minutes sometimes, it's actually not that hard because you're not going to sit and watch it for 25 minutes. You're going to go do something else. You're going to go to another tab or go get lunch or whatever. And then you'll come back and it's done. When it's like 2025.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't know. Maybe I should have expected this, but one of the things that's been funny for me is the extent to which you can kind of reason when you're trying to figure out how some product should work with AI, you can often or even why some AI thing happens to be true. You can often reason about it the way you would reason about another human. And it kind of works. Maybe a couple examples. When we were first launching our reasoning model, right? We were the first to build a model that could reason, that could, instead of giving you just a quick system one answer right away to every question you asked, it was the third emperor of the Holy Roman Empire. Here's an answer. You could ask it hard questions and it would reason the same way that if I asked you to do a crossword puzzle, you couldn't just like snap fill in everything. You would be, well, okay, this one across.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“They can do deep research. They can operate on a canvas. They can write code. And so ChatGPT can kind of be this one-stop shop where all the things that you want to do are possible. And as we go forward in it, we have more agendic tools like Operator where it's browsing for you and doing things for you on the web. More and more, you're going to be able to come to this one place, to chat GPT, give it instructions, and have it accomplish real things for you in the world. There's just like something fundamentally valuable in that. And so we think a lot about that. We think, and we move, we try to move really fast so that we are always the most useful place for people to come to.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Think being first helps, which is one of the reasons why we're so focused on moving quickly. We like being the first to launch new capabilities, things like deep research. We've also, our models are very, they can do a lot of things, right? So they can take real-time video input. You have speech-to-speech. You can do speech-to-text and text to speech.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“It's possible than it is to forge a path through the jungle in doing something brand new. So I just think as an example, it was like nobody could break four minutes in the mile and then finally somebody did in the next year, 12 more people did it. I think there's that all over the place. And it just means that competition is really intense and consumers are going to win and developers are going to win and businesses are going to win in a big way from that. It's part of why the industry moves so fast. But all respect to the other big model providers, models are getting really good. We're going to move as fast as we can. And I think we've got some good stuff coming.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, kudos to Anthropic. They've built very good coding models, no doubt. We think that we can do the same. Maybe by the time this podcast is shipped, we'll have more to say. But either way, all credit to them. I think this intelligence is really multidimensional. And so I think there's the model providers, it used to be that OpenAI had this massive model lead, you know, 12 months or something ahead of everybody else. That's not true anymore. I like to think we still have a lead. I'd argue that we do, but it's certainly not a massive one. And that means that there are going to be different places where, you know, the Google models are really good or where Anthropics models are really good or where we're really good and our competitors are like, ah, we got to get better at that. And it actually is easier to get better at a certain thing once someone's”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“But we don't spend that much time building scaffolding around the parts that don't match that because our general mindset is in two months there's going to be a better model and it's going to blow away whatever the current set of limitations are. And so if you're building, and we say this to developers too, if you're building and the product that you're building is kind of right on the edge of the capabilities of the models, keep going because you're doing something right because you give it another couple months and the models are going to be great and suddenly the product that you have that just barely worked is really going to sing. And that's kind of how you make sure that you're really pushing the envelope and building new things.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Ends up being a part of our product philosophy is this sense of like model maximalism. The models are not perfect. They're going to make mistakes. You could spend a lot of time building all kinds of different scaffolding around them. And by the way, sometimes we do because sometimes there are things of errors that you just don't want to make.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“And I think it's more important to ship and iterate. So we have this philosophy that we call iterative deployment. And the idea is like we're all learning about these models together. So there's a real sense in which it's way better to ship something even when you don't know the full set of capabilities and iterate together, like in public. And we kind of co-evolve together with the rest of society as we learn about these things and where they're different and where they're good and bad and weird. I really like that philosophy. There's also a bit of, I think the other thing that”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“On key projects. So we do product reviews and things like that, like you would expect. There isn't a ritual because there isn't, I would never want us to be blocked on launching something, you know, waiting for a review with me or Sam if we can't get there, if I'm traveling or Sam's busy or whatever, that's a bad reason for us not to ship. Obviously, for the biggest, most high priority stuff, we have a pretty close bead on.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“It's absolutely atrocious. And we know it. And we'll get around to fixing it at some point, but it's not the most important thing. And so we don't spend a lot of time on it.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Right, or you know, we launched this thing, it didn't work, we'll roll it back. Look at our naming, our naming is horrible.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“So I think just expecting that you're going to be super agile and that there's no sense writing a three month roadmap, let alone a year-long roadmap, because those technology's changing underneath you so quickly, we really do try and go very strongly bottoms up, kind of subject to our overall directional alignment. We have great people. We have engineers and PMs and designers and researchers who are passionate about the products they're building and have strong opinions about them. And are also the ones building them. And so they have a real sense of what the capabilities are too, which is super important. And so I think you want to be more bottoms up in this way. And so we operate that way. We are happy making mistakes. We make mistakes all the time. It's one of the things I really appreciate about Sam. He pushes us really hard to move fast, but he also understands that with moving fast comes, we didn't quite get this.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Things partnership with research here. And so you want to have a second to kind of check your dependencies, make sure you're good to go, and then start executing. We try and keep that really lightweight because it's not going to be right. We're going to throw it out halfway because we will have learned new things. So the moment of planning is helpful, even if you're only going to, you know, it's only partially right.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, we try and have a sense of where we're trying to go, point ourselves in a direction so that we have some rough sense of alignment thematically. I don't for a second, and we do quarterly roadmapping. We laid out sort of a year-long strategy. I don't for a second believe that what we write down in these documents is what we're going to actually ship three months from now, let alone six or nine. But that's okay. I think it's like an Eisenhower quote, plans are useless. Planning is helpful, which I totally subscribe to, especially in this world. It's really valuable if you think about quarterly roadmapping, for example. It's really valuable to have a moment where you stop and go, okay, what did we do? What worked? What went well? What didn't go well? What did we learn? And now what do we think we're going to do next? And by the way, everybody has some dependencies. You need the infrastructure team to do the following.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“There are immense opportunities in every industry and every vertical in the world to go build AI based products that improve upon the state of the art. And there's just no way we could ever do that ourselves. We don't want to. We couldn't if we did want to. And we're really excited to power that for 3 million plus developers and way more in the future.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, one of my, so this is something that Ev Williams used to say back at Twitter that's always stuck with me, which is no matter how big your company gets, no matter how incredible the people are, there are way more smart people outside your walls than there are inside your walls. And that's why we are so focused on building a great API. We have 3 million developers using our API. No matter how ambitious we are, how big we grow. By the way, we don't want to grow super big. There are going to be, there are so many use cases, places in the world where AI can fundamentally make our lives better. We're not going to have the people. We're not going to have the know-how to build most of these things. And I think, like I was saying, the data is industry-specific, use case specific, behind certain company walls, things like that.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“And you're going to measure that with custom evals. And so what I was referring to is just like these models are really smart. You need to still teach them things if the data is not in their training set. And there's a huge amount of use cases that are not going to be in their training set because they're relevant to one industry or one company.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Still, most of the world's data, knowledge, process is not public. It's behind the walls of companies or governments or other things. And same way, if you were going to join a company, you would spend your first two weeks onboarding. You'd be learning the company specific processes. You'd get access to company-specific data. You can teach the models are smart enough. You can teach them anything, but they need to have the sort of the raw data to learn from. And so there's a sense in which I think the future is really going to be incredibly smart, broad-based models that are fine-tuned and tailored with company-specific or use case specific data so that they perform really well on company specific or use case specific things.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, these models, they're intelligences, and intelligence is so fundamentally multidimensional. So you can talk about a model being amazing at competitive coding, which Taking a whole bunch of code that's written in COBOL and turning it into Python. And that's just within the software engineering world. And so I think there's a sense in which you can think of these models as incredibly smart, very like factually aware intelligence is.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Okay, on a certain set of things, you can teach the model. You can make this a continuous learning process. And so as we were fine-tuning our model for deep research to be able to answer these things, we were able to test, is it getting better on these evals that we said were important measures of how the product was working? And it's when you start seeing that and you start seeing performance on evals going up, you start saying, okay, I think we have a product here.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Hours of reading on the web, and then you might need to read some papers, and then you would come back and start writing up your thoughts and realize you had some gaps in your thinking. So you go out and do more research. It might take you a week to write some 20-page answer to this question. You can let ChatGPT just chug for you for 25, 30 minutes. It's not the immediate answers you're used to, but it might go work for 25, 30 minutes and do work that would have taken you a week. So as we were building that product, we were designing evals sort of at the same time as we were thinking about how this product was going to work. And we were trying to go through like hero use cases. Here's a question you want to be able to ask. Here's an amazing answer for that question. And then turning those into evals. And then he'll climbing on those evals. So it's not just that the model is static and we hope it doesn't.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, it gets back to what I was saying. You need to know whether your model is going to. There are certain things that models will get right 99.95% of the time, and you can just be confident. There are things that they're going to be 95% right on and things that are going to be 60% right on. If the model's 60% right on something, you're going to need to build your product totally differently. And by the way, these things aren't static either. So a big part of evals is if you know you're building for some use case. So let's say let's take our deep research product, which is one of my favorite things that we've released maybe ever. The idea is with deep research for people who haven't used it, you can give ChatGPT now an arbitrarily complex query. It's not about returning you an answer from a search query, which we can also do. Here's a thing that if you were going to answer it yourself, you'd go off and do two.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Sure. I think the easiest way to think about it is almost like a quiz for a model, a test to gauge how much well it knows a certain set of subject material or how good it is at responding to a certain set of questions. So in the same way you take a calculus class and then you have calculus tests that see if you've learned what you're supposed to learn. You have evals that test how good is the model at creative writing? How good is the model at graduate level science? How good is the model at competitive coding? And so you have these set of evals that basically perform as benchmarks for how smart or capable the model is.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“If the model gets it right, 60% of the time, you build a very different product than if the model gets it right 95% of the time versus if the model gets it right 99.5% of the time. And so there's also something that you have to get really into the weeds on your use case and the evals and things like that in order to understand the right kind of product to build. So that is just fundamentally different. You know, if your database works once, it works every time. And that's not true in this world.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“LLMs are completely different than that, right? They're good at fuzzy, subtle inputs, all the nuances of human language and communication. They're pretty good at. And also, they don't really give you the same answer. You probably get spiritually the same answer for the same question, but it's certainly not the same set of words every time. And so you're much more, it's fuzzier inputs and fuzzier outputs. And when you're building products, it really matters whether there's some use case that you're trying to build around.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Everything we're used to with computers is about giving a computer very defined inputs. If you look at Instagram, for example, there are buttons that do specific things and you know what they do. And then when you give a computer defined inputs, you get very defined outputs. You're confident that if you do the same thing three times, you're going to get the same output three times.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it's probably the pace. I mean, maybe it's two things one is it's the pace. The second is, you know, everywhere I've ever worked before this. You kind of know what technology you're building on. So you spend your time thinking about what problems are you solving? Who are you building for? How are you going to make their lives better? How are you going to, is this a big enough problem that you're going to be able to change habits? Do people care about this problem being solved? All those good product things. But the stuff that you're building on is kind of fixed. You're talking about databases and things. And I bet the database you used this year is probably 5% better than the database you used two years ago. But that's not true at all with AI. It's like every two months computers can do something they've never been able to do before and you need to completely think differently about what you're doing. There's like something fundamentally interesting about that makes life fun here. There's also something, you know, we'll maybe like talk about Evals later, but it also really in this world of”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“And then it was like Monday, Tuesday, Wednesday. I still didn't hear anything. And I reached out to folks on the OpenAI side a couple of times. Still nothing. And I was like, oh my God, I screwed it up. Like, I don't know where I screwed it up, but I totally screwed it up. I can't believe it. And I was going back to Elizabeth, my wife, and being like, what did I do? Like, where do you think I, you know, getting all crazy about it? And then it's still nothing. And finally, it was like, it was like nine days later. They finally got back to me. And it turned out, you know, there was like a bunch of stuff happening internally and this, that, and the other thing. And, you know, there's just a million things happening. And they finally were like, oh, yeah, that went well. Let's do this. And I was like, oh, okay, cool. Let's do it. But it was like nine days of agony. And they were just super busy on some internal stuff. And there I was like fretting.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Going really well. We're really excited. And I say, cool. So, how do I think about tomorrow? And he said, Oh, you'll be fine. Don't worry about it. And if it goes well, we're basically there. And so I go in the next day, meet a bunch of people, have a great time. I really enjoyed everybody I met with. In any interview, you can always second guess yourself. Like, oh, I shouldn't have said that thing, or that thing I gave a bad answer on. I wish I could redo, but I came away feeling like, I think that went pretty well. And I was expecting to hear like that weekend basically because they'd sort of set expectations as soon as this goes well, we're ready to go. I didn't hear anything”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Again. And this time, it wasn't like, oh, you should go talk to these guys working on fusion. He said, actually, we're thinking about something. You should come talk to us. I was like, okay, that sounds amazing. Let's do it. And it goes really fast. Really, really fast. Like I met most of the management team in a brief period of time, a few days. And they were telling me, look, we're going to basically move as fast as we as we want to move. And kind of, if everyone, if you talk to everyone, everyone likes you, you're ready to go. Sam came over for dinner and we had a great evening together just like talking about open AI in the future and getting to know each other better. And at the end, I was like, I was going to go in the next day for like a bigger round of interviews. And Sam was saying, you know, hey, it's”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“If I'm remembering the timeline right, we communicated a planet. I was leaving, and I was planning to just go take some time. I wasn't going to stop working. But I was also happy to take the summer. This is like maybe April or something. I was like, cool, I'm going to have the summer with my kids. We're going to go to Tahoe or something and I'll actually get to hang out rather than what I usually do going up and down and all that. And then Sam and I had known each other lightly for a bunch of years. He's always involved in so many interesting things. You know, like companies building fusion and all these things. So he'd always been somebody that I would like call occasionally if I was starting to think about my next thing because I like working on big like tech forward sort of, you know, next wave kind of things. And so I called him and I think Vinod also helped put us in touch.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Just an expected part of your life from then on. And there is really something in the way that we all are adapting to AI that's kind of like that. These miraculous things happen and computers can do something they've never been able to do before. And it blows our mind collectively for like a week. And then we're like, oh yeah, like, oh, yeah, now it's just machine learning on its way to being an algorithm.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“I had the same experience when I first got into a Waymo, right? Your very first ride, at least my very first ride, my first 10 seconds in a Waymo, it starts driving and you're like, oh my God, watch out for that bike. You're holding on to whatever you can. And then five minutes in, you've calmed down and you realize that you're getting driven around the city without a driver and it's working. You're just like, oh my God, I am living in the future right now. And then like another 10 minutes, you're bored. You're doing email on your phone, answering Slack messages and suddenly this miracle of human invention.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, there's something like that with everything. If I showed you when GPT-3 launched, I wasn't at OpenAI then. I was just a user, but it was mind-blowing. And if I gave you GPT-3 now, I just plugged that into ChatGPT for you and you started using it, you'd be like, what is this thing? Like mess”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“There, there's this quote I love, which is AI is whatever hasn't been done yet. Because once it's been done, when it kind of works, then you call it machine learning. And once it's kind of ubiquitous and it's everywhere, then it's just an algorithm. So I've always loved that we call things AI when they still don't quite work. And then by the time it's like an AI algorithm that's recommending you follow, you know, oh, that's just an algorithm. But this new thing, like self-driving cars, that's it. I think to some degree we're always going to be there. And the next thing is always going to be AI. And the current thing that we use every day and is just a part of our lives, that's an algorithm.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Think it's just the style people love Model is really capable at emulating style or understanding what it's very good at instruction following. That's actually something that I think people, I'm starting to see people discover with it, but you can do very complex things. You can give it two images. One is your living room and the other is a whole bunch of photos or memorabilia or things you want. And you say like, Tell me how you would arrange these things, or you can say, I'd like you to show me what this will look like if you put this over here and this thing to the right of that”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, especially social things because. You have a very tight network as a company socially, so you know each other and your experts and your product, hopefully. And so there's some sense in which if you're doing something social and it's not taking off internally, you might question what you're doing.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“There have been a handful of times in my career when you're working on a product internally and the internal usage just explodes. This was true, by the way, when we were building stories at Instagram more than anything else in my career, we could feel it was going to work because we were all using it internally and we'd go away for a weekend before it launched. We were all using it and we'd come back after a weekend and we would know what was going on and be like, oh, hey, I saw you were at that camping trip. How was that? You were like, man, this thing really works. ImageGen was definitely one of those. So we've been playing with it for, I don't know, a couple months. And when it first went live internally to the company, there was kind of a little gallery where you could generate your own. You could also see what everyone else was generating. And it was just like nonstop buzz. So yeah, we had a sense that this was going to be a lot of fun for people to play with.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source
“Libra is probably the biggest disappointment of my career. It fundamentally disappoints me that this doesn't exist in the world today because the world would be a better place if we'd been able to ship that product. We tried to launch a new blockchain. It was a basket of currencies originally. It was integration into WhatsApp and Messenger. I would be able to send you 50 cents in WhatsApp for free. It should exist. To be honest, the current administration is super friendly to crypto. Facebook's reputation is in a very different place. Maybe they should go build it now.”
2025-04-10 · Lenny's Podcast · OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · IDENTIFIED FROM THE TRANSCRIPT · source