YouSaid · the spoken record

Simon Willison

lines on the record: 120
first: 2026-04-02
most recent: 2026-04-02
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“3.1 Gemini 3.1 just a few weeks ago they had a video which featured a Pelican riding a bicycle like animated and I'm like oh my god it's my pelican but I thought it's okay because the way my benchmark works is I've actually got a bunch of secret alternatives in my pocket because obviously what happens if the AI labs train them to draw really good pelicans riding bicycles and I'm like well then I'll get into an oslot on a moped and if the ocelot on the moped sucks but the pelicans are really good I can prove that they cheated on the benchmark and that would be amazing right that would be a great thing to be able to say hey look they cheated except that when Gemini 3.1 came out they did all of the other combinations they were like and here's a giraffe and a little tiny car and so I'm like wow they they they've beaten me they've beaten they're doing all of the animals and all of the modes of transport”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“All about the code generation. The other thing is they're generating SVG and it has comments in. So you can see little code comments that say things like making sure the pelican's legs are hitting the pedals and added a fish for whimsy. And that's really fun. The Chinese AI models, I love playing with the Chinese open weight models. Some of those have drawn quite good pelicans and they run on my laptop. So I have my laptop drawing these pictures of pelicans with these little comments about what it's trying to do.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“And the weirdest thing happened where there appears to be a very strong correlation between how good their drawing of a pelican riding a bicycle is and how good they are at everything else. And nobody can explain to me why that is. But as I started looking at these things, I realized, wow, the better models really do draw better pelicans riding a bicycle. Because the point now, it's a meme. The AI labs are all very aware of this and they relish in how good their pelicans riding a bicycle are. The other day, OpenAI released GPT 5.4 mini and nano at five different thinking levels that you could have them do low thinking, medium thinking, high thinking. So I did a grid of 15 pelicans riding bicycles for the three GPT 5.4 models across the things. And sure enough, GPT 5.4 running at X high did draw the best pelican.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“So, this one's fascinating. It was about a year and a half ago. I started benchmarking. So there were lots of benchmarks of these models and there were all these numeric things like it scored 72% on terminal bench or whatever. And I was always frustrated me because they don't really tell you anything interesting. Like if this one got 74 and this one got 72, does that actually mean that one of them's better at something than the other? And so basically to make fun of the benchmarks, I started my own benchmark, which was generate an SVG of a pelican riding a bicycle. And it's an SVG. This isn't a test of the image models. This is a test of the text models because they can all output SVG code. And if you ask them to draw you an SVG of something, they're almost universally terrible because they don't have good spatial reasoning and like drawing things by plotting out vectors is difficult anyway. So I started getting the models to render, generate an SVG of a Pelican on a bicycle because then you can look at them. You can say, here's one, here's one model, here's the other, which is best.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“So I use Claude for Culled for the code stuff. The other thing that I use a lot of is for research. And this is this thing where a couple of years ago, if you told me that you were replacing us with Google with ChatGPT, I'd assume that you just didn't understand how this technology works in its limitations because that was a terrible idea. Now that all of the major models have really good search integration, they're just better at searching than I am. I can ask them a question and watch them fire off five searches in parallel for aspects of answering that question, pull the data back. And if it's something I'm going to publish, I always double check, make sure it didn't hallucinate a detail because that would be embarrassing. But honestly, most of, I hardly use Google search directly at all. I'm always using it via, I'm doing searches via Claude or via Chatch GPT, or sometimes via the Gemini app, like that. That's a good option as well. And then I mean, for image generation, I'm using Gemini because of Nano Banana, but I only use that for fun.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“But the memory feature is that thing that all of the labs are trying to be more stitchy with. That said, when the whole, the Open AI military stuff happened a few weeks ago, Anthropic took advantage by saying, hey, why don't you move to Claude? And the way they did that is they had a Claude onboarding page that said, transfer your memories from ChatGPT by clicking this button and then pasting it into ChatGPT. And it was just a prompt. They had a prompt, which was, hey, ChatGPT, tell me everything that you've remembered about me. And so you paste that prompt into ChatGPT and it gives you all of the memories. And then you paste them into Claude. And I thought that was hilarious, like a whole export, like move from one to the other just by prompting it to give you the information you needed.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“The stickiest thing is meant to be memory, like all of the, they all have these features where they will remember things about you. And I hate those features and I turn them off wherever I can. Because mainly because as an AI researcher, I need to see what everyone else sees when I'm prompting. Like I don't want to say to the world, oh my goodness, look, this thing works now and it only works for me because it's based on previous conversations that I've had. And maybe I missed.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“It's almost matches my taste, but not quite. And maybe that's because I've just spent more time with Claude, so my prompting style has evolved more to fit the Claude way of thinking. I don't know. This's also weird. It's vibes all the way down.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Other now. They're both very, very good pieces of software. And I kind of expect this to happen. Like the next Gemini model comes out might become the best coding model for a couple of months, in which case I might switch myself into that ecosystem, partly because I write about this stuff as well. I like to stay familiar with as many of the offerings as possible. But I keep on coming back to Claude Code, mainly because it fits my taste. Like there's this weird thing where I've got a very specific taste in how I like code to work, which coincidentally happens to map to how Claude Code likes to work, which is kind of interesting. And GPT 5.4.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“And my code is all open source, so I don't care. That's a useful trick there. But, yeah, so I use that on my phone. I often have two or three of those running. A lot of my major projects are done mostly prompting on my phone. If it's security adjacent or super important, I might pull it down to my laptop to do a thorough review later on. But most of the review you can do through GitHub, like these things will file pull requests, and then you use the same tools you'd use to review code from other people to review the code from the agents. That said, OpenAI came out with GPT 5.4 about three weeks ago. It's very, very, very good. I think it's on pile with Claude Opus 4.6 and possibly even better. These companies are constantly leapfrogging each other. So I have been leaning back. It's also cheaper. So I've been leaning on GPT 5.4 a lot more this month and open AI codecs. And OpenAI Codex and Cloud Code are almost indistinguishable from each.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Run these things in the YOLO mode. This is Claude calls it dangerously skip permissions. OpenAI actually do call it YOLO. They've got an option for that. And that's the mode where the agent doesn't ask you if it should do something all the time. And that is a different product. I think a lot of people who haven't got on board with coding agents yet haven't tried them in the unsafe mode. They're using coding agent where it's like, oh, can I run this piece of code? Can I edit this file? That means you have to pay complete attention to it the whole time. And it's like working with a really frustrating toddler that's constantly nagging you about what it wants to do. The moment you take the safeties off, now I can run four of them and go and have like a cup of tea and come back and they've achieved something useful for me. But it's inherently unsafe. If it's running in Claude Code for Web, the only bad thing that could happen is maybe it accidentally leaks your private source code.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“So, right now I'm mostly Claude. I do a huge amount of work using Claude. Well, I'm mainly still a Claude code person, but there are two sides of Claude Code that I use. There's the Claude code that runs on your computer. And then there's Claude Code for web, which is their hosted version of Claude Code. And I use that one more than the one on my own computer, partly because that's the one you can access through your phone. If you've got the Anthropic Claude app installed on iPhone, there's a code tab and you can go in there and you can tell it to write you things. And that is running on their servers. You need to give it a GitHub repository of yours that it can work within. But it's also great from a security point of view because if you're running Claude Code on your laptop, there's risks that bad things can happen. It might accidentally delete things. If I'm running on an anthropic server, I couldn't care less. Like it's their computer. It's not my computer go wild. So this means that you can.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Prototyping is almost free, I think. And that really impacts me because throughout my entire career, my superpower has been prototyping. Like I am, I've been very quick at knocking out working prototypes of things. I'm the person who can show up at a meeting and say, look, here's how it could work. And that's kind of my unique selling point. And that's gone. Anyone can do what I could do. It's like, but you still have to learn when it's appropriate to prototype, how to think about prototyping, how to get the tools to build useful prototypes that you can use to explore things.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Can churn out 10,000 lines of code now in the time that it would take me to write 100. How do I make that code good, right? How do I make sure that I'm not just turning out total slop that adds up to technical debt that slows me down? How do I take the fact that code is now cheap and use that to produce better code? Because I don't just want cheap code. I want really good code that does what I need to do, like I can extend in the future that's got all of those characteristics of code that's useful and can be used in production.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“That vap's changed completely. My programming work, I need two minutes every now and then to prompt my agent about what to do next. And then I can do the other stuff and I can go back. I'm much more interruptible than I used to be. But yeah, so the thing that used to take the time is now the thing that takes way, way less time. What does that mean for everything else that we do? And that doesn't just affect programmers, it affects entire like teams of around software development, but as an individual programmer, you have to start thinking, okay.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I think this is the single biggest shock in all of this. The reason that we have to rethink how we build, how we work as software engineers, is that the thing that used to take the time takes way less time. Like it's never been the case that programmers spend 90% of their typing code into a computer. There's always so much additional work around that. But it still used to be like people talk about how important it is not to interrupt your coders, right? Your coders need to have solid two to four hour blocks of uninterrupted work so they can.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“But the question there is always how much of that is AI and how much of it is overhiring during COVID and recorrections and all that kind of thing. It's always very difficult to tell. So that the number of open jobs on the one hand, maybe that's a better signal, but on the other hand, the recruitment market has been driven completely crazy by all of this stuff, right? Like all of the job ads are written by AI. The resume is the AI. People in recruitment are saying that this is never been this hard to filter through and hire people and people who are hiring jobs say they applied to 200 things and got nobody hearing back. So it's hard, right? The macroeconomic indicators for this stuff are lagging. And at some point, we should start getting more confident numbers about what the impact actually is.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm not an AI doomer in the slightest. The economics of it do make me nervous. Are we really going to wipe out a tenth of white collar knowledge work jobs in the next few years? I really hope not because I don't know how the economy adapts that. So yeah, that's.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“For my definition of good code, at least. So we're saying 50% of engineers, let's say 50% of engineers, majority of their code, it could happen by the end of this year. It could because the technology is good enough now. And I feel like the challenge now is getting people to learn how to use this stuff, which is difficult because using the stuff everyone's like, oh, it must be easy. It's just a chatbot. It's not easy. Like that's one of the great misconceptions in AI is that using these tools effectively is easy. It takes a lot of practice and it takes a lot of trying things that didn't work and trying things that did work. But yeah, I expect by the end of this year it will not be uncommon to have an engineer say that almost all of that code is written by AI.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“You'll get the Europeans in a lot more AI skeptic than the Americans are generally. So I think different countries are going to have different sort of different cultures around this. At the same time, I think it's become undeniable this year that this stuff produces good code. Like it used to be that you could say, I don't use this stuff because the code is bad. And that was a justifiable position. That's not justifiable anymore. The code is now good. It's good code for.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I'm going to refactor that to 95% of the code. I don't think. Get to it, but It's very difficult to say worldwide because partly because there are cultural differences. I spend way too much time on hacker news. And something I've noticed about hacker news is a conversation that starts at midnight specific time and goes until 8 a.m. Very different tone because it's the Europeans.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“That's fascinating. That's the pre World War II, the metal that you can dig up from old shipwrecks, which is before the first nuclear explosions. And so it's not got the radiation baked into the metal. It's that whole thing.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“And I can stick it up on GitHub and everything. And yet, I don't believe in it. And the reason I don't believe in it is that I got to rush through all of those things. I think the quality is probably good, but I haven't spent enough time with it to feel confident in that quality. Most importantly, I haven't used it yet. Like it turns out when I'm using somebody else's software, the thing I care most about is I want them to have used it for months. I want other people to have put that software into practice. So I've got some very cool software that I built that I've never used. Like it was so quicker to build it than to actually try and use it. And so the way I've been dealing with that is always put alpha on it. Like if you see my software and it says it's an alpha, that probably means I haven't actually used it yet for most of my project, which is a bit of a cheat code, you know, alpha this. But isn't that interesting? Like it used to be if you looked at software and it had high quality tests and documentation and everything, it meant it was good. And now that's...”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Fill out the word artisanal does a handcrafted software, I think is going to be valued more. Something I've noticed in my own work is sometimes I have an idea through a piece of software Python library or whatever, and I can knock it out in like an hour and get to a point where it's got documentation.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“A lot of my friends have been talking about how they have this backlog of side projects, right? For the past 10, 15 years, they've got projects they never quite finished and ideas they thought would be cool. And some of them are like, well, I've done them all now. Like last couple of months, I just went through and every evening, I'm like, let's take that project and finish it. And that one and that one and that one and that one nut. They almost feel a sort of sense of loss at the end where they're like, well, okay, my backlog's gone. Now, what am I going to build?”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“And I think that's the concern there comes down. It's always expectations from other people. If you work for a company that's expecting you to get five times more done, that's going to be exhausting. And maybe I think the good companies with good management are paying attention to this and that they don't want to burn out their best employees for the sort of short-term gain but lose people over it. But yeah, it's a big tension. I think we're those of us on the sort of leading edge of the AI boom are feeling it first. I imagine it's going to come for everyone else as well.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Your brain is exhausted. My brain is exhausted. I've got more time to go and do things and I do things, and it's great, but it is that the exhaustion from that sort of intensity of work has been a really big surprise for me. Like that, that's been something which I've been observing, especially since November, like as all of this stuff started ramping up.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“I think I really hope it's a novelty thing. And I am actually getting much more, I'm getting more time, but I'm exhausted.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“It's fun. I'm enjoying myself. I think I'll probably get to the end of the year and I'll be like, wow Think the most important things I should have been focusing on did not get done, but that's the case when it is my ambition to do them. So, you know.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“My New Year's resolution this year was the opposite. Every previous year, I've always told myself this year I'm going to focus more. I'm going to take on less things. This year, my ambition was take on more stuff and be more ambitious. Like we've got these tools. Bring it all in. Let's try and do everything. I don't know if that was a good New Year's resolution, but that's what I went with.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Human beings have agency, and we use that agency to decide what problems to take on and where to go. I think agents have no agency at all. Like I would argue that the one thing AI can never have is agency because it doesn't have human motivations. Like, sure, you can tell it, make more money or whatever, but it's never going to be able to decide on what makes sense for it to act on next. So I'd say that's the thing is to invest in your own agency and invest in how do I use this technology to get better at what I do and to do new things.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“You know? And previously, the fact that it would have taken me like two or three months to learn basic Apple script was enough for me never to use it. And now I've got all of these technologies that I'm using because that two to three month initial learning curve has been shaved right down. I think that applies to everything else. Like I'm getting much better at cooking. I've been using Claude, it turns out, excellent chef, which doesn't make sense because it doesn't have taste buds, but it does it can give you the global average of the world's guacamole recipes, which turns out is good guacamole. So that's been really interesting, like trying to apply this stuff just for sort of self-improvement. I think that's a really useful skill to have. Because honestly, everything is changing so fast right now. The only universal skill is being able to roll with the changes. That's the thing that we all need. Weirdly, the term that comes up most in these conversations about how you can be great with AI is agency.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“That's a big responsibility you're putting on me there. I think Think the way forward is to lean into this stuff and figure out how do I help this make me better. A lot of people worry about skill atrophy. If the AI is doing it for you, you're not learning anything. I think if you're worried about that, you push back at it. Like you have to be mindful about how you're applying the technology and think, okay, I've been given this thing that can answer any question and often gets it right doesn't always get it right. How can I use this to amplify my own skills, to learn new things, to take on much more ambitious projects, something I've been enjoying, I think the thing I've enjoyed most about this as a software engineer is that my level of ambition has shot right up because now I used to like never, I never used Apple script because Apple script is a whole programming language you have to learn and I've been using AppleScript for like two and a half years now because ChatGPT knows AppleScript and I don't have to and so now I can automate things on my Mac and that's great.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“That's the group which ThoughtWorks Resolved worked probably in the most trouble right now. And it's not as beneficial, like they've got all of the boosts that the beginners were getting they've got already. So that's an interesting open question right now for me. It's more the sort of mid-level as opposed to the beginners or the advanced people”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“So there's an interesting. So, ThoughtWorks, the big IT consultancy did a off-site about a month ago, and they produced, they got a whole bunch of engineering VPs in from different companies to talk about this stuff. And one of the interesting theories they came up with is they think this stuff is really good for experienced engineers. Like it amplifies their skills, that's great. It's really good for new engineers because it solves so many of those onboarding problems. If you talk to Cloudflare and Shopify, both said they were hiring a thousand interns over the course of 2025 because the intern onboarding costs, it used to be, takes a month before you intern can do anything useful. Now they're doing something useful within like a week because the AI assistant helps them get up and running faster. The problem is the people in the middle. Like if your mid-career, if you haven't made it to sort of super senior engineer yet, but you're not sort of new either, that's...”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“You can be the first person in the world to spot that AI can now do acts just because you were the person you found it couldn't do it and you've been keeping that sort of backlog of interesting tasks for it.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Problem, and I can say this problem is a one sentence prompt, and I know it'll find that bug and fix that bug, as opposed to this other problem, which is who knows how big a problem. There is a flip side to this, which is that I've got 25 years of experience in how long it takes to build something. And that's all completely gone. Like that doesn't work anymore because I can look at a problem and say, okay, well, this is going to take two weeks. It's not worth it. And now it's like, yeah, but maybe it's going to take 20 minutes because the reason it would have taken two weeks was all of the sort of crafty coding things that the AI is now covering for us. And that I've been finding really interesting and challenging. I constantly throw tasks at AI that I don't think it'll be able to do because every now and then it does it. And when it doesn't do it, you learn, right? You learn, okay, opus 4.6 still can't do this particular thing. But when it does do something, especially something the previous models couldn't do, that's actually cutting edge AI research.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Losing sleep because they're like, My coding agents could, my agents could be doing work for me. I'm just going to stay up an extra half hour and set off a bunch of extra things. And they're waking up at four in the morning. That's obviously unsustainable. I hope that that's a novelty thing that agents only really got good in the past sort of four to five months. We're all learning what that looks like and what that lets us do. But it's concerning. There's an element of sort of gambling and addiction to how we're using some of these tools. But to stand in defensive software engineers, I get great results out of these things because they are amplifiers of existing skills and experience. And I have 25 years of existing like pre-AI experience, which I can now amplify because I can talk to the agent at a very high level. I can use very, I can use sophisticated engineering language that I've mastered over the years, which they appear to know as well. And we can collaborate incredibly effectively. That means I can look at a”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“And actually, I want to stand in defense of software engineers for a bit because on the one hand, these things can write code. That used to be our thing, right? I'm finding that using coding agents well is taking every inch of my 25 years of experience as a software engineer. And it is mentally exhausting. Like this is something which people are talking a lot more about now. I can fire up like four agents in parallel and have them work on four different problems and buy like 11 a.m. I am wiped out for the day. Like I have, because there is a limit on human cognition in how much, even if you're not reviewing everything you're doing, just how much you can hold in your head at one time. And it's very easy to pop that stack at the moment. Like there's a sort of personal skill that we have to learn, which is finding our new limits. Like what is a responsible way for us to not burn out and for us to use the time that we have? And I've talked to a lot of people who”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Tricks like this, like you can tell AI to combine weird fields. You can say, okay, I want ideas for marketing my new SaaS platform inspired by marine biology. And you see what happens. And most of it will be complete junk, but there might be a spark that gets you to the good idea. So I love them as brainstorming companions on that front.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, that's been the case for probably a couple of years now. They've been strong enough to do really good brainstorming. And I like to compare it to the thing where when you've got a group brainstorming exercise, you book a meeting room for an hour, you've got a whiteboard, you get a dozen people in, and the first two-thirds of that brainstorming session, honestly, it's kind of just everyone going through the most obvious basic ideas, right? And you get them all out on the whiteboard, you get them all up, and then things get interesting when you start saying, okay, well, let's talk about these, let's start combining them. The AI is so good at that first two-thirds of the ideas. Like, I brainstorm with them all the time. I just get them to spit out all of the obvious stuff and they'll come up with 20 things and they'll all be kind of done. They just won't be very interesting. What gets interesting is when if you ask them for 20 more and now by the sort of end of that list, you're beginning to get things which are not good ideas, but they point you in interesting directions. There are so many other.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“But then, what do you do? How do you, given your three options now that you have instead of one option, how do you prove to yourself which one of those is the best? I don't have a confident answer to that. I expect this is where the good old-fashioned usability testing comes in. Like get somebody on Zoom screen shared, using your software, see what happens. You can tell the AI to do it and you can simulate your users with the AI. I don't think that's credible. I don't think you're going to get as good results from ChatGPT pretending to click around on your prototype than you would from an actual human being.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“We can test things so much faster now because we can build workable prototypes so much quicker. So there's an interesting thing. I've been doing in my own work where any sort of feature that I want to design, I'll often prototype three different ways it could work because that takes very little time. And then I can start experimenting and trying them and seeing which ones I like. And that feels to me like the really transformational step here is that when you get AI involved in your ideation phase, it's much more about the prototypes. It's about, okay, we can see a UI prototype is free now. ChatGPT and Claude will just build you a very convincing UI for anything. And that's how you should be working. I think anyone who's doing product design isn't vibe coding little prototypes is missing out on the latest, like the most powerful sort of boost.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“So, this is one of the most interesting problems we're having with all of this we've taken the writing code bit and we've massively accelerated that. Now the bottlenecks are everywhere else, right? Like how do we redesign our processes now that the bit that used to take the longest? It used to be you'd come up with a spec and you hand it to your engineering team and three weeks later, if you're lucky, they'd come back with an implementation for you to then stop. And now maybe that takes three hours, depending on how well established the coding agents are for that kind of thing. So now what, right? Now where else are the bottlenecks? I don't think it's, I mean, as coming with the initial ideas, anyone who's done any product work knows that your initial ideas are always wrong. What matters is proving them, right? It's testing them.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“One as well, because we're seeing a lot of this in the wild, and it's just incredibly frustrating for maintainers because there are these people who don't know what they're doing, who are asking ChatGPT to find a security hole and then reporting it to the maintainer. And the report looks good. Like ChatGPT can produce a very well-formatted report of the vulnerability. It's a total waste of time. Like it's not actually verified as being a real problem. The difference with Anthropic and Firefox is that Anthropic security team actually did do the work. They didn't report whatever the agent said. They actually verified that it was a good quality report before they handed it over.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“And this is a new thing. I think in the past, again, in the past sort of three to six months, they've started being credible as security researchers, which is sending shockwaves through the security research industry. They were like, wow, we didn't think that they'd get to this point. What's interesting there is both OpenAI and Anthropic have specialist security models that they will not release to the general public because they can be used to break into websites. So they have like invite only like registered security researchers can apply for access and they've been producing vulnerability reports against popular open source software. I think Firefox just a few days ago, maybe last week, said that they'd done a release, which was assisted by Anthropic. Anthropic had discovered 100 potential vulnerabilities in Firefox and responsibly reported them to Mozilla, who then fixed them. That's an interesting.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“You'd have trouble getting it to run 24 hours a day. I mean, maybe it would work. But certainly I think that what's interesting to me isn't so much the software you're using. It is these big IEDs, these techniques that you're using to try and answer these questions. Because even if your QA team, your virtual QA team says this is good doesn't mean it's secure, right? It doesn't mean that you've got all of those other characteristics you care about. At the same time, the agents are getting really good at security penetration testing now.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“I think they can, like you can tell Claude Code, fire up a sub agent that uses Playwright to simulate a browser and all of that kind of thing.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“And all of these different systems that they could then build their software against, which cost them nothing because once they've spun it up, it was a little go binary that sat there. And they even had interfaces. They had like a fake version of the Slack interface that they'd like vibe coded up that let them see what was going on. Absolutely fascinating.”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source
“Example of thinking outside of the box taking this question, how do we tell our software is good if we're not reviewing the code and trying to find creative answers to it? The other thing was interesting is that the Slack channel itself wasn't actually Slack because it turns out if you test against real software like Slack and so forth, they all have rate limits. And they won't let you just run 10,000 simulated people at a time. So what they did is they built their own simulation of Slack and Jira and Okta and all of this software they were integrating with. And the way they did that is they basically took the API documentation for the public APIs for Slack and the client libraries that the open source client libraries and they told their coding agents build this build me a simulation of this API and they did. So this company is, and this is one of the things that I went to a demo that they gave back in October, one of the things that really sat with me is that they had their own simulated version of Slack and”
2026-04-02 · Lenny's Podcast · An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon Willison · IDENTIFIED FROM THE TRANSCRIPT · source