YouSaid · the spoken record

Nathan Banesh

lines on the record: 40
first: 2024-12-05
most recent: 2024-12-05
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Hard to say if it's because of AI because we did go through this growth spurt probably more than the spurt during 2021 where teams just became enormous. So in the overhang of that and the reductions in force that were conducted subsequent to the pandemic, entrepreneurs already at that point were saying I don't want to have to deal with big teams. I much prefer to have smaller teams, not too many offices, more senior people. And now with generative AI and technically speaking, you could probably do more with less. So it's a bit hard to disconnect those two. But I don't think people are hiring more because they have AI tools.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“principle the dogma is you'll need less people to do the same job i think that's probably true i do still think that there are Still want to have a third party do them even if you could technically audit yourself as a company with O1 because it was better than some top four auditor like no one who actually asks you for an audit is going to take that seriously is they're still going to want you to buy from a top four auditor so I think if we can disconnect the kind of pieces of work that companies and people purchase because somebody else did it and put their stamp of approval on it from like pieces of work that they can totally do themselves and this is acceptable then I think like agents will do like far better on onto secondary option versus the first unless like the auditors like use it themselves and then hopefully increase their margin and turn around their product faster.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Guess an interesting thought exercise there is if you're a travel AI agent, what do you charge someone for successfully creating and booking a trip? Right? You probably would look at what a human would cost to do that and at least it or undercut it. But like you said, the cost of these reasoning models is very high. People can experience this with open AI's 01 in ChatGPT, which is their reasoning model. It takes a lot of time to run and think. And that's, like you said, expensive, that's all being powered by NVIDIA. Even though you may be trying to undercut what human labor would cost your costs or run a product like this may be going up drastically. So what does that do to the companies and the balance sheets of these companies?”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Something you really had to think about that couldn't be encoded as an equation or something deterministic. If you do follow this idea that complex reasoning would require more thinking or more thinking time, however much thinking time there is is just a coin operated purchasing system on NVIDIA GPUs or other compute systems. So the more complex reasoning, the more time spent, the more compute cost. And then I think it's a question of, hey, is whatever task you're trying to use an agent for is that some total of the seconds and minutes it spends thinking and completing a task with the success ratio and failure ratio cheaper than what a human can do? Or are there like interesting handoffs that you can have between an agent and a team of humans depending on the complexity of the task? Perhaps like a task might involve some like real world manipulation, like the hopefully the agent can't jump out of the computer and go to something.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“We spent the last 10 years doing robotic process automation and task mining. Companies like UiPath, Solowness are probably the biggest names here. They made billions of dollars going to enterprises, diagnosing the various click streams and software tools that individuals would work on and finance HR, et cetera, and trying to figure out which ones of those processes showed the most repeatability, like the cow path. They would write scripts to automate those processes. And so now the promise with this sort of software agent, general purpose agent idea, is to do much of the same thing, but give that capability to everybody and have software systems that can automate processes without them having to be so codified where there is maybe a bit of nuance, a bit of reasoning nuance with these kind of RPA agents could not reason if there was some ambiguity or a change in the data structure or”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“When I talked to Nathan about this, he was confident that the rise of AI agents will translate to company spending much more on AI. As you'll hear us discuss, it's going to take considerable time to work out the cost structure here, though, as running an AI agent will need to eventually be cheaper than the cost of a human doing the same task.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Oh, it's almost assuredly happening already, particularly in customer support issues right now. I mean, you are more than likely, if you're on a website trying to return a pair of shoes that you bought from some site online, talking to software that was constructed as an agent, giving it the order number, you know, the address, they're giving you the shipping label to print out. Can guarantee that most of your audience has already interacted with agents, whether they realize it or not.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“On the enterprise use of agents, I guess just to stay on this for a second, it seems profound, but at the same time, like when do you think customers will start to interact with agents, not just like a company using it for its code generation?”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“I'm not going to go so far as to say that. Again, I think of it more as how software was built more than anything else, as opposed to replacing people or specific tasks. It's more about going back to that security example that I mentioned a moment ago, having agents that can go off in parallel execute a series of tasks that a human would have to do serially, right? Like that kind of, that's where the throughput acceleration really comes from is the automation, but the parallelization of the efforts as well.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“From prompt engineering, and almost, I think, more than half that, I want to say, and went into agents. So really what it is, is a mindset shift in terms of how people are building software more than anything else.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Well, I think it's important to understand what an agent really is, not to sound overly reductive or anything, but what it really is an orchestrated series of steps to execute. We talk about it a lot. Really what that is is an architectural shift more than some kind of revolution, I would say, necessarily that's going to change the way software is written or replace humans. Yeah, we talk about it a lot, but that's because we spend a lot of time meeting new companies or pitching to us and even our existing companies and disarticulating their architectures and really understanding how they're built and what we're seeing is that I talked about in the report a little bit some of the rag architectures that were pretty extensive in 2023 have been replaced by agentic style architectures and it's in the report as well you can actually see how the primary architectural approach and application construction has changed in 2024. A lot of it went away.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, you all are very bullish on agents in the report. And I feel like it's the topic de jour. It's the thing I hear constantly, agents, agents to agents. I don't think people have seen a lot of them practically in the world yet, but anthropic is an early mover here through their API. You can already do some kind of basic agent-like stuff on your desktop. What are agents going to do, especially as it relates to like the enterprise? I feel like we'll see some big consumer swings from Google and OpenAI in the coming months. When they get deployed into companies, what happens?”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“The software is just getting quite good, and they move to agentic workloads is getting quite strong. And the capabilities of what an agent can do, again, I'm not going to say that it replaces what a human being can do. And frankly, people should not be writing agentic workloads and architecting them in the way that a human does. It turns out that that doesn't really work well, is my understanding. But when you do deploy these things, they're quite productive and it certainly accelerates workload, like I said.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Investors like Tim and Nathan see a major opportunity in AI agents. Right now, companies are mostly paying for access to LLMs attached to a text box. You type in a question or a command, maybe you feed it some data or code to work with, and it spits something out in return. But what if these LLMs could go off on their own and complete tasks for you? That's the dream being pursued by Anthropic, OpenAI, and scores of their competitors. So far, though, only anthropic has publicly put out something trending in that direction. With the launch in October of a beta version of its clawed model that can move a digital mouse cursor, click buttons, and type text all on its own. It's too early to tell right now, but companies might pay a whole lot more for these kinds of agents than they do for the AI models of today.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Extremely bullish on the idea that the cost to train models over time is going to go down, right? There'll be better silicon. There'll be new ways to train models. There'll be optimizations that I'm probably not even aware of that are being cooked up right now and being produced in literature that will come out soon. So I'm confident that we'll find a path to have cheaper inference and training over time. On the application layer, I think it's already there. Most of these folks are just calling in using API calls into the foundation models anyways. And so the cogs and OPEX for them sort of to produce these software solutions is pretty small. We need to take another quick break. We'll be right back.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“The foundation model there and the application layers, not for every single company, obviously, but for certain ones are doing quite well in they're selling a lot of software and seeing a lot of usage and obviously a lot of API usage. They're generating real revenue. I mean, this is not a sort of blip in time or a blip on the radar where we're going to get some false positive results. Like this is real. And you can see it in the board meetings that I attend or my partners attend in the results of their companies are returning. So this is very real. And I think you'll see it from both layers. And everyone broadly knows that companies like OpenAI Anthropic are, yes, they're making money. They're growing their revenues, but they're also burning way more than they're bringing in. And maybe that will invert at some point. It's really hard to put a pin on when that's going to happen. I feel like...”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“I asked him about this too, especially considering Menlo Ventures invested in at least two separate financing rounds for Anthropic over the last several years. He sees the ROI on AI arriving alongside more superior chips and models. Even if it's difficult to see exactly how it will all play out. When you think about the return on investment that affirm like yours needs to have, where do you think it's going to come from? Do you think it's going to come from the models or from the applications that are built on top of the models or something else? I think it's going to come from both for sure. I mean,”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Just like the nature of what drives these kinds of individuals. They want to be there when the AGI happens, whatever that means. Like technology history teaches you that you can sustain lack of profits for a very, very long time as long as you give investors in the market other reasons to believe why they should suspend that disbelief for another year or another year, another year. And for the moment, I think that's still the case.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“And the CEOs driving this, do you think they see a pot of gold at the end of the rainbow that we all can't? Like they see, oh, you know, I get to this point and the profit is going to be just so immense that we'll have all been worth it. Or is this more FOMO driven and hype driven and kind of wisdom of the crowd driven?”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I think the revenue key is still growing. OpenAI is still really growing, anthropic according to reports is like eating into enterprise quite well. I mean, NVIDIA had another blowout quarter. I mean, the fact that it does or doesn't meet Wall Street estimates is kind of like neither here nor there, I think, because Wall Street has a tough time estimating that company anyway. And then a big cloud is still printing money on these services. So I think it's still real for sure. And we're going to see way more of this sovereign investment in AI. Every nation state is increasingly getting convinced that they need to be investing in this. I think big finance hasn't really come to play very much yet with regards to financing new data centers and really providing the capital for this kind of scaling. So I think it's still pretty early.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“You're of the. So we're talking about this is the scaling wall conversation that's happening. Where everyone is having questions about can we keep just plowing more money into these new models and continue to get better results? And you are of the there is no wall camp?”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“There's early signs that you see like Klarna say they've let go of hundreds of customer support roles because of AI. There's signs that's starting to happen. But yeah, when Klarna cuts hundreds of jobs, that doesn't make open AI more profitable. There's no direct correlation. Maybe it's indirect, but you're suggesting that Klarna example needs to happen a lot for the margin profiles of these model providers to really change.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Probably say each of those companies, if you assume that the race for bigger and bigger models continues. If we stop now, I think we could have fairly profitable companies. And if there were more price discrimination with the services. Companies sell one license to somebody in a hedge fund for the same price that they sell a license to a student doing their master's degree. So that, you know, that doesn't really make a ton of sense. So if there's better price discrimination, maybe some more vertical products. And then we stop on this race of making ginormous systems that require a huge compute, then I think there's a path. If we don't, then you're going to need like hundreds of aliens based on the compute spend.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Very profitable Have you thought about what that scenario would be, how they get there? This is tough. Think you really need to see, like. Hundreds of billions of dollars of revenue, I think, for that to be true.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“To some degree, it's because the people contributing to progress in AI were pretty much AI research people, not people who worked for decades on cloud infrastructure and optimizing this software runs really fast and is cheap on all sorts of devices. And all those people have started to move en masse into AI. And so I think you can bet just by this year capability of individuals who have done optimizations in other software settings and other technical settings moving into AI will actually bring a lot of cost improvement, which will eventually like help margins. So that's one of the reasons why I think you see an order or two orders of magnitude reduction in cost per million tokens, which is the unit economics of these models from one or two years ago to models that exist today for systems that are of similar intelligence, if you will. Do you think there's going to be a moment where OpenAI, which famously loses billions of dollars a year, anthropic, et cetera, these big leading players, the model providers?”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Well, The big model companies, which are not profitable and by far, those are kind of like the have-nots with regards to profit margins. And then the haves would be companies that probably provide much cheaper outputs, like where that's images, mid journey being famously quite profitable business, audio generation is much cheaper than it is to run like massive models because the models are smaller and the data is easier to work with. These companies are pretty profitable. In some ways, you're just in the early days of this technology, so you're just trading off profit margins and profitability for growth. And I think that makes sense for so long as the market's willing to provide you those tools. I will say, though, that a year or two years ago, the path to profitability was a lot fuzzier than what it is today, in part because there weren't that many optimizations that came into how the models were built, how they're run, and all the data infrastructure and cloud infrastructure around it.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“We think about it a lot. I mean, we have some investments in companies that are doing diffusion type models as well, not just in the video and image side, but also on the biotech side as well. So, you know, we're definitely investing, but I think you hit the nail on the head, right? The compute cost on those is a bit higher. And so it's a little bit more challenging to have some of the success that maybe the text to text folks have had.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“I think photo and video generation has been largely held back from being at scale by compute bottlenecks. And we've seen that with OpenAI and Sora and the fact that they still haven't released it. And I'm curious how much you think about the multimodal aspect of AI and how much that will affect the enterprise when that gets scaled out.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Definitely legal tech is growing very, very fast. Customer support as well. And really what that is, is the early movers who deal primarily in text, right? Because the foundation models do text to text translations effectively is what they're doing, right? And so what you saw is legal fintech digital marketing. These are the ones that were the early adopters of these things. And so because of that, obviously their revenue profiles are going to be a little bit higher because they were at the gates earlier. So it's really those. But I think growing very, very quickly is obviously anyone who's doing code generation as well.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“I think everyone's benefiting, frankly. I mean, across the board, I mean, you're seeing every department, every vertical sort of thriving, to be honest, on what they're doing with the AI. It's pretty breathtaking, to be honest. So it's not just, you know, obviously the coding guys are doing quite well. It's also transcends into other departments also, right? You see legal tech companies doing quite well.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“The sectors of startups that are already doing really well, making a lot of money kind of the same as where big enterprises are spending? Does it map directly? So like coding or are the coding startups that are supplying that tech the ones really benefiting right now at the earliest stages?”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“We didn't ask questions like, why did you switch necessarily? But what I hear anecdotally is that people are using anthropic heavily for code generation. It's that use case that we talked about earlier. You talk to folks and I talk to the same folks that you probably do and you ask what model you're using and why and you hear that there's certain use cases that people prefer clawed for, right? Code generation, some content generation as well. Structured data tends to do really well with. So I think people just are starting to prefer the output that they get from cloud and they see it as a better model for certain sets of use cases. And that's why you saw it double this year.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“And on the foundational model layer, you have an interesting point here, which is that open AI, you say seeded market share and enterprise AI usage declining from 50% to 34%, while Anthropic, a company that you guys backed doubled from 12 to 24. Why is that happening?”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“You guys don't see a ceiling in the near term in terms of appetite for budgets to shift more towards these AI applications as they get better and better. You think that we will see another, you know, maybe not 500%, but triple digit percentage jump potentially next year.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“I think this is just the tip of the iceberg in terms of what that revenue is going to look like. You hit the number correctly, it was six and a half billion for foundation models. But I think myself and others include who invest in the space strongly believe that these are the APIs of the future upon which the next generation of enterprise software is going to be built. So that 6.5 number, what if it multiplies by five or six X again next year and becomes 36 $40 billion, right? Then you're starting to approach valuations that you mentioned in your point. So I think it's still early days and when you see growth like this, you get excited and you want to invest and this is the result.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“So obviously that 500 surge is a big surge. At the same time, when I see 13.8 billion spend in 2024, that's big, but it's really when you just combine open AI and XAIs funding rounds this year's together, it's that number. Like in terms of the actual money being ploughed into these companies, especially at the model layer, we're still not really seeing anywhere near the level of ROI as what's going in. And I'm curious as an investor and as an engineer, someone who's been in the AI space for a while, how do you look at that?”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“From a use case perspective, it was obviously coding. This is probably making me 70 to 80% more productive as an engineer because I'm not really just an investor. I write code every day. And the productivity for me is off the charts from the code generation standpoint. So it's very helpful. Second after that would be what we call support chat bots. And then enterprise search and retrieval after that. So that'd be companies like Glean, for example, where they're indexing documents in the various departments throughout the enterprise.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Were the biggest spenders and what buckets of spend are growing the most? I think you guys called out coding. That's something I just hear and see anecdotally coding assistants are growing pretty rapidly. But I guess aside from coding, what did you guys see in your research?”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Talk about it as departmental spend. Does departmental spend where IT departments, RD departments, go to market departments are buying? And then there's more vertical I spend in healthcare legal financial services. So people are buying legal software that's heavy on AI. People are buying clinical documentation software for healthcare. That's heavy on AI. And then accounting software that's heavy in AI.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT
“Last year in 2023, a lot of the dollars were going into simply infrastructure sort of picks and shovels, places where VCs like myself like to invest early on when they feel like there's a big, big inflection point in what's happening in technology, like the mobile phone, and we see it being similar to that. You see a lot of these infrastructure platform level investments. Now it's turned to the application side. You saw in the report there's a lot more application spend this year. It increased pretty significantly. And it went to places that would really surprise you like legal fintech healthcare. I mean, that was really, really surprising to us. I mean, that we heard it anecdotally, but the see it in the data was, you know, pretty grounding for us.”
2024-12-05 · Decoder with Nilay Patel · AI is a money pit — here’s why investors don’t mind · IDENTIFIED FROM THE TRANSCRIPT