YouSaid · the spoken record

Eiso Kant

lines on the record: 73
first: 2024-10-07
most recent: 2024-10-07
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“No. $600 million that we've raised till date and the latest $500 million round translates to us being able to be an entrant into the race. And what that means is that the 10,000 GPUs that we've now brought online this summer that came from this capital allow us to make incredible advancements in model capabilities because of the ability to take reinforcement learning from code execution feedback and generate extremely large amounts of data and then train very large models with it. It is enough for this moment in time, but over time it won't be enough.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“GitHub today has this incredible data set, almost all of the code in the world. GitLab is a player, but only in the private side, what sits behind the accounts of developers. GitHub is massive in public code and it's massive in private code. But private code, no one's allowed to train on. Not us, not OpenAI. So all of us have access to the same public data. And it's the output data. And so there isn't an inherent advantage from a capabilities race perspective. Another thing that we frame in our company over and over again is there's a capabilities race in the world. And to your point earlier, we say there's four things that matter. Agree with you on the three, but I'm going to add one. Compute. It's data. It's proprietary applied research. It's the algorithms. And talent. Talent is absolutely key in this industry. In the go to market race, it's talent first and foremost, but it's also product and distribution. And distribution, Microsoft definitely has an incredible position in the world.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“The way I think about this is there's things in the world that today we consider economically valuable, and that ranges from scientific progress to very mundane things. Office buildings full of things that we look at today and say, why can't that be automated? So if we take what's economically valuable, the next thing we need to ask ourselves, what's the gap between models today and human-level capabilities? And how large is that gap? And in some cases, the gap is actually not that large anymore. We were talking earlier about speech recognition. Models today are pretty much there. Maybe there's a tiny bit left to say, but we've closed that gap into an incredible amount. In other areas, the gap felt like it was going to be impossible to close, but we're making a lot of progress. Come back to full self-driving.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“To those things. What we're fundamentally doing is we're bundling intelligence. More and more people got connected together. And I think that's the true underlying thing that has underpinned this technological exponential curve run, right? If you think back about 100 years ago or 50 years ago, you can truly see it's an exponential. I don't think we want to live in any other moment in time. And the reason I mention this to your question is that I think we are now going to go from a world where human intelligence and the amount of humans we had was the entire bottleneck to now we are having machine intelligence and so we can pair investments in energy and chips and compute together with humans and have an extreme explosion on this exponential of all of the places in the world where we want to direct it to. My point is that I think there is a huge amount of places where this is going to be valuable. We will figure out the compute efficiency along the way, the hardware will get more efficient because that's capitalism. As the opportunity is big, we'll”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“This is my personal view of how the world plays out. It's really easy to stay focused on the tactics and things that matter in this moment. And they're exactly the right questions of what matter right now in the moment. But we are moving towards a place where we are closing this gap between human intelligence and machine intelligence. And I think it's going to be an incredible amount of problems and challenges and places where we want to apply this intelligence. If I have a view on modern history, and my view on modern history is that if you look at what happened from the printing press onwards, is that what we've done is we've connected more and more people around intelligence. We went from the telephone to personal computer to the internet to the mobile phone. Fundamentally, what we've been able to do is we've been able to take hard challenges in the world. That's cancer research or if that's even, you know, building a business, a SaaS company, anything. And we've been able to connect more and more people together to direct research.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“Their hardware. And then you can see a big difference between what an Amazon is able to do and a Google and a Microsoft and OpenY and Anthropic, but they also do it on the intelligence layer. We spoke earlier about large-capable models that distill down into smaller models. If you have the most intelligent largest model, you can distill it down into a smaller model and you can have advantages at that layer as well. My view is though that in the extreme of it the compute margin, like the hardware margin really matters as this gets lower and lower in price. And this is what we've seen in cloud computing as well.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“That this there all the time, but I think it's an important thing to understand because when you are buying NVIDIA hardware and you're putting it in a data center and you're working with an Oracle or you're working with whoever it is in the space or even a Google or an Amazon or Microsoft, you are taking that margin of that chip, right? the H100, the H200, the Blackwell generation coming up. And that has to be baked into your costs. The way that I think about this is that the extreme end of it, Amazon and Google and Microsoft as well as they come out with their own silicon have a lot more margin to play with. And now then it comes down to business decisions. And right now, since this is a war, it's a total, you know, like it's an incredible race that's happening. I often refer to it as a drunken bar fight that's happening in our industry is that we are, you know, all these companies are massively incentivized to drop the cost of their models as quickly as possible. And they do that in two ways. They do that by cutting more and more of their margin down to their actual cost.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“And who have as much of that infrastructure already online and brought into the world? And this really is the hyperscalers. This is Amazon in number one, Microsoft in number two, Google in number three. But there's something interesting about all of those. Each of those at different moments in time understood that they couldn't be reliant on hardware built by NVIDIA or AMD by someone else, that they also built their own. Furthest along on that has been Google with their TPUs, things create their fifth generation. They started early on this and they've been improving it ever since. Then you have Amazon who's been working on their trainium and inferential chips, their neuron cores for some time. And Amazon has an incredible background, by the way, in manufacturing chips. And this is something I don't think people take enough credit to because while Google has to work with Broadcom to be able to bring TPUs into the world, Amazon is working with the FABs directly and they have an incredible skill, right? They've done an amazing job in the cloud. And Microsoft's still earlier in their own chip journey. Now, I notice this is, again, a preamble to your question, and I'm sorry for bringing.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“We should separate the price and the cost of models. Okay. If you look at what's happening in the world of general purpose at LLMs, LLMs for everything, it's an incredibly competitive price war that's happening. And it's happening between the large hyperscalers and it's happening between kind of referred to as the escape velocity AI companies, anthropic and open AI. And then you throw in the mix the vendors that are putting up the open source models from meta and such. I often think about what sits in that stack of costs. Well, what sits in the stack of the cost is a server, a box, the networking around it, a data center, the chips, right, the GPUs, or, and then the energy that goes into that. And everything after that is marginal cost or variable cost that the running of the models. So we have to think about who has the lowest cost profile in the space, right? Who has the cheapest first principles capex that they're doing to run these models? Well, that's the people who have as much of that vertically integrated.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“But they're something that we don't really talk about in our industry as much. Well, training is extremely large models. And by the way, we until very recently weren't even capable of doing so because we didn't have the compute and the capital. This is why our fundraise has been so important to us so that we can have the capital to scale up. But what everyone does is that extremely large models, we can't run cost efficiently for ourselves.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“We are starting to understand that the first version, the scaling laws that came out spoke about the amount of data we provided during training and the size of the model, more data, longer training and size of the model larger requires more compute. And so we often say, hey, the scaling laws are about applying more compute. And that's actually more correct than we initially realized because the importance of synthetic data for models to get better is another form of using compute. But we're using it at inference time. We're running these models to generate these hundred solutions, generate a thousand or 100 or 50. I think we have a lot of room still for scaling up models. We can do this by scaling up data and we can do this by scaling up the size of the model. Now, our opinion is that there's a lot of room to scale the number of parameters and size of models.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“Think the biggest cognitive dissonance that people have around synthetic data is a model is generating data to then actually become smarter itself. It feels like the snake eating itself. There's something that doesn't make sense in it. Now, the way that you need to look at that is that there's actually another step in that loop. There's something that determines from all the data that the model generated in my domain and software development, I have a task, in a code base, and the model generates 100 different solutions. If I would just feed those 100 different solutions back to the model and its training, the model won't get smarter. That's the snake eating itself. But if you have something that can determine an oracle of truth that can help say this is better and this is worse or this is correct and this is wrong, that's when you can actually use synthetic data.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“We are horribly inefficient at learning today. If you think about what drives efficiency of learning, it's the algorithms and it's the hardware itself. We've got probably decades, if not hundreds of years of improvements still left there and different forms of it over time. If we look very practically, in the coming years, we are going to see increasing advantages on the hardware. We're going to see increasing advantages on the algorithms. But I hope everyone takes away that this is table stakes. This is something that you have to do to be in this space and you have to be excellent at it. It's not what differentiates you. It's what allows you to keep up with everyone else”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“The small models are the small resolution. We have generalization, but you're losing things. But in the infinite extreme, an infinitely large model wouldn't be doing any compression. So there is definitely a limit at some point to model size. But what underpins all of this, to directly answer your question, is the compute. And the compute really, really matters. Your own proprietary advantages in your applied research to get great data or to gather it matter equally as much. But if you don't have the compute, you're not in the race.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“Of data. But once we have all of this data where we started today, we spoke about neural nets essentially being compression of data that forces and generalizes learning. Now, when we have small models, we are taking huge amounts of data and we're forcing this generalization of learning to happen in a very small space. And this is why we essentially see these difference in capabilities, larger models require essentially it's easier for them to generalize because we're not forcing so much data into such a small compression space. And so my personal mental model of this is that the scale of your models, and this has been shown over and over again. By the way, we owe a depth of gratitude to Google, to open AI, proving out the scaling laws, which essentially say as we provide more data and more parameters, more skill, hence more compute for these models, we get more and more capable models. Now there is a limit that most likely. If you think about it as the analogy to compression, your image that you had a high resolution compressed down to small resolution.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“We are making in our space, especially, I think, post the ChatGPT moment, like incredible advancements in the algorithms that are making learning more efficient. Internally, I have this thing that I say to the team, and they're probably tired of me hearing because I say it every single day. So all the work we do on Foundation models, on one hand, is improving their computer efficiency for training or running them, or on the other hand, improving data. Now, the way to think about the algorithms and the improvement of computer efficiencies, that's table stakes. All of us, open AI, anthropic, Google, et cetera, are doing this, and we're just constantly improving here. And it's engineering and research combined. But the real differentiation between two models is the data. Compute matters tremendously for data. Because if you think about pool side, and we spoke about how do we get this data, and I mentioned the word synthetic, it means that we're generating it. to then run it and so compute hugely matters on the side of the generator.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“This is the right question. The way I think about the world is that there are problems that we cannot simulate. The real world is impossible to perfectly simulate. It's messy, it's multivariable. How do we deal with the real world when we're trying to close the gap between human capabilities and AI? We have to gather data. The best example of this is Elon and Tesla. Elon has put millions of cars on the road that are actually capturing”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“Actions that we took to get there. And that's the missing data set. The missing data set in the world to go from where models are today to being as capable as humans at building software is the data set that represents being given the task all of your intermediate reasoning and thinking, the steps that you do, the code that you write and try to run and then it fills and you learn from those interactions. And all the way to kind of getting that final product and that intermediate data set, that's what pool sight exists on creating.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“Capable areas. And I mean improvements in reasoning, improvements in planning capabilities, improvement in deep understanding of things. Well, as humans, we don't require so much data. The way to think about models is that they require magnitudes order, more data to learn the same thing. Our focus is on software development and coding. And it's for a very specific reason. The world has already generated an incredibly large data set of code. To put a little bit into context, like usable code for training, what we refer to as about 3 trillion tokens. And if you look at kind of usable language in English on the internet for training, we're talking about anywhere between 10 and 15 training tokens. There's a massive amount of code that the world has developed, over 400 million code bases are publicly on the internet. So why don't we have this incredible AI that's able to already do everything in coding? It's because coding is not just about the output of the work. The code that we have online represents the final product, but it doesn't represent all of the thinking and”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“Capabilities in areas that are massively economically valuable and can drive abundance in the world for all of us that are not going to be equally distributed, not for every single thing. And what I mean by that is that if you think about foundation models today, and I have a kind of simple mental model about them, which is that we are taking large web skill data and we're compressing it into a neural net, and we're forcing generalization and learning. And this has led to things like incredible language understanding in these models. But it's also led to things where we look at and we say these models are quite dumb. Why aren't they able to do X, Y, or Z? And our point of view is that the reason why they're not able to do X, Y, or Z has to do with how they learn. The most important part, I think, of what I said is the scale of data. When we have web scale data, we can get language understanding. But when we have areas where we have very little data, models really struggle to learn truly more.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“Sides in the race towards AGI. We think the future is going to play out, that the gap between machine intelligence and human level capabilities is going to continue to decrease. But are the path towards that, in our opinion, is by focusing on building the most capable AI for software development. And all of this comes back to a set of foundational beliefs that we have that I would say are different than some of the other companies in this space in terms of where both research is heading and where capabilities are heading. The term AGI is a loaded term. And the way that I like to kind of take the definition that is most commonly used is that at some point we are going to be in a world where across all sets of capabilities that we have as human beings, machine intelligence is going to be as capable and if not more capable than us and surpass us. Now our point of view is that that world is still quite a bit out and that we are actually going to end up in a place before that where we see human”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“Thank you, Harry. It's a pleasure to be here, and it's glad that we finally met in person. It's been a minute since we've known each other.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source
“Who has earned the right to be in the race to AGI? We're going to look back on this moment 10 years from now, just like we would look back to the moment of mobile, internet, and realize that that was the moment where the table got set. You do not want to look back on that moment and not have given it everything you've got because it's a race and the latest $500 million round translates to us being able to be an entrant into the race. We don't get the luxury of stumbling on the capabilities race or to go to market race.”
2024-10-07 · The Twenty Minute VC · 20VC: Raising $500M To Compete in the Race for AGI | Will Scaling Laws Continue: Is Access to Compute Everything | Will Nvidia Continue To Dominate | The Biggest Bottlenecks in the Race for AGI with Eiso Kant, CTO @ Poolside · IDENTIFIED FROM THE TRANSCRIPT · source