YouSaid · the spoken record

George Sivulka

lines on the record: 92
first: 2025-01-22
most recent: 2025-01-22
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“I've loved it and appreciate all the research that you've done and all the crazy lines of questioning. So thank you, Harry. I appreciate it a lot.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think so. Yeah, I think so. I think they've heard about it. There was one moment where I think my father's boss ended up calling him and it's like, you know, your son's kicking ass. And I was like, well, Was a very happy moment for me.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it's one of those activities where you can channel emotion or intuition or latent thoughts that are somewhere in your subconscious and connect things in a really meaningful way. And so in a world where there's all the stimulus or you're always kind of thinking or churning through something or all this distraction, you're standing in front of a canvas for like 10 hours with some nicotine and you're just lost in this art. I think great artists will tell you that they don't even know where paintings come from. It just, you know, is this channeling something. It's one of the best places to think. It just gives you connections. It brings up these parts of your subconscious, these connections that I think you can't really access without being creative, whether you're making music or writing or painting. I actually think that's one of the best ways to process anything.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I love to write and probably not as good a poet. But I actually think that other creative outlets are really, really good for it.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“That one of the most interesting questions is where does creativity stem from or where do you get inspiration from? Or kind of like how do you come up with new ideas? I don't believe that people come up with new ideas by brainstorming or I just think that's again Fugazi Fugazi. But I think that ultimately that question of where creativity comes from, I'm also a very big painter. I'm a very big, so I do large scale like 10 foot plus oil canvas oil painting.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“Not a company. I'd love to be mayor of New York, believe it or not. I just think that's like a fascinating job. I think it would be really, really interesting and would love to make some change there.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“That's a deep question. I would probably buy NVIDIA. It's a lame answer or AMD rather. I think AMD because I believe that they will benefit from the shift to inference scaling more than in an outsized way.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't think they are. I think that Salesforce has built a very, very, very sticky network effect with people. And people are the shifting function at the end of the day. It's not a technology problem. Claude can build a Salesforce. I think Klarna, again, had another Fugazi story about building going off Salesforce because Claude had built them a CRM. I just think that the switching costs, the network effect of changing human beings' habits is too high. Salesforce is one of those monopolies in that they have so much stickiness, habitual stickiness.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I don't think that in case you needed the content, I know exactly what you're getting at here. I really, you know, I'd probably kill them all. I don't think that traditional enterprise B2B applications are sexy or an enterprise AI company.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“Longer than 12 months, probably like 18 months ago, it was the scaling at inference thing, like the belief in a new set of scaling laws that they would be really, really, really important.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that we are a Silicon Valley company in terms of our style of work, in terms of how hard we work, in terms of how we actually pursue new technology and invest in technology. And we started in Silicon Valley. And we have almost only Silicon Valley investors”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“Hebia, I think there was a podcast that actually recently came out where everyone's like, if you're going to build an AI company, you've got to build it in Silicon Valley. But there is one company in New York that is doing a really amazing thing. That company's head yet. It seems like they're actually doing something interesting. And I do think we are the exception rather than the rule, unfortunately. So I'm a big believer in Silicon Valley.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I get off for like 8 20. My first meeting is at 8 30. There's like a I try to work out every day. I actually end up mostly channeling the startup pressures and anger and anxiety into heavier and heavier things and lifting heavier and heavier things. So it's nothing that's like in particular.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“When you dive into the human psyche and you're not looking at your phone and a lot of the time, it's also a really great channel to think. I think a lot of the best ideas that I've had at Hebia have come from moments of silence.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“You know, I live by myself sometimes, but sometimes it's all in my head. I think it's incredibly good for the human mind. I think it's almost an antivirus for the human mind.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I do. I pray for an hour every morning. What? Yeah. I wake up. I sit on a meditation cushion. And I used to meditate. I think meditation is also great. Praying and then putting something out into the universe or, you know, actually having a dialogue with whatever you believe. I actually think is even more powerful. It's almost”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“Punch pieces. I think that ultimately when you're doing hard things or when you're chewing the glass or working all like those really late hours, believing in something larger than yourself or believing in what you do as a vocation or something that's deeply purposeful and deeply meaningful is actually it's additional fuel. It helps you in a way that is, I think, good for the soul. It really charges you up. Do you pray?”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't think I'm ashamed of it per se. But one thing that I always hid was the fact that I'm deeply religious in an industry that's very atheistic or agnostic. It was like something that was very personal to me. And I think it's been massively contributing to how...”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I have a crazy one. I believe that UFOs are real. I think a little bit more on the nose right now, but I actually believe there's fundamentally different propulsion technology. And that I think the U.S. government has access to it.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“It's somewhere between Peter talking to me about anything but the business and deeply academic things and Mike taking me on a walk around the Woodside Horse Park.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I actually think that we are human first. We're business user first to the point where CTOs like to pay for consumption or API, et cetera, and like, you know, business users like to pay per seat because it's how they map back to value. But also we want to incentivize change. Tech is not the hard part of all of this. It's hard. But the hardest part of AI change management, no matter what company you are, are people and actually getting people to use the software. When you charge for consumption or API pricing, you're disincentivizing the change. You're saying, okay, well, I'm going to penalize you in a monetary way for every time you use an AI application. What the heck? Versus here's a perceipt fee. It might be expensive, but use it more. You could run more LLM calls on Hebia effectively for free than any other platform if you actually are driving real change. And that's what I love to see.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think ultimately the people in the enterprise that are most excited about AI and positioning it so strongly are CTOs and information technology people. And maybe the thing that Hebbia has always said is that the CTO or the IT folks are actually the people that know the least about the business. The people that actually understand how to use AI in a business context are those that are closest to the business. And so we're jumping the gun a little bit with the CTOs trying to build the CRM before it's been invented. And you actually need business people to build the CRM in Excel first in kind of that order of operations. And so there's a lot of unbundling of AI applications or CTOs trying to go out and build a very specific vertical application. But I actually think that building this platform, Heavy Matrix, is the thing that will unlock users ability to discover what they can use AI agents for.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that 90% of the market is still in experimental budget phase, but we're starting to see early promises of actual value. And my entire business is focused on just those repeatable use cases.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“Probably be large tech providers and AMD. I don't know about Intel, right? I would probably bet on them. There's definitely an opportunity in the market, but chips are hard.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“The minute they deploy them, they can deploy them on cheaper infrastructure. And that actually, I think, will be a big change. So I'm actually still bullish on NVIDIA, but I'm even more bullish on other chipmakers and custom ASICs to do inference because I think there will be a larger shift to inference moving forward.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“You are not alone So Nvidia has, I think the best motes aren't technological modes. They're not data motes. They're actually people motes. Networks have the most friction to change. One of the things that NVIDIA does best is the fact that they made this early bet on machine learning. They created CUDA, which is the way that, as I mentioned before, almost everyone learns how to train models. They learn how to interface with NVIDIA chips for training. And as you're starting to see, maybe that prediction that I made earlier, the shift away from training to inference as a fundamental, like almost macro shift in how people deploy AI, I actually think that will destabilize slightly the dominance of NVIDIA chips. You can start to actually use AMD chips or even custom architectures, which all the major model providers are also currently exploring to do inference. So you have your academics and your researchers, you know, training large models on NVIDIA chips.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think the thing that I care most about in the world is that we as an industry have very clear guardrails that we can follow and understand to build the best possible tools to get our tools out to the economy, to make sure that everyone transitions in the best possible way. So I'm ultimately regardless of thriving.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it will be his greatest challenge. There's a lot of self protecting mechanisms in the largest organization in the world, which is kind of a U.S. government by SPEN. It's just this massive unruly organization. It's not going to be as simple as Twitter.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I actually think that the better agents are, the more work that they do, the more important it will be that they are easily understood by humans. The idea would be, okay, let's say we have a bunch of employees, 10,000 employees or 10,000 AI agents drop at a company. They're all experts at doing something. That ends up not becoming a problem of giving them the right tasks, but actually becomes a management problem. There's this whole infrastructure orchestration layer, the thing I always come back to of making these things work together. And that's actually going to be the challenge. And that's going to require a very human first, ultimately a product. And that's what we're trying to build.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that Hebia is the Bell Labs. And I conceive of ourselves as the Bell Labs of defining AI interfaces, right? I think that RAG was one of them, i.e. this idea you could find things in the data really fast, decomposition in agents are another this idea of scaling at inference with our matrix product is another. You can look at a lot of the other things where agents are controlling four screens at once and you're actually looking at someone use a computer or computer use where AI models are moving cursors are others. Almost all of them have agents”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I ultimately do not think so. I think that chat was always a useful feature. It's a useful interface. It's like a single cell in Excel. It's like asking if the TI-84 was the right interface for computers or the terminal was the right interface for computers. We have not even started to explore the opportunities for interfaces. I actually think that...”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I actually always like in technological revolutions to what HEBI is doing right now, where people invented, we discovered the technology of fire and then someone invented the torch. I don't know how many years later, you know, we invented the engine and then someone invented the car or the wheel and then the chariot. And so this idea of encapsulating and building a useful product on top of a technology change is actually the thing that takes more time. And I think that Hebia has built, if Excel was that product for compute. I actually think Hebia has built that product for AI. And I think that when you have a good product, that transition will be very, very, very quick. Right now we have these chat bots or these surface level search engines that give you facetious service level value. Yeah, it'll help your kid cheat on their homework. But to drive to whether or not something's a good investment is a much, much more rich problem.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I genuinely believe that computers made legacy businesses better if you use them correctly. And so it's a massive disrupting force. But if you can ride the wave of change, a AI agent.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I would, but ultimately, I think all of them are undervalued. I genuinely believe all AI companies and the S&P 500 are all undervalued, which is a very hot take. If we're about to create $100 trillion of value, I think this is a real tangible technological shift. It's a massive unlock on the order of what computing did for the entire economy over the last 60 to 80 years. I think this will do for the next 60 to 80 years. I think all these companies are massively undervalued, including the non-AI companies.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“Think geopolitics is actually very important. I think that governments will be some of the largest users of AI, especially with some of the recent things that the new administration in the United States has been talking about with increasing government efficiency. I think that ultimately energy is a very big bottleneck. It's a very common thing in Silicon Valley to talk about, hey, we need nuclear reactors to flatten the duck curve so that we can continue to drive to larger and larger data centers, et cetera, et cetera, et cetera. And those are ultimately geopolitical resources. And so I think all of these things end up being very important. And then Elon's just operationally so talented, right? So I think that ultimately if this becomes commoditized and whoever can really operationalize model creation and serving models the fastest, I think it might certainly.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think Elon is very well positioned in the geopolitical sense. I think Elon can run a more efficient business and not have to deal with as much administrative bloat or as much friction from employees. How important is you?”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think XAI is the most undervalued company and a really spicy take. I actually think XAI might overtake OpenAI and anthropic in value over the next 12 to 24 months, which is crazy. But I think they're all undervalued.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“Absolutely. Whoever has the best models will continue to attract the right amount of investment. The different thing about clouds too, though, is that the cost of switching is much higher. So to refine my earlier point, right? Like I can switch models readily. I think there's even entire businesses now. There will be an entire industry of being able to switch models from OpenAI to Anthropic when OpenAI goes down. But to switch clouds is for any substantially sized startup, like a 10 million to 20 million dollar investment just to switch. It's almost always never worth it. It's much, much, much stickier. Whereas here, it's a very simple API key. It's very simple to switch models. And so I think that's also a different shader.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it might. There's probably fewer players and more entrenched players in cloud. And ultimately, I think those players honestly kind of have like an OPEC oligopoly where they can control pricing. I just think that ultimately cloud is actually more complex than training larger and larger models.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“Ultimately, the model layer, and I think this is not a hot take anymore. I've been saying it for a few years, but I think it will become commoditized. I think that a lot of value will accrue at the hardware layer. And we could talk about what that means for NVIDIA, especially as NVIDIA has a stranglehold on training, but not as much a stranglehold on inference. And so you might actually see other chipmakers actually start to, their chips start to be used in a more meaningful way because CUDA is what all ML scientists were trained on in their PhDs, but then inference doesn't matter kind of what you're using. And I think it will be the infrastructure layer and then actually the application or agent layer that will accrue the most value. Ultimately, why does”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“You know, I think the one thing that people in my position will always tell you is that the cost of intelligence will go to zero. The cost of intelligence will go to zero. I mean, I think that since Hebia started, the cost of inference over a fixed number of parameters has decreased by seven orders of magnitude in four years. And so I genuinely believe that scaling compute is like a no-brainer. And yes, we run more large language model calls than anyone might even say would ever be necessary, but we have the best accuracy in the business. We can answer much more complex problems. We're driving real value for enterprises. And I actually think that every single quarter, like our margin goes, we're not spending money fast enough.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“So ultimately, that might mean that, hey, we're training larger and larger models, XAI, again, just created the largest GPU cluster of all time, and they're going to try to train larger and larger models. But regardless of how the scaling laws for training larger models or parameter count and accuracy or performance carry out, I'm starting to believe that you could still get better compute, not by building a larger engine to use a metaphor, but by actually putting a bunch of smaller engines together. Hebia by orchestrating large amounts of inference to answer one single question ends up kind of building like a Tesla where Tesla is made up of a bunch of smaller engines or a bunch of smaller electromechanical motors that make a lot of torque and are really amazing larger engine.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that we're starting to run up against the constraints of it. That's a gut feel. I'm not looking at particularly in data collection myself, but I think we're starting to run up against the limits of really good data that we can.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think, you know, it's a bit of a marketing distinction, right? But ultimately, the idea is that the way that we got here over the last five, seven years of training models has been let's build a bigger and bigger model and let's give it more and more data, more and more clean data. And then maybe we'll do some RLHF or some reinforcement training to fine-tune it after pre-training. And that worked great to get us here, but we're running up against the amount of good data that exists in the world. We're running up against...”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“So then GPD4 was released, I think, like a few weeks later. I don't know exactly the right timeline, but it just destroyed Bloomberg GPT at every single”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“This makes me think of the story of Bloomberg, which has the best financial services training set of all time, and they trained a GPT 3.5 class model. It was called Bloomberg GPT, and they released an archive paper, and everyone on LinkedIn was like, wow, Bloomberg's, you know, cutting edge, and they're going to steal finance AI.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“We've seen that for certain types of documents like the dense legalese or more colloquial documents, anthropic works better. But for other types of documents like OWAN or OpenAI 40 works better. And it's always trade-offs between accuracy and speed and all kinds of things. Actually, a lot of the time when we're decomposing a task, we'll use mixes of open AI and probic, even Gemin”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“We pioneered, it doesn't matter if you're using Claude 3.5 or if you're using O1 itself, i.e. scaling at inference at the orchestration layer with something that was scaled at inference with the training layer. But you get way better results. And it's a way to drive to more accuracy.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“Right now, you actually, so Open AI is starting to do this with O1, where they'll have a model recursively think about a question over and over and over again before it produces an answer. And so instead of training a larger model, they're using effectively a similarly sized model and just telling you to run multiple cycles, i.e. compute more before answering. Hebia has actually pioneered something different, where a year almost 18 months ago, we said, hey, we can't wait for these models to catch up. What we'll do is infer simple single question, let's actually run hundreds or even thousands of submodels of the best models in the world to compute over every single document, to answer the same question. And so ultimately, if you can't train larger and larger models fast enough, you could take whatever state of the art or cutting edge and run it more times to get more compute, i.e. more computational power, better decision making for the same user right now. And so this is an idea that”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source
“We are completely model agnostic. We use all of the major model providers, some of our own models, but ultimately the foundational difference that Hebia is capitalizing on right now is fundamentally new and very important difference, which is I actually think on the order of creating rag and creating agents and decomposition is this idea of us in the last year or so having pioneered scaling at inference.”
2025-01-22 · The Twenty Minute VC · 20VC: Why All AI Companies Are Under-Valued | The Future of Foundation Models: Scaling Laws, Generalised vs Specialised, Commoditised? | From Unable to Afford Rent to Raising $130M From Index and Peter Thiel with George Sivulka @ Hebbia · IDENTIFIED FROM THE TRANSCRIPT · source