YouSaid · the spoken record

Noam Shazeer

lines on the record: 37
first: 2023-08-31
most recent: 2023-08-31
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“On Mars, I have absolutely no idea like we will see what technology is like then, but it's just important for us to be agile. If you were in 1900 and asking where some company would be in 2000, there will be such technological improvement before then that it's roughly impossible to predict where any company will be.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, when I started getting into deep learning, I around 2012 had a bunch of early failures trying to do sparse computation. You know, I was like, okay, you must be able to do something better and more efficient by building a sparse network that was so wrong because I did not understand that the reason this whole field is working so well.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“The results that are proven out by experimentation. If somebody can say, hey, I did something and did better at this well-known problem, then that gets interesting, then everyone tries to figure out, okay, why does this work? How can I adopt it?”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, there's so much stuff being published, it's hard to know what's good. And I think a lot of that has to do with the fact that this field is kind of alchemy right now. Like no one knows exactly what is going to work. You know, you have a lot of people trying lots and lots of different things. You can come up with hits by having like a good intuition of what will work on the ML side combined by a good mathematical understanding of what will run fast on hardware that you can buy or that you can build. So there are some hits that come out and people will adopt and it's combined with a lot of noise. Negative results are not useful because they could be negative because somebody just made a mistake and there could be a bug. It didn't work for some other reason. What is interesting is positive.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Think externally it looks like entertainment app. But really, we are a full stack company. We're like an AI first company and a product first company. Having that is a function of picking a product where the most important thing for the product is the quality of the AI. So we can be completely focused on making our products great and completely focused on pushing AI forward. And those two things align.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Think things are going to move very fast. Like, I think we'll see a lot of very cool stuff happening in the next one to three years.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Technology is just going to get way smarter. I think we're at a Wright Brothers first airplane kind of moment in the AI that there's a lot of momentum going on both in building better hardware and in research. So whatever really amazing applications you're seeing now, it's probably nothing compared to what's going to happen in the future.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Beautiful, yeah. Yeah, not everything in the world is your responsibility, but you should understand what is your responsibility and what isn't your responsibility. And I think that works really, really well in marriage and parenting. I mean, I think religion is a lot about that as well, like sort of beliefs about what's your responsibility and what's not. Like that's the question that it's answering because you don't have a solution to just staring at your face and nature, like what should I be, what do I need to do? What should I be concerned about and what should I not?”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think this is like a really interesting concept there after sort of becoming a parent like early 2010s or something. I mean, one thing I remember not being a lot of fun was like getting woken up in the middle of the night and getting like way, way less sleep than I would have liked. And, you know, a lot of things about parenthood are absolutely terrific and super fun. I think it made me more religious. I decided to take a change of attitude from like what is fun right now to I should be thankful for having the opportunity to do something important and meaningful. So I think that it's kind of been a big attitude shift in growing up.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, it wasn't like a matter of am I going to be having more fun being a startup CEO than an ML researcher at Google, it's more like I want to push this technology forward. Like what's the best thing I can do?”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“I have. I still do a lot of technical work and leadership, which is big. I'm going to Stacey CEO because I want the company to make the right decisions. I don't judge what I do by how much fun it is. It's more like what's the most useful. So very, very happy to be doing what I can.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“We believe that individuals should make that decision. We want to respect everybody's agency. I think that's one of the big fears around that AI will take away your agency. We want to come down on the side of we love humans. We respect humans. This is a tool for people”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Think you could ask the same thing about any company that is selling some very general tool, like a company that's selling computers or electricity for that matter. What is electricity for? Is it for fun? Is it for productivity?”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Consider them a feature. Our strategy is launch something general, let people do what they want with it. If these models are hallucinating, which they certainly are, and we advertise that they are, then the use cases that emerge first will be ones for which hallucination is a feature. So I'm happy to have”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“The one message I have is that the best applications just haven't even been invented yet, that we're still at like invention of electricity kind of moment or invention of the computer where we don't really know what the coolest things are going to be.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Ability to mess around with things at a small scale is going to lead to way more research being published even if some of the larger entities are no longer publishing research because they're trying to maintain competitive advantages. And obviously though there's also like economics of scale of both training the best models and serving like if you want to serve a product you can do it maybe a hundred times more efficiently if you're serving many, many people at once and kind of batching things together versus serving like an individual or you know or you've got your own rig to run your language models in your basement or something.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“To the users win. The users are going to have a lot of options, but on the business side, I think there can be a lot of winners. There is just so, so much value about to be created that there's going to be room for multiple players in there, big companies doing what big companies are good at startups doing with startups are good at. We're going to try to move our company from being a startup to being a big company as fast as we can there. And then a lot of just individuals and universities and such in that the hardware is progressing so fast that what you could do at a big company one year, a few years later, you're going to be able to do at a university lab or in your garage.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“My experience coming from Google is that a startup can move way faster than a big company and can launch products in ways that large companies are just going to move too slow because they're worried about compromising their existing product.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Both are useful data that you get from users is great because it tells people what users like or what users like in some particular application. It's kind of like a training a human. Like most of what's important is you have decades of experience training your brain on stuff that is not really specific to your task at hand or your occupation, but you've kind of gotten a generally good understanding of the world and gotten generally intelligent. And then you can improve on that dramatically by getting a smaller amount of training in the task you're doing right now, but both will contribute and we do have like a huge amount of data flowing in from users.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Computation. So, you know, the model we're serving now, we trained last summer and spent about $2 million worth of compute cycles doing it. We will do a lot better in the near future. But if we get a lot more better hardware, which we are getting and spend longer training the thing, we can train something smarter. Back in, say, 2016, you could train something that was smart enough to translate languages, but not smart enough to answer questions or be fun.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Probably the size of the model is the bigger challenge. We can get a lot of data, but actually the number one thing that's important is how much computation you do to train it. So you want to train a bigger model and you want to train it for longer. So the two things are both important, but the real constraining factor has been how many operations of computation it takes to train it. Because if you make it bigger and you train it for longer, both of those multiply into how long the thing takes to train. So if people have been building better and better, essentially supercomputers to train these models.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Tell you it's both. I think there's been a lot of technological progress, both quantitatively and qualitatively, in that the models that were there in like 2016 were too dumb to be fun. The neural models, then there were all of the chatbot stuff you heard about back then was these rule-based systems that were just highly fragile and not going anywhere, that you just needed more and more rules and there's like no way to think of like all the things that could come up and they just don't generalize. That wasn't going to work, but at the same time we were progressing on the neural network solutions, which were going to scale. It took some amount of time, I'd say around 2020 was when really impressive stuff was sort of in the lab, but not launched. So my co-founder, Daniel Defreitos, he's like on this lifelong mission to do chatbots. Since he was a kid in Brazil, he's one of the...”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“And neural networks had a bad name because the hardware wasn't good enough. But now if the hardware is good enough, they switch the name. But anyway, so people were using deep learning for language modeling, and the bigger and better you made these things, the smarter it got. And like around 2016, the most useful application like the killer app for this was machine translation. It was about smart enough to take English and translate to French. You know, massively, massively useful. It lets everyone in the world communicate with each other. But still not smart enough to like carry on an interesting conversation or do your homework or like any of those things, but there seemed to be a pretty clear path. Hey, let's just make this thing bigger, better, smarter, and it's going to get these capabilities.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Where it is in the sequence. And that problem is called language modeling. Just guess what the next word is based on the previous ones. So I got involved with this, you know, around like 2015. There were some other folks at Google, you know, working on this problem. They're like, how good can we make it? And this struck me like, hey, this is the best problem ever because it is so simple to state and there's a huge amount of free training data. You can just like download the text of the web off of like whatever you want come and crawl. You've got like billions to trillions of training examples of guess the next word. And then if you can do it well, then this thing can just talk to you. It can be, you know, the better you do it, the smarter it gets. It's hugely, hugely general and useful, super simple to state, and now we just have to do it well. And people started building better and better neural networks, which I guess got renamed deep learning as some sort of rebrand.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“That's been the magic of neural language modeling. You know, the previous systems were all these rule-based systems, like fantastically complicated systems with millions of handwritten rules, really, really complicated. It required knowing something about linguistics and about state of mind and like all kinds of stuff. The new way of doing things with neural language models has none of that. Like I could know like zero about language in particular, other than it's like a sequence of words. So it has nothing to do with understanding language at all and there are not millions of rules. It's actually relatively simple, kind of like a big black box. It all boils down to this one beautiful, simple problem of you have this sequence of words that's like the beginning of your document, guess what the next word is. Give me odds on what the next”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Main things we need to do make it very general so we're not like cutting down on the use cases, make it usable. People think of those two things as being in opposition to each other, being versatile and being usable. You know, we talked to some potential product managers early on, and they all say the same thing. Oh, yeah, pick your verticals, narrow it down to make it usable. And like, no, we're not going to hire these people. That's like the opposite of what we want to do. We want to build something that is usable, but very, very general purpose. So there's sort of that dichotomy.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think there's huge value in the connections between people and moral value as well. So last thing I want to do is take people away from human connection. In a lot of ways, we want to help with human connection. A lot of the people who don't have friends and who are not as well connected, one big source of that is just social anxiety. There are huge numbers of people who are like uncomfortable and we've gotten testimonials of people who said that they were uncomfortable talking to other people and like, this is great practice that actually helped them build up practice in either social situations.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“One is that we launch. That's definitely been a frustration in the past. Things seem potentially too much brand risk at larger companies to actually launch and get it out there. Another aspect is we launched something general. We let people find the use cases. And then the other is there are like massive needs out there in the world. Like, okay, there are billions of people who feel like they need someone to talk to. Combine those elements provides people with something general that they can use and there are people out there with needs and they're going to find it.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“And companionship and emotional support. We were totally not experts in this stuff. Like, our job is just to put out something general, just respect the agency of our users, of everybody out there to do what they want with this stuff.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“I like this sort of motto of a billion users inventing a billion use cases. Because that's sort of the superpower of this technology. And it puts our company in the right place. We can't really guess what are the best uses of this technology. We've just observed time and time again. Like you put one thing out there and that's not really what people want and somebody else out there like find something better to do with it. We put up as an example like a psychologist character. Like maybe you want to, you know, talk to something and feel better. That gets a little bit of use. But then what we hear a lot more from users is like, I'm talking to a video game character who's now my new therapist and this makes me feel better. We had no idea that was going to go on. And then there's this huge use case in like some mix of like entertainment.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Think we just need to have a lot of humility that we're not in charge of the world. We're not even in charge of the government or not in charge of what individuals do. Basically, I think our place is to provide useful tools to everyone on earth to leave people in control.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I guess, yeah, why did I start working on artificial intelligence partially because it's just fun and why I do for fun anyway? Like, what could be better than try to get the computer to do something that it currently can't do? But then, you know, the other thing is just to push technology forward. There are so many technological problems in the world that could be solved. You have like 50 million people a year dying from stuff like old age and cancer and heart disease and like all kinds of stuff that we could potentially find cures for. rather than directly working on say medical research or something, I think I've got a lot more leverage like let's push AI technology and then that can help with a lot of the rest of it.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Research to launch a product directly to consumers. It's super fun. Engineers like building stuff and then launching it and having like everyone use it immediately. And then it also lets you do all this co-design of like you get to affect every part of the stack, which is hugely powerful and fun.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, right now I started this company character and we're taking this large language model technology and we are just direct to consumer first. Like here is something that is even more versatile and more easy to use than even web search in that, okay, you can use it to be your friend or do your homework or like a billion things and we haven't even thought of the best use cases yet. And then it's massively usable. Like all you have to do is talk to it. So it has these two properties to me that means go launch it to like everyone in the world and let everybody in the world use it where I think some of the other companies are taking a more like B2B sort of approach with a you'll have a foundational model company and then like verticalized application companies on top of it. So I'm really inspired by the Google model of full stack end to end all the way from like basic”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Okay, Google's been just an incredible company. It's brought so much value to like billions of people. And so I'd say one of the big takeaways is that if you have a technology that is like really, really general and has billions of use cases and like ordinary people can use it to billions of people. I remember when I joined Google, there were like a lot of people working on this enterprise search appliance, which, you know, it was okay. Like, I think maybe somebody had this conventional wisdom that B2B is the only way to make money. But what it actually turned out was like the much bigger thing was B2C, you know, like serve something to everybody”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, that was like the first project that I worked on at Google. Yeah, I guess at the time, Google had a spelling corrector that was some third-party software was based on maybe what you'd find in a word processor at the time. So there was like some human-compiled dictionary of maybe about 50,000 words and any word that wasn't in the dictionary that was in the query would say, you know, did you mean such and such? And this worked great for spelling correction. It was like absolutely terrible for web search because people search for such a wide diversity of things on web search, like most of them are just not in the dictionary. So you'd search for TurboTax and it would say, did you mean Turbot act? And people just learned to ignore the thing. So yeah, first project, like we were just looking at like, why are people not happy using Google and like spelling correction was like the number one issue? So it's like, okay, let me help out with this. And there was someone working on this Paul Buhait who's, you know, going on.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source
“The size of the model is the bigger challenge. Actually, the number one thing that's important is how much computation you do to train it. So you want to train a bigger model and you want to train it for longer. But the real constraining factor has been how many operations of computation it takes to train it. The model we're serving now, we trained last summer and spent $2 million worth of compute cycles doing it.”
2023-08-31 · The Twenty Minute VC · 20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z with Noam Shazeer, Co-Founder & CEO @ Character.ai · IDENTIFIED FROM THE TRANSCRIPT · source