YouSaid · the spoken record

Mike Krieger

lines on the record: 107
first: 2024-09-09
most recent: 2024-09-09
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Good product design pushes extremes in some direction. Like, this is the lots of data, but also push the latency extreme and see what happens when you combine those two axes. And that's a thing we'll continue pushing for the rest of the year.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, so on the Claude side, I think the time we talk, and it airs, we're launching Cloud for Enterprise. On the surface, unexciting acronyms like SSO and Skim and Data Not Management and Audit Logs. But the importance of that is that you start getting to push into really deep use cases. And we're building data integrations that make that useful as well. So there's that whole component. And then on the API centers, we didn't talk as much about the API set, although I think of that as much as an important product as anything else that we're working on. The big push is how do we get lots of data into the multiple models or ultimately they're smart, but I think they're not that useful without good data in there. It's like tied to the use case. How do we get a lot of data in there and make that really quick? So we launched explicit prompt caching last week, which basically lets you take a very large data store, put it in the context window, and retrieve it 10 times faster than before, look for those kinds of ways in which the models can be brought closer to people's actual interesting data. Again, this always ties back to Artifact and get you personalized, useful answers in the moment at speed and at low cost. That whole push, I think a lot about like”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“That we seek predictability where we can find it. But I think I've also seen the value of like within that constraint with the right tools and the right sort of infrastructure around it, how it could be more robust to needed messiness of the real world.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“And the to-do list system was down. And it's like, oh man, I tried to use the to-do. I couldn't do it. You know what I'm going to do? I'm going to set a timer for when you meant to be reminded about this task. So it set an absurd timer. It was like a 48 hour timer. You would never do that on your phone. It would be ridiculous. It to me showed that non-determinism also leads to creativity. And that creativity, like in the face of uncertainty is ultimately how I think we are going to be able to solve these higher order, more interesting problems. And that was the moment I was like, it's non-deterministic, but I love it. You know, it's like non-deterministic, but I can put it in these odd situations and it will do its best to recover or like act in the face of uncertainty. Whereas any other sort of like heuristic basis, if I had written that, I would never have thought of that particular workaround. But it did. And it did in a, I think, pretty creative way. So I can't say it sits totally easily with me because I still like determinism and I like predictability and system.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“It's a huge adjustment, right? Like I'm an engineer at heart, I like determinism in general, you know, like we had a insane issue at Instagram that we eventually tracked down to using non-ECC RAM and literal cosmic rays were flipping RAM. Like when you get to that stuff, you're like, I want to rely on my hardware. Here's the moment. It was actually a moment maybe like four weeks into this role where I was like, okay, I can see it's the perils and potentials. We were building a system in collaboration with a customer and we talk about tool use, right? Like what the model has access to. And we had made two tools available to the model in this case. And one was a to-do list app that it could write to. And one was like just like reminder sort of like short-term or like timery type thing.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“This all just adds up to my feeling that prompt engineering and then teaching a model to behave itself feels non-deterministic in a way. Like the future of computing is just like misbehaving toddler and we just have to contain it and then we'll be able to talk to computers like real people and they'll be able to talk to us like real people that just seems wild to me that even if you're going to release the system prompts I read the system prompts and I'm like, this is how we're going to do it like Apple's system prompt is do not hallucinate. And it's like that's where this is how we're doing it. Does that feel right to you? Does that feel like a stable foundation for the future of computing?”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Shaped mistakes, so you can be like, oh, danger zone, like talking outside of our piece there. I even like the idea of even having some almost syntax highlighting for like this is rounded from my context. This is from my model knowledge. This is out of distribution like Danger Will Robinson. I'm not sure if this is exactly where I'm like, I'm not exactly sure what I'm talking about. Maybe there's something there.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“App that has done this most well recently. I have no affiliation with this other than we listened to her parenting advice all the time, which is like Dr. Becky, who's like a parenting guru, has a new app out. And I really, I like playing with chat apps because I really try to push them. And I push this one so hard around like, you know, trying to like hallucinate or talk about something I wasn't familiar with. And I have to go talk to the maker. They're actually pinging them on Twitter. They do a great job of like if it's not super confident that that information is in its sort of retrieval window, it will just refuse to answer and it won't confabulate it. It won't go there. And I think that that is an answer as well, which is like the combination of model intelligence plus data plus the right, like prompting and retrieval so that you don't want it to answer unless there actually is something grounded in the context window helps tremendously on that hallucination front. Does it cure it? Probably not, but I would say that like all of us, all of us make mistakes. Hopefully they're like predictably”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“LMs are very smart. So are humans. I still use calculators all the time. In fact, over time, I feel like I get worse at mental math than rely on those even more. So I think there's a lot of value to, hey, give it tools, teach it to use tools, which is a lot of what the research team focuses on. And then really emphasize the time where like, yeah, I know you think you can do this. The joke I do is like the CSV version is like, yeah, I can eyeball a column of numbers and give you like my average. It's probably not going to be perfectly right. So I'd rather like use the average function. So that's on the data front. On the citations front,”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“I think we have a really good shot there. The two places that most recently this came up one was LLMs will oftentimes try to do bath. Sometimes they actually are, especially given the architecture, impressively good at math, but not always, and especially not when it comes to higher order things or even things like counting letters and words. I think you could eventually get there. And so one tweak we've made recently is just helping Claude, at least on Claude AI, recognize when it is more in that situation and explain its shortcomings is it perfect? No, but it's like it significantly improved that particular thing because from an enterprise, then this came directly from an enterprise customer that said, hey, I was trying to do some CSV parsing. I'd rather you give me the Python to go analyze the CSV than try to do it yourself because I don't trust that you're going to do it right yourself. So I think on data analysis, code interpretation that front, I think it's a combination of having the tools available.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Search GPT. So it sounds like right now the focus on work, right? You described a lot of work products that you're thinking about, maybe not so much on consumer. I would say the danger in the enterprise is it's bad if your enterprise software is hallucinating just broadly. It seems risky. It seems like those folks might be more inclined to sue you if you send some business haywire because the software is hallucinating. Is this something you can solve? I've had a lot of people tell me that LMs are always hallucinating and we're just controlling the hallucinations and I should I should stop asking people if they can stop hallucinating because the question doesn't make any sense. Is that how you're thinking about it? Can you control it so that you can build reliable enterprise products?”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“It's not on my mind for any kind of near term thing. I'm very curious to see, I haven't gotten access to it probably for good reasons, although I know Kevin all pretty well. I should just call him. So I haven't gotten to play with it. But like that space of the perplexities search chat GPT search, I forgot how they actually branded. Search GPT. Yeah, I mean, it ties back to the very beginning of our conversation, which is like search engines in the world of summarization and citations, but, you know, probably fewer clicks. And where does that end up? How does that all tie together and connect? And it's less core, I would say, to what we're trying to do.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“I mean, I opened up the App Store and ChatGPT is regularly second. I don't know what their numbers look like in terms of that business, but I think it's like pretty healthy right now. But long term, I think it's, yeah, I actually optimistically believe yes because I think that even on a like, let's conflate mobile and consumer for a second, which is not a super fair conflation, but I'm going to go with it for a second, which is so much our lives still happened there that whether it's within LLM plus Rex recommendations or LLM plus shopping or LLMs plus even dating like I have to believe that at least a heavy AI component can be in a $7 billion plus business, but not one where you're trying to effectively be like Siri++. I think that's a hard place to be.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Is where you can differentiate either in a cross platform way, either in a depth of experience way, either in a like novel take on how work gets done way, or be willing to do the kind of work that some companies are less excited to do because maybe at the beginning they don't seem super scalable.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Integrated into the desktop. I think as an independent company trying to be that first call, that Siri, I've heard the pitch from startups even before I joined here, like, we're going to do that. We're going to be so much better. And the new action button means that you can bring it up and then press up. I'm like, no, like the default really, really matters there. Like Instagram never tried to replace the camera. We just tried to make a really good thing about what you could do once you decided that you wanted to do something novel with that photo. And then sure, people took photos in there, but by the end when we left, it was like 85% library 50% camera, right? Like there's real value to like the thing that just requires the one click. So it was interesting because, you know, every WWDC that would come around pre-Instagram, I loved watching those announcements. I'm like, well, what are they going to announce? And then like it changes like, oh, what are they going to announce? And then you get to the point where we realize like they're going to be really good at some things. Like Google's going to be great at some things. Apple is going to be great at some things. You have to find the place.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I love the question. I get asked this all the time, even internally what should we be pushing harder into an on-device experience? And I agree, it's going to be hard to supersede the built-in model provider there. Even if our model might be better at particular use case, there's like a utility thing. I get more excited about can we be better at being close to your work and like work products have a much better history than the built-in sort of thing like pages comes with and plenty of people do their work on pages i hear i don't know but like you know there's still real value for a google docs or even a notion and other people that like can go deep on a particular like sort of take on that sort of productivity piece so i think it's why i lean us heavier more into help people get things done and some of that will be mobile but almost maybe as a companion and uh provide and deliver value that is almost like independent of needing to be exactly in”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Deployed clods. And this is early kind of product thinking, but it's things I get excited about being able to think about like what value are you delivering and like really align over time is the way we're like, I think it just creates a very sort of full alignment of incentives there in terms of delivering that product. So I think that's an area we can get to over time.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Data dog is another one like usage based billing is like you know the new hotness if we had like subscription billing now we have like usage based billing and like the thing I would like to get us to it's hard to quantify today although maybe maybe we'll get there is like a real value based billing like what did you actually accomplish with this and you know there's people that will ping us because like a common complaint I hear is that people hit our rate limits and like I want more Claude I saw somebody who like oh I have two clauses I have like two different browser windows I'm like God we got to do a better job here but the reason they're willing to do that they write in and they say like look I'm like working on a brief for client they are paying me X you know amount of money like I would happily pay another hundred dollars to get me to finish the thing so I can deliver it on time and move on to the next one that to me is like an early sign of like where we fit where we can provide value that is like even beyond you know a $20 subscription but when I think about like”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“This debate with somebody around the price point for that is much higher if you're delivering that kind of value. But I was debating with somebody around what Snowflake and Databricks and those have shown like.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Remediation of issues. It's all the stuff that you actually do in training a good person to be good at their jobs. That to me Even within a particular discipline, like some coding tasks, some particular tasks that involve coalescence of information or researching, like each of those, getting to like have the incremental person on your team, even if they're not, in this case, I'm okay with not net plus one productive, but net 0.25, but maybe there's a few of them and coordinated. I think that I get very excited about the economic potential for that. And that's”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“To me, it's like right now Claude is an assistant and a helpful kind of sidekick is a word I heard it internally at some point. She's like, at what point is it a co-worker? Because like the joint amount of work that can happen even in a growing economy with assistance I think is very, very large. So I think a lot about, you know, we have Claude for Work. Claude for Work right now is a sort of almost a tool for thought. You can put in documents. You can sync things and have conversations and people find value. Somebody built a small fission reactor or something that was on Twitter and I was like not using Claude, but Claude was there, you know, there's their tool for thought. So the point where like it is now an entity that you actually trust to execute autonomous work within the company. That delivered product. It sounds like a fanciful idea. I actually think the delivery of that product is way less sexy than people think. It's about permission management. It's about like identity. It's about coordination. It's about”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“I guess an interesting way of asking the way I think about it is the LLMs today deliver value, but they also deliver our ability or help our ability to go build a thing that delivers that value.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Doing it, like that trade off is at different points on the AI curve, and I think that would be the bet is can we shorten that time to value so that you can trust it to do more of those things. Probably nobody really gets excited to”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Or, like a particular eval, and to me, it's like again, what problem are you solving? And right now it's like I joke with our team, it's like right now talking to Claude is like a very intelligent amnesiac. It's like every time you start a new conversation, it's like, wait, who are you again? Like, what am I here for? Like, what do we work on before? And it's like instead, it's like, all right, like, can we carry continuity? Can we have it be able to plan and execute on longer horizons? And can you start trusting it to get some more things in? Because there's things I do every day that I'm like, I spend an hour on, you know, some stuff that I really wish I didn't have to do. And it's not like particularly leverage use of my time. But I don't think Claude could quite do it right now without a lot of scaffolding. And right now, here's maybe like a more succinct way to put a bonnet. Like right now, the scaffolding needed to get it to execute more complex tasks doesn't always feel worth the trade-offs because you probably could have done it yourself. I think there's an excd comic on like time spent automating something versus time that you actually get to save.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Embedded in the current models, I would agree with you that the big open questions to me, I think it's basically for longer horizon tasks, what is the sort of horizon of independence that you can and are willing to give the model? Like the metaphor I've been using is right now LLM chat is very much you've got to do the back and forth because you have to correct, you know, you got to iterate, no, that's not quite what I meant. I meant this. A good litmus test for me is like, when can I email Claude and generally expect that an hour later, it's not going to give me the answer it would have given me in the chat, which would have been a failure. But like it would have done more interesting things and gone find out things and iterate on them and even self-critiqued and then responded. And like that, I don't think we're that far for some domains. I think we're far from some other ones, especially ones that involve sort of like either longer range planning or thinking or research. But I use that as sort of my capabilities piece. It's like less like, you know, parameter size.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Think current generation, yes, in some areas, no in others. I think part of, I think maybe what makes me an interesting product person here is that I really believe in our researches, but of like default belief is everything takes longer in life and in general and in research and in engineering than we think it does. I like do this mental sort of exercise of the team, which is if our research team got Rip Van Winkle, all fell asleep for like five years, I still think we'd have five years of product roadmap and we'd be like, we are bad at our jobs. We're terrible at our jobs if we can't think of all the things. Even in our current models could do in terms of improving work, accelerating coding, making things easier, coordinating work, even intermediating disputes between people, which I think is a funny LLM use case that we've even seen play out internally around like these two people have this belief, like help us even ask each other the right questions to get us to that place, such as a good sounding board as well. Like, there's a lot in there that is...”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Doors is accepting to keep walking back and forth between them, so you have to kind of like walk through the door and say the earliest I would be willing to go back the other way is, you know, two months from now, or with this particular piece of information. And hopefully that kind of quiets the like even internal critic of like, oh, it's a two-way door. I'm always going to want to go back there.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“The trade off was not getting too much into the technical details, it's basically of the next generation of models, like what particular optimizations we're making. And can't sure exactly what, but like it will make one thing really good, another thing just like okay or pretty good. And like the thing that's really good, I think is a big bet and it's going to be really exciting. And everybody's like, yeah. And they're like, but, you know, and yeah, but so I'm actually having us write a little mini document that we can all sign. I know this sounds kind of cheesy where it's like, we are making this trade-off. This is the implication. This is how we'll know we're right or wrong. And here's how we're going to revisit this decision. And I want us all to like at least cite it in Google Docs and be like, this is like our joint commitment to this or else you end up with the like next week of like, but, you know, it's that revisit. So it's like, it's not even disagree and commit. It's like fill the pain, understand it. Don't go blindly into it forever. Like I'm a big believer when it comes to like hard decisions, even decisions that can feel like two A doors with two.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Strategy frameworks. The one that's resonated the most with me consistently is playing a win. I go back to that often and I've instilled some of that here as well as we start thinking about, you know, like, what's the winning aspiration? Where are we going after? And then more specifically, and we touched upon this in our conversation today, like where will we play? Because we're not the biggest team by size. We're not the biggest chat UI by usage. We're not the biggest AI model by usage either. We've got a lot of interesting players in the space. We have to be thoughtful about where we play and where we invest. So, yeah. And then, and this morning I had a meeting where the first 30 minutes were people being in pain due to a strategy. And the cliche is like, strategy should be painful. And people forget the second part of that is that then you will feel pain when the strategy creates some trade-offs, but also just recognizing that in Instagram, we always talked about doing fewer things better. That was like a foundational company value. Wait, what was the?”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, we run a doc about that like two months ago. I'm like, well, did we do anything about it? And so my whole decision making pieces, I want us to get to truth faster. Like, none of us individually know what's right. And getting to truth could be, let's de-risk the technical side by building a technical prototype. If it's on the product side, like, let's get it into somebody's hands. Like Figma mockups are great, but like, how is it going to move on the screen? And so minimizing time to iteration and time to hypothesis testing is my fundamental decision making philosophy. I'm trying to instill more of that here on the product side. Again, it's a thoughtful, very deliberate culture. I don't want to lose most of that. But I do want there to be sort of more of this hypothesis testing and validation components. And I think people feel it when they're like, oh, yeah, we had been debating this for a while, but like we actually built it. And it turns out neither of us were right. And actually, there's a third direction that's more correct. And Instagram, we went through sort of, we ran the gamut of...”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Actually, maybe I'll go meta for a quick second, which is the culture you're anthropic is extremely thoughtful and very document writing oriented. So if a decision needs to be made, there's usually a document behind it. There's pros and cons to that. It means that as I joined and I was wondering, like, why did we choose to do this? People would be like, oh, yeah, there's a doc for that, you know, and there's literally a doc for everything. And then, which helped my ramp up. But sometimes I'd be like, why have we still not built this? They're like, oh yeah.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Coach I was working with, I would always push him like, well, I want to start another company that has as much impact as Instagram. He's like, well, there's no cosmic ledger where you'll know exactly what impact you have, first of all. And second of all, what's the equation by positive or negative? And I think the right way to approach these questions is with humility and then an understanding as things develop. To me, it was I am excited and overall very optimistic about AI and the potential for AI if I'm going to be actively working on it. I wanted to be somewhere where the risks and the sort of mitigations were as important and as foundational to the founding story, maybe to bring it back to like why I joined. That's how I balanced it for myself, which is like you need to have that internal run loop of”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Instagram filters are kind of an interesting comparison here. Instagram starters photo sharing, Silicon Valley nerds, and it became this, it became Instagram. It is a dominant part of our culture. And the filters had real effects on people's self-image, had real negative effects, particularly on teenage girls and how they felt about themselves. There's some studies that say teenage boys are starting to have self-image issues and body image issues at higher and higher rates because of what they perceive on Instagram. That's bad, right? And it's bad weighed against the general good of Instagram, which like many more people get to express themselves, we build different kinds of communities. How are you thinking about those risks with anthropics products?”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Also, open area of research and a very interesting one as well. And then the last one is if you're meta, if you're Google, maybe the bull case is that if primarily you're servicing content that is generated by models that you yourself are building, there is probably a better closed loop that you can have there. I don't know if that's going to play out or whether people will always just flock to whatever the most interesting image generation model and, you know, create it and go publish it and blow that up. I'm not sure. Well, I think that jury is still out on that one. But I would believe that, you know, the built-in tools like Instagram, 90 plus percent of photos that were filtered were filtered inside the app because it's just the most convenient thing. And in that way, a closed ecosystem could be one route to at least having some verifiability of generated content.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, there's on the research side, now outside my area of expertise, like active work on what are the techniques that could make it more detectable? Is it watermarking? Is it probability, et cetera? And I think the open question, but also open very active area of research as well. I think the other pieces, well, actually, I'll break down at three. There's like what we can do from the detection and watermarking, et cetera, side. On the model piece, also have it be able to express some uncertainty a little bit better. I actually don't know about this. I'm not willing to speculate or I'm not actually willing to help you filter these things in because I'm not sure. I can't tell which of these things are true.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Now, from your seated anthropic, knowing how the other side works. Is there anything you're doing to make the filtering easier? Is there anything you're doing to make it more semantic, make it more understandable what you're looking at to make it so that the systems that sort the content have an easier job of understanding what's real and what's fake.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“I think an interesting question And I don't know what the current Josk Adamasari, what he would say, what percentage of Instagram content could and should be AI generate or at least AI?”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Let me bring this all the way back around. We started talking about recommendation algorithms, and now we're talking about classifiers and having filters on social media to help you see stuff. You're on one side of it now, right? Claude just makes the things and you try not to make bad things. The other companies, Google and Meta, are on both sides of the equation. We're racing forward with Gemini. We're racing forward with Lama. And then we have to make the filtering systems on the other side to keep the bad stuff out. And it feels like those companies are at decided cross purposes with themselves.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Chaos or proliferation piece of our abilities to both learn, adapt, and then like grow the right mechanisms in place. So I remain optimistic that we'll continue to figure it out on that front. The AI component, I think, increases the volume and the thing you would have to believe is that it could also increase some of the parsing. I'm going to say it was a William, no, it was a Neil Stevens novel that came out a few years ago. It was one of those two of them had a concept of in the future perhaps you'll have a social media editor of your own and that gets deployed as a sort of gating function between all the stuff that's out there and what you end up consuming. There's some appeal to that to me, which is, you know, if there's a massive amount of data to consume, probably not most of it is going to be useful to you. And I've even tried to scale back my own information diet. And to the extent that there are things that are interesting, you know, I'd love the idea of like.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I think the information piece, like living in a society with this amount of internal, without AI already, I think there's already just take textual, primarily textual social media. I think some of that happens on Instagram as well, but it's easier to disseminate it when it's just a piece of text that you can has already been like a journey, I would say, in the last 10 years. But I think it comes and goes. I think we go through waves of like, oh, man, this is like, how are we ever going to get to truth? And then good truth tellers emerge and I think people flock to them. And I think some of them are traditional sources of authority and some of it are just people that have become trusted and then we can get it to separate conversation on verification and validation of identity. But I think that's an interesting one as well. But I think I'm an optimistic person if you can at heart if you can't tell. And I think that part of it is my belief from an information sort of, you know”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“We need to have a conversation with them. What are they actually doing? What is the use case? I think it's important to be clear as a company. Like what you consider bugs versus features, you know, and like it would be an awful outcome if anthropic models were being used for any kind of like coordination of fake news and election interference type things. And so we've got the TNS teams actively working on that. And to the extent that we find anything, like that'll be a combo additional model parameters plus trust and safe for you to shut it down.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Kamala with a machine gun. It was like it was, you know, crazy stuff. I go between believing that actually having examples like that in the world are actually helpful and almost like inoculating what you take for granted as a photograph or not, you know, or even a video or not. I don't think we're far from that as well. And getting, you know, maybe it's calling the Denver Post or like a trusted source or maybe it's like creating some hierarchy of trust that we can go after, you know, I don't, there's no easy answers there as well. But like that's, I would say like a industry almost like grand you know, society-wide thing that we're going to reckon with as well and like the image and video pieces. And then on text, I think like what changes with AI is like the mass production. So one thing that we look at is any type of coordinated effort. We looked at this as well on Instagram. Like individual levels, it might be hard to catch the one person that's like commenting on a, you know,”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I would like maybe split internal anthropic and just what I was just seeing out in the world. The Grok image generation stuff that came out like two weeks ago was fascinating because it was almost like, you know, because I think there was a maybe they've introduced some, but at launch, it felt like there was almost, it was a total free-for-all. It's like, do you want to see?”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“I think, you know, I'm very worried about AI generated fakery across the internet. This morning I was looking at like a Denver Post article about a fake news story about a murder that people were calling the Denver Post to find out why they hadn't reported on it, which is in its own way the correct outcome, right? Like they heard a fake story, they called a trusted source. At the same time, the Denver post had to go run down this like fake murder true crime story because an AI had just generated it and put it on YouTube. It all seems very dangerous to me. There's the death of the photograph. We talk about it all the time. Like, are we going to believe what we see anymore? Where do you sit on that? Anthropics is obviously very safety-minded, but we are still generating content that can go haywire in all kinds of ways.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“The Swiss cheese method, which is like no one layer will catch everything, but ideally enough layer stack will like catch a lot of it before it reaches the end.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Wait, like people have usernames? How do they get reported? And it was like ended up that we delayed that launch for a week and a half to make sure we had the right just TNS or trust and safety pieces around moderation, reporting, cues around taking it down, like limited distribution, figuring out what it means for the people on teams plans versus individuals. It's one of those things where I got very excited. Let's ship this, like sharing artifacts and a week later, like, okay, now we can ship it. We got to like actually sort these things out. So that's on the content moderation side, I would say. And then on the response side as well, we also have additional pieces that sit there that are either around preventing the model from reproducing copyrighted content. It's like that's something that we want to prevent as well from the completions. And then other harms that are against the way we think the model should behave and ideally have been caught earlier, but if they aren't, then they can get caught at that last mile. So it's like our head of trust and safety was talking to him last week.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“That we give to the model around how it should act. Of course, ideally it gets baked in earlier. People can always find ways to try to get around it, but we're fairly good at preventing jailbreaks. And then the last piece is where our trust and safety team sits. And that's, you know, the trust and safety team is the closest team. At Instagram, we called it at one point trust and safety, another point well-being. But that same kind of like last mile remediation piece. And I would kind of bucket that work into two pieces. One is what are people doing with Claude and publishing out to the world? So with artifacts, it was like the first product we had that any amount of social thing at all, which is like you could create an artifact, hit share, and actually put that on the web. And that's like a very common problem and kind of like shared content. Like I lived shared content for almost 10 years at Instagram. And here it was like,”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“I would broadly put it in three places. One is in the actual model training and fine tuning where part of what we do on the reinforcement learning side is saying we'd like to find a constitution for how we think Claude should be in the world. And that gets baked into the model itself. Like early, like before you hit the system prompt, before people are interacting with it, that's getting encoded into that around like, how should it behave? Should it be willing to answer and be willing to chime in on where should it not be? And that's, I think, very linked to the responsible scaling piece. Then next is in the actual system prompt. So we actually in the spirit of transparency just started publishing our system prompts. People would always figure out like clever ways to try to reverse them anyway. And we're like, that's going to happen. Why don't we just actually treat it like a change log? So just be transparent. So it's the thing this last week you can go online and you actually see where we've changed. That's another place where there's like additional kind of guidance.”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“Yeah. Tony Fidel, who did the iPod, he's been on Dakota before, but when we were starting the merge, he was basically like, look, you're going to go from, I don't remember what the actual numbers were, but he said something like, you're going to go from like 15 or 20 people to 50 or 100, and then nothing will ever be the same. And I've thought about that every day since because we're always right in the middle of that range. And I'm like, when is the tipping point? Where does moderation live in the structure? You mentioned safety on the model side, but you're out in the market building products. You've got what sounds like a very horny golden great bridge people can talk to. Where in your running test there? Sorry, that's just my conversation has one joke about how horny the models are. Where does moderation live, right? It Instagram, there's the big centralized meta trust and safety function at YouTube. It's in the product org under Neil Mohan there. Where does it live for you?”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT
“That's depressing. And like, just like, I think it creates all this other swirl around just team culture dilution, et cetera. So that's like something I'm personally passionate about. I was like, how do we take what we know about how these models work and actually make it so the team can stay smaller and more titanate?”
2024-09-09 · Decoder with Nilay Patel · Anthropic’s Mike Krieger wants to build AI products that are worth the hype · IDENTIFIED FROM THE TRANSCRIPT