YouSaid · the spoken record

Jackson Shuttleworth

lines on the record: 103
first: 2024-12-15
most recent: 2024-12-15
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Popular feature. It's just been hard to keep up on. So, you know, if you have interesting streak insights to send my way, I would love to hear them.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, you can find me on LinkedIn. That is where most of my online social media, so Jackson Shuttleworth. And then how people can be useful to me, yes, as you said, we are hiring. We're actually hiring for my team. Are you interested in thinking about streaks as much as we do? You know, we might be the right home for you. So please, you can apply on our website. We're also hiring for a number of other product management roles and they're all as thrilling as this work is. And then, you know, I'm always interested about how other companies have implemented streaks and what they've learned. And so what I'd say is if you're a company who's implemented a streak maybe in a different way than Duolingo has or you found a whole ton of success and another vector, another element of the feature that we didn't talk about today, I would love to know more. I used to catalog basically every streak I found out there. And as it's become more of a”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“I will say it took a while to get people behind shouting yeehaw. But now that we have people doing it, you can't take it away from it. We all love it.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Have way too many traditions to count. I will share the weird tradition that we do at every retention stand up and this started during the pandemic. We obviously used to stand up in person and then when we went remote, we did this thing where whoever's the last person to go would count down three, two, one. And then we'd all try to clap at the same time, which was kind of fun and dorky, but we fell in love with it. And four years later, we're still doing it. Recently, we've added We all say yee-ha in unison afterwards. I can't tell you why, but trying to synchronize a clap via Zoom and then all shouting yeehaw, I did this in a phone booth the other day. And after I came out, someone told me, you know that those aren't as soundproof as you think, but you know, when you get a good opportunity to give a yeehaw, you kind of can't pass up on it.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“This probably will not be much of a surprise based on how I've talked about our willingness to test things, but you miss 100% of the shots that you don't take. I'm a big fan of just trying things, even if your possibility of success is not 100 because you learn a lot among the way.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Last week I went to Home Depot and I bought a new ladder and ladder innovations, you don't think of often. But you can make one of the legs go a little bit further than the other leg. And as somebody like myself who has a house that is built on a slight slope, every time I go up on my ladder, I take my life in my hands. But with this ladder, I'm always even, I cleaned my gutters twice last week just because of how awesome this ladder, how much this ladder has changed my life. ladder innovation. I don't think it gets talked enough about. And so I'm happy to give it the spotlight it deserves.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“So I have two kids. I watch a lot of Bluey. It's really good. I swear it brings me no shortage of joy. But adult show that I, or show not meant for four-year-olds that I have watched, I just finished the latest season of Emily in Paris. Man, wonderful. I realize it's not the highest brow of television. Beautiful people in beautiful cities solving problems that are not earth shattering. Sometimes it is nice to just tune out. Also, I'm learning French on Duolingo, slight plug for the app. I can understand a lot of the French that is being spoken. And there is no better joy than having invested as much time as I have in French and actually being able to use it. So huge fan of Emily.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“My wife is German, and I gave it to her understanding. It's a German Very different. Another book, this is a good one, Fate is the Hunter. So this is a really cool book. It's a memoir of one of the early commercial airline pilots. And it is wild to hear the stories about what flying was back in the day. I'm a former management consultant. I flew every week for almost six years. And I never once had to worry about, am I going to make it to the other end of this flight alive? That was not the case back then. And so some of the stories about what it used to be like to be a pilot on some of these planes before modern aviation technology is fascinating. It makes you really appreciate what we have.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“All right, I'll start with a guide to Midwest conversation. So I'm based in Kansas City. I'm a proud Midwesterner. And us Midwesterners talk in a certain way. I think you hear about Minnesota Nice, but we tend not to say what we mean. And it is a very funny primer into what Midwesterners actually mean when they say what they say. So highly recommend reading that.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“That's a hard. Yeah, that is a hard use case. But the vast majority of companies, I think, have a good idea of like, all right, here is my ideal use case. I want users to come here three times a month. That would be ideal, or four times a month. You can build a street to work. I mean, Peloton has weekly streaks because the idea of doing a Peloton, you know, workout every single day is hard for a lot of this user during COVID. It's just like every now and then you get on the Peloton. That was great. But the idea of a weekly streak was something that I could keep up. And so I think figure out what your usage pattern is is a user and then build your streak around it. But as long as you're not like a really, again, the tax example is probably a good counterfactual, but as long as you have some degree of frequency in your use, I think almost anything can have a streak.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“I am well known for saying in the company that I think every team, every app could benefit from a streak. Now, how you implement it is very different. And I think you got to like, what is your user's use case? Like if they're going to come use, I don't know tax software. Okay, you know, not that I say this, tax software would be a hard one, but maybe it's all about, you know, you need users to come back every day during the tax season or how many times, I don't know, you know, now that I say this out loud.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Maybe final thing is, look, we ran so many tests on our Duolingo streak to figure out what worked. I mean, we have a philosophy of Duelingo if tested first. We are a lot of times willing to test things. I really think that if you're going to try to introduce a streak or you want to improve on the streak you have, don't get too caught up in the philosophy of everything. I mean, make sure your hypotheses feel like they're good. But my recommendation is just try things. And this is, again, you said it earlier. It's like, this is most human psychology is anything. And as soon as that becomes the case, like you kind of just got to understand what users respond to. And the easiest way to do that is to stop spending time batting around ideas in a conference room and just try some stuff. So huge recommendation to like, if you're going to invest in a streak, try and figure out what works through testing with you.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Seconded off into the corner of your app where users aren't going to see it. They're probably not going to care about it as much, which might be fine because there might be other lovers that you think are more important to pull on. But there's a reason why we focus as much on the streak as we do. And that's because we want it to be top of mind for users. And that's not by accident then that users start to care about. And so I think just as you're thinking about building the streak, making sure that you're giving it the visibility, it deserves if you want it to have the kind of impact the Duelingo has sort of an important hierarchy principle to think about as you design things.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“No, and honestly, it'll probably cause more problems if what you end up focusing on is how do I make the street highly engaging, but your app is not. I mean, you're just wasting time that could otherwise be better spent on solving more critical problems. So that's one learning. The other thing that I'll say is we met with one of our board members, Bing Gordon a few weeks or a few months ago rather, and he had this comment where he was just like The reason why users care about your streak so much is because you care about your streaks much, you being Duolingo. Like the reason why users care about our streak so much because Duolingo cares about the streaks so much. We're like, what do you mean? Well, he's like, well, after every session, you see a big streak screen and it's animated cooler than almost any other screen in the app. And then sometimes you see some other screens and there's all these other feet like you don't let a user forget it. You talk about them and messages. And so I think it's worth thinking about, look, if you're going to build a streak and then you're going to.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I really do think it starts with, you know, streaks are an engagement hack. You can make your app more retentive. I'm almost positive. Almost every app out there can make it more retentive. It is loss aversion. That is, you know, again, armchair psychologist Jackson. It's just a human thing that works on humans. But if your app is not something that users want to use every day or whatever, cadence you want your app to be, you know, to work on it's going to be, you're only going to get so much from that. That really sets the stage for something to layer a streak on. So resist the temptation if you haven't, you know, if you don't think you've reached that point to go too hard down the path of streak.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Strategy. I do think it's really important to not just do a bunch of random stuff, but do it with intent, with a goal in mind. Otherwise, you do end up in these local maximas. But yeah, I mean, there's still a bunch of stuff that we haven't tried that, you know, I think we have high confidence in working out. And so we'll keep doing that.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“When I joined Duolingo, the PM that I took over for Anton, who used to lead the retention team, I remember saying, dude, the streak, like it just counts up. You guys have been tested on it for years. How much more work can we do on the streak? And he was like, Jackson, you child. You didn't say exactly this, but this is how I felt it. Like, Jackson, you child. We're not even 30% of the way optimized. And four years later, I say that with like such conviction. We are so far away from, I mean, we've made a ton of strides, but we are still so far away. And every quarter where we ship a ton of, you know, wins and, you know, improvements to the streak, it just continues to prove to me that like there is so much more to be done. So I think you're framing of it, and I would say there's a lot of thought that goes into, you know, again, I talk about the strength of the hypothesis that you have, you know, you have to have as you start to build out larger features.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“There's a lot of times where it's like, cool, then it won because all the whistles you added, not because of what your core hypothesis was. And a lot of times if you can just really simplify what the feature is, it's also much easier to ship. It's easier to design. You're not designing for a whole system. You're designing for something much simpler. And so getting everybody to think that way ends up allows us to end up shipping faster, shipping simpler, designing faster, getting faster approval, getting insight, and then doing what I talked about with Street Goal, being able to run iteration after iteration after iteration, add these things iteratively. And then not only by doing this are you able to move faster, but you get confidence at each step of the way that, hey, my series of hypotheses is actually borne out. Or if it's not, cool, well, then we're going to drop that part of the feature and then just ship what actually matters.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Another thing that I'll just say is we really resist the urge to do the big V1. And I think this is, you know, I shared the street goal example where a lot of times when we're exploring something, we will say, okay, well, that's cool. How do we strip away a bunch of stuff and figure out what our core hypothesis is and then just ship that thing first as a V1 because it's easy, and I found this time and time again. It's easy to add things to features that make them win. Like I work in retention engagement long enough. I can like add, I know enough things to pull and bells to add and whistles and, you know, to make something win.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“You can do a lot of it in Jira. There is a non zero amount of Google Docs that we have that sometimes does things a little bit. I don't know. Sometimes it just looks a little bit nicer. It's a little bit more flexible. But JIRA is our. It is where the mother load of process is.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“All the way down to really detailed roadmaps around all right, this experiment is based on the results of this experiment or my hook into an element of this feature. How do we make sure that we're lining up implementation on this so that as soon as this thing runs and we're ready to go, we can start rolling out the next one. You know, I hate features just sitting around and us not again continuing that thread. So it's not just thinking about what's our engineering bandwidth, but also what's the design bandwidth to make sure that we have the next iteration of this feature ready to go. We're planning months out as we think about these feature iterations, even small ones, you know, feature iterations, because again, you just lose when you lose cycles, you know, not pushing on a feature. It's just sort of lost opportunity. And so everything from managing thoughtful about engineering roadmaps to design roadmaps to product roadmaps, all of that needs to come together in a system.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Again, my team lead runs is, Antonio runs like the most really process. If you're going to run this mini experience, you have to be really process-oriented and really thoughtful about which experiments am I going to run when? How is that going to set up the next one? There's heavy JIRA automation. I think sometimes the Atlantean Suite makes my eyes bleed, but there's a lot of times where that degree of process helps the team unblock engineers and make them move really fast. And so making sure that you have really good process around how are you going to run so many experiments, it's worth investing in.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“If other teams want to come and mess with it, okay, we got to just figure out how that is going to work with what our plans for the streak were, how do we make sure a lot of times when teams are coming in thinking, hey, let's do this to the streak, they're like in context that we might have. And so there's as much of like a much simpler version of what we're doing now, a bit of a knowledge sharing of saying, all right, well, this is what we think about the streak. This is what we've seen work hasn't worked. How does that influence some of the hypothesis that you have? And so I think getting that really, making sure the juice is worth the squeeze.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“If you're, again, this is where I say there's soft ownership, we're not against teams doing things to the streak, but if we're going to do something, given we probably have multiple quarters worth of a roadmap around the street.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Retention owns streak, I guess, but that's only because we've seen Streak drive cur better than any feature. But we are not, I mean, we have this IAP hook with our street freeze purchases. There are other teams that work on that can and have worked on the streak because it's not ours to say, no, no, we do all the iterations here. We just know that it drives our metric better in the same way that like leaderboards, we have a team that focuses on how much time you spent. We want users to spend more time on Duolingo so they're learning more. Leaderboards are the best vector for doing it. So that team does a lot of leaderboards work, but every now and then I have an idea that I think will be highly retentive and I will go in and I will pitch to them and then we'll do some change to the leaderboard to make it more retentive. And so, but I do think having that clear metric of we're trying to drive Kerr, not we're trying to just make this feature better helps at least make sure give the”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, maybe just a little bit of context. Duolingo cares a lot about metric. So most of our teams are metric-based teams. So we do the most work with Streak, but the metric, what we really care about at the end of the day is Kerr and DAUs because we see that DAUs hit Kerr. And so when you can be really laser focused on my goal each quarter is to make this metric go up. I think it's much easier to make sure that you're working on the highest ROI thing. I think when you think more about like, oh, I want to make this feature better. I think it's easier to get lost in what better means and how you think about better. And so I do think that like having a really strong degree of focus as a team on what is the metric that I'm caring about and how is that directing my efforts is.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, and it's funny, you talked about the danger of feature bloat. We sort of talked about the danger of feature bloat. This is actually something I'm constantly thinking about with this. We have the streak, but then we also have the perfect streak, and we can count how many, we count how many weeks you've had a perfect streak. Well, all of a sudden we have two streak numbers that are kind of competing with each other. It's funny, we actually don't introduce the concept of a perfect streak until after you've hit seven days. And some of this is just because the cognitive load of additional streak features a lot of our like cooler streak features, you got to get on a long enough streak. And not to say we haven't tested it because we have because we tested everything, introducing these features earlier. But what we've found is that pretty universally they lose when we introduce too many things, too many concepts to users too early in the experience. It's just hard for them to manage.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“The longest time, it would be a product designer. Or it initially started as like the engineer would be like, all right, we'd cobble together haptics based on what they felt good. Then it became a product design role where they would kind of use their best judgment. We actually just recently acquired an animation studio, Hobbes in Detroit. And now they are the sort of keepers of do a lot of motion design work, haptics very close to that. And so they do a lot of that. I do remember trying to hire for a while a haptics contractor, like haptics design. And it was the saddest hiring I've ever done because it was just like, I don't know, like it was such a specific, like, I don't know. I just went through a lot of people who, you know, it's just a really tricky space of like kind of sound effects kind of motion design, sort of technical.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“All of that comes together to make you really focus on that moment. And all of that just gets users more connected to their streak. Animation in the right times works well. And it's something we've had win quite a lot.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Exactly a phone, you know, like there being a really cool haptic pattern as you extend your streak, all of this stuff wins. And it's cool because I think it wins. There's a few reasons. One is it just makes you feel good, right? You get some cool moment in your streak and we celebrate you and we celebrate you in this visual way and your phone's buzzing. It just like feels awesome. The other thing it does is it causes you to pause on that screen. And I think there's this desire as you think through a lot of, you know, as PMs think through, like, oh, how can I get users through this funnel as painlessly as possible? I talked about good and fortunate earlier. There's a lot of times where I don't, I want you to stop. I want you to stop and like land on this screen. You got to be careful not to do this for too many screens, right? But like, but the one big ones, sometimes I just want you to pause there and enjoy the moment because if I can get you to enjoy the moment more, you're going to care more about your streak and you're going to be coming back tomorrow. And so animations that are cool and that cause you to like really soak it in, haptics, you know, that feel good.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Is another thing where it's like when users care about the feature using not only animation haptics sound effects, using, and it's funny, we don't have sound effects on the streak. This is probably something we'll look at in the not too distant future, but like haptics are like something we have done a lot of testing on.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“We have very much we have if people lose their streak, there's ways to get in contact with us, but we've actually thought about that where it's like, okay, we should just like build a self-service feature.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Use a streak freeze, which again, candidly, I would love for you never to use a streak freeze. If you don't use a streak freeze, your streak will stay perfect. It's funny. We actually just, you know, we're constantly responding to bug reports about the streak. It is, I swear to God, we have that best infrastructure around this feature because it is so important. We had an employee who lost her like a four-month perfect streak. And, you know, it was a big deal for her because she did her lesson and she was crossing an international date line. There was like a bunch of stuff going on that was like, it was just kind of weird in her back end. But like people start to care about perfection as much as they do their streak. And, you know, for that person, it was a big deal when they lost their perfect streak. And so, you know, this is just an example of, like, if you're going to go after flexibility, which is good, finding a way to pull users back into perfection is a really important counterweight to have.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“I talked about streak freezes, and we've done a lot with streak freezes, but I think if you're going to make flexibility a thing, it's probably also useful thinking about how do you celebrate perfection. And so we have a feature that we have, it is the simplest thing in the world. It's called Perfect Streak. And it's just if you don't use a streak freeze for a few days, we make your streak look gold and we make your little progress bar on the calendar just look a little bit nicer. There's no reward for doing it. You don't get anything other than this nice little indication and it is awesome. It is a simple feature. It is ultra not complex. And it is really powerful, not only for getting users to, you know, as a bit of a reward, they behave, hey, you know, get to seven days without using a streak freeze. And, you know, your streak becomes perfect. But it's also a really nice indication of users who aren't using streak freezes. Here is the thing that if you, you know, if you don't.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“It's really funny, all of the stories that you hear about people extending their street. If you look around a Duolingo party where it's like 10, 11, 30, 11, 45, all the Duolingo employees that are like doing their lesson at the last minute, you always see these pictures of people in the club doing or like at a concert doing Duolingo. And yeah, because it's like. Otherwise, that you're going to use a streak freezer. God forbid you will lose your streak.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Certainly get a not get away with more, but you could be thoughtful about notification load and when to send notifications. And again, for us, this late night message that's also highly impactful, super good, is actually something that could be perceived as spammy, but a lot of our users really do somebody who it's often late at night and I work here and I'm like, oh, forgot to do my, you know, think about Duolinga all day. You know, here it is, 1115. And I still haven't done that message as really powerful.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, exactly. So, again, we send this practice reminder 23 and a half hours later. The other thing that we'll do though is we'll send it, what we call a streak saver. And this is at 10 o'clock at night if you have not extended your streak. We'll send you a message saying, you know, like it's your last chance. This is it if you don't extend your streak. And you would think that's kind of spammy. That's kind of annoying to get a notification from an app at 10 p.m. But what we found is because people care about their streak. Their streak is this good thing that they attach positive emotions to, that they don't really want to lose that notification reminding them, hey, come back and like people see this by and large as a positive notification and not a negative notification. Obviously, it serves our purposes well of getting users to come back and not lose their streaks. But again, I think if you can think about your notification strategy related to what is the feature that it's tied to? How do users perceive that?”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“And we actually moved, we used to let us set this practice reminder time. And our thinking was cool, you're going to say 7 p.m., that's when I really want to extend my streak each day. And then, you know what? I say, this is somebody with two kids. Life gets in the way. Life always gets in the way. And when you think you're going to practice, will change, your life will change, whatever. And what we realize is the best indicator of when you should practice was when did you practice the day before? We could almost certainly get more detail. We have tried a bunch of ways to have much more complex logic and would always wins is 23 and a half hours. That's so interesting.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, you mentioned notifications. I've mentioned this a few times. It's funny, you tend to think of exactly as you say, you can just always send another notification. It's going to be some win. And at some point, it'll be a bad experience, but it's tough to see that. There's actually a notification that we, so we send two notifications related to your core streak each day. The first is a practice reminder. We send it, this is actually an interesting insight, 23 and a half hours after you practice the day before.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, policies, but I think with a lot of things, it's hard, at least for the streak, it's harder to create policies for in the same way. A lot of it has to be done based on feel. And so you kind of just got to use your best judgment at times.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“It's good if you can have that. I think push notifications are also easier because there's a lot of things you can do around. All right, we'll set a budget cap for how many notifications we'll send.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“But you kind of got to hold the line at some point. And it's not clear where that line is. And once you talk about one way doors or two-way doors, there's a point where you go too far and it's a one-way door. And all of a sudden those users, those 9 million users on one year streaks don't care about their streak anymore. And that is, I don't know, again, retention PM perspective. That'd be an extinction level event for us. I don't want all of these users to stop caring about their streak. And so to have somebody who is invested in the sanctity of the streak and you know, for us it's Antonia and Luis. He's very good about this is really important just so you make sure you don't go too far.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, Streak Freeze was the other big one that we've recently demonetized or introduced a free option for is getting back a lost streak. Lose the streak. We had a feature in the day back in the day called streak repair. We'd give you your streak back. You had to pay gems. But what we found out worked way better was a feature called Earn Back. And this is basically where you would have to do a certain number of lessons as long as you came back within a window soon after losing your streak. Do a few lessons and we just give you your streak back. And it was such a retention winner. And again, what we thought about was it feels like you've earned it so much more when you've done, like you deserve to have your streak back. We haven't cheapened the streak because you've done something. And in this sense, this idea of cheapening the streak is something like from a philosophical philosophy of the streak. From a philosophical level, we wrestle with all the time of, cool, we're giving out more streak freezes. At what point do we cross the line and users start to realize their streak?”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so you can buy a streak freeze, and the way it works is you can buy gems, and then you can use those gems to buy a streak. And this is something we wrestle with. We're actively working on experiment right now that's having a small hit revenue, but it's a really nice one for attention. And I think it's actually worth thinking about from day one is you're building a streak, do you see this more as a modernization feature? And do you see this more as a retention feature? What's the role of monetization in this? What's the role of retention? I think for us it started out much more organically. And so we have a lot of like modernization hooks that, again, is the retention PM, I would love to get rid of. But again, it's sort of part of how the street works right now. And so we always have this tension of, hey, if we start to make it harder to buy streak freezes, then fewer people buy them, buy gems to buy them. And so there's this more convoluted series of impacts that happen.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Eventually, once people get on long streaks, you kind of don't want to give them as much flexibility because There's a lot of times where, yeah, users don't, and I'm like this. I've got like a 400 day streak. Note that that is a lot less time than I've been at Duolingo. I have lost and restarted streaks a lot of times in my time at Duolingo. But you start getting online streaks and you really care about this feature. You really care about your streak. And most people, as long as you're not like backpacking through, I don't know, you know, the backcountry of Utah, you know, you'll be in a place where you can get service. And so figuring out where's the, you know, who is the user that actually could use Duolingo and not conditioning them to start taking days off that they didn't otherwise need to do is important to figure out sort of where that line is for your future.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“You had to buy them with gems. That's our in-up currency, you know, because we wanted the whole idea of this to feel like it was really something you earned that, you know, there was a little bit of pain to getting that street freeze. And so we tested, though, what if we just give users when they start off their streak to streak freezes and holy smokes did that win? And it's sort of obvious now, you know, in retrospect. If you have a one or a two or a three-day streak, it's really easy just to let it die and restart. You know, again, you need to get to seven days, what we've seen in the data for it to really lock in. And so giving users more flexibility so that it's harder to lose their streak initially. And then conversely, and this is what we keep learning.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“These competing things where users might more users might be returning to a streak, but a lot of users were also just not coming back. We were training them to take more time off. So that flexibility, what's the right amount of flexibility? I mean, we've, again, this is another area we've run hundreds of experiments on what is the right number of flexibility. And we are constantly surprised here. We don't. I still don't have the answer for at every point in your street journey. How much flexibility you need? One thing that I can say with certainty though is give more flexibility when a user is starting their streak. Again, one of our biggest streak freeze experiment went. I feel like I'm constantly saying this. One of our biggest wins, but they all were really, really big. One of our biggest streak freeze wins was when you start a new streak, we give you two streak freezes. And again, it's so funny to think back. It's like, how are we not doing this to begin with? But at the time, the street freeze was kind of an overly gamified mechanic.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Because we were basically saying, hey, you can take a day off, you know, that's okay. But it was really good for, this is going to be like Alphabet Soup IWAR, an active weekly active user return rate. So basically users who had taken a day off, we were getting them to come back more at higher rates. And so it made up for our losses incur. But effectively, what this meant is that why two streak freezes work better than one was, I don't know, sometimes people just need a little bit more flexibility than one day. But one of the, again, the really interesting insight of this experiment was that three-street freezes was actually no better than two street freezes. And there were two competing things here. And I think this is important if you're going to build a street, you know, to figure out what your flexibility mechanic is. We are getting more users to return after longer times away to an intact streak. But if you start taking three days off from any habit, it's just going to be less likely that you return even four days later. And so we had.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, well, I guess for podcast listeners, we'll have to get an image somewhere. This idea of flexibility versus perfection and then regardless celebration is core to how we think about the street. Because, you know, I think for the streak for us, it's very much a bend knot break. If you're going to miss an A, I'd rather you come back having missed that day to an intact streak. But if you don't have to miss a day, I'd much rather you don't. I'd much rather you come back and use the app every day. That thing on flexibility, though, that's almost certainly been the biggest from a mechanic perspective, the biggest DAU driver. One of the earliest experience we ran was going from used to only be able to have one streak freeze and then we let you have either two or three. So we tested two different arms. It was, again, another huge DAU win. This actually is funny. It was something that, and this is, again, kind of a callback to that growth model post from Jorge. It actually was really bad for Kerr.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source
“Luise, grant me the serenity to accept the flexibility I need, the courage to reach perfection when I can, and the wisdom to celebrate regardless. That actually is like kind of our strategy with the streaks.”
2024-12-15 · Lenny's Podcast · Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team) · IDENTIFIED FROM THE TRANSCRIPT · source