YouSaid · the spoken record

Laura Schaffer

lines on the record: 98
first: 2023-03-09
most recent: 2023-03-09
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Yes, I know there's a few that don't have JDs open yet. They're that hot off the press, but I'm happy to, yeah, surface a few things there for sure, because I know that makes it easier for people to know.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Crazy world we're in. Things will get better, and I would be feel very lucky if I could help even one person land. So feel free to hit me up about that too.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, find me on LinkedIn. I don't post a lot. Yeah, I will blame my free children eventually, I promise that I will. But I'm pretty good about responding to messages. So definitely link with me there. And then, yeah, what listeners can do? I think, yeah, I mean, I'm always happy to hear feedback, suggestion, all that, but I'll just say I also know that it's a little bit crazy out there right now, especially folks working in tech. So I'm also cognizant of what I might be able to do to help all of you. I know there's a few places I advise and, you know, rapid hiring. I know of a few folks that are hiring strong growth people and product folks. So if you are interested in learning more about that, don't hesitate to hit me up. I want to make sure that I help as many people as I can in that respect because it's, you know, trying times and I'm sure you've heard it and read it, but if you're laid off, this is not about you. It's not your fault.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Is very powerful, but in this case, just that simple use of kind of connecting and threading those technologies together could really get a good picture of that without needing to engage customers. So kind of the power, I'd say, of the tip would be how you can really amplify where you get of amplitude when you use it with.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“You tell me day two and a half, but I'll say one thing that was super cool actually that someone put together on my team at Rapid, literally before I left, he put together a video of how powerful amplitude could be when linked up and integrated with other things, like in this case, hot jar and segment. There was like a amplitude report that someone had kind of created and there was something that was kind of an anomaly happening there. Like users were kind of using something in a way we didn't expect and amplitude one of the reports surfaced, but of course then we want to know like, why is that happening? And so we could take, find out what the event is and then using segment, find out what that name was and look at hot jar and actually go in and get screencasts of people doing that exact event. And from that, we were able to kind of form some really concrete hypotheses about what actually was causing it. And so, you know, obviously talking to customers.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, the be embarrassing, like I mentioned earlier, like be embarrassed by the first iteration. If you are not embarrassed, you've gone too far. That really speeds up ships and helps people celebrate the unpolished as opposed to feel embarrassed about it. So embracing that.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“That in to our sex people who don't do that are probably going to lose out or like smart AI or whatever bots. But that would be it for you, Lenny. But if you ask me in a few months after I've actually been in amplitude for a bit, I'm sure I'd give you a different answer.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Do you actually like it? Slack, it's boring. Everyone says Slack, but I just have to hand it to them. It makes life so much easier and just not. And then builder, which also put an asterisk on that one, but I really want to serve this. A lot of people don't know about it, and it's really helpful. I do advise them. So I'm like, you know, in their corner, but this is another one I also say would be a powerful one. I think a lot of teams get stuck. They're not able to, they're relying too much on their engineers to make changes. Again, we talk about rapid experimentation, getting these out, out, out. And Builder makes it really easy for folks to do that. you know CMS you can like drag and drop headless CMS so they do make it easy for non-engineers to make changes so especially if you're trying to figure out how to figure out how to get around that 80man that I mentioned this builder would be a good way and then yeah if you want one more I'll give you chat GPT which is really boring and you know everyone's saying that but I think I'll just say I don't have any like crazy things to say about except that I do think we all need to figure out how we pull”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“I know, right? Still learning which ones we have here. Yeah, I'll share, I'll just share sort of like the ones that I like a lot that I've used elsewhere. So one is Hodjar. Hodar Kualaroo also works. Just anything that allows you to put some quick little thing in front of customers, get that qualitative feedback we talked about. It's a critical, critical supplement to quantitative data to understand what's really causing the change or not causing the change that you're seeing. So that's important. I will say amplitude is a fantastic tool that I have used. And I would have said that if I weren't hadn't just joined Amplitude. I got to use it. I know. I got to use it for the first time at Rapids use Amplitude and it was awesome. So again, like asterisk, I'm like working there now, but it is.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“I love asking about a ship or release that is not cherry picked by the person you're talking to. You can get it in a lot of different ways. The things everyone has like a big success story everyone does. It really doesn't actually tell you very much to ask someone like, what's a great thing you released because everyone can tell that. Instead, take that away. What's the most recent ship is a really easy one because like recency time. But, you know, there's other things you can do to like kind of take that out, like just give specific parameters for ship that they've shared or whatever. And that will allow you to listen more and learn more about their frameworks versus the outcomes. Because if you're picking a random ship, odds are it probably wasn't like fantastic. So they're going to want to talk more about how they approach getting there. And that's what you want to know about to know if they're going to succeed, what their frameworks are they approach things.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“I know. I know. I feel like I'm really, really sinking deep in there. But it's true, though. It's very interesting to me. And I love that show.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Uneabashedly, the great British baking show. I love that show. I love that show for like all the reasons everyone loves that show. It's like heartwarming and makes you feel good, not lips you, but also because it is a competitive show. Like they're trying to be the best baker and they're out there helping each other. They're like a big family. Like I've most reality competitive TV shows that I see all of them are like cutthroat or sabotaging. So I'm just endlessly fascinated also by like the psychology of what's happening here. I want someone to do like a research paper on it. What gets the bottom of why they're all like helping each other. It's wonderful though. Wonderful to walk.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Happiness, which is atomic habits by James Clear. It is if you ever want to change something on yourself or something's not quite working for you, this guy will give you a framework to change it. Guaranteed.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm a big believer in happiness, not just like, you know, because I'm being crunchy, or you know, we should all be having it also because it helps us do our best work and we're more creative and all of this. So one is simple path to wealth by jail collins. I don't ignore the data that money is something that often gets in the way of our happiness. I know so many smart people that just have not figured out the whole like managing their finances thing and this book will cover all of your basics. It's very easy to read He's got an audiobook that he does he married himself Simple past wealth Dale Collins he's fantastic”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“That is providing, that requires developers to build, right? You must invest in self-serve experiences in order to effectively convert your audience. And you should be thinking of them as something akin your self-serve function and growth folks, someone akin to Salesforce because your developers are not going to accept sales coming in and trying to convert them at that stage.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Wanted it to, that's on them. Everyone lifts a developer when it's not working right and it cannot work right in so many ways. That's their failure. They can cost them their job. It could cost them the trust of their team. It cost them their reputation, you know. And that means that the stakes are very high for them every time that they're adopting something new. So they can't afford to take someone's word for it, especially a sales rep who might be have some other motivations, right? From their perspective, right? They can't afford to trust your content or someone's word. They must do it. They must prove it to themselves. And so that's why they for developers to be bought in, they need to do something, build something, a proof of concept at the very least, if not kind of like moving further than that. And so that means they're going to be pretty darn deep in their self-serve experience with you before they're ready to commit. And so if you're a company.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“But the thing that's most important, these are fun facts, but the thing that is, I would say, is the most important to leave with listeners here is what makes them so different? Like, why? What's the deal here? And it stems from their charter and their responsibility. So, you know, if you put ourselves in developer shoes for a minute, a developer, if a developer is required to use your product, especially if they're like the primary user, the primary builder, it's really important to recognize that they are responsible for that. If your service goes down, that's their responsibility, not just for themselves, but their team. If the pager wakes up someone because the service they bought from you goes down, that's on them. If, oh, turns out that it doesn't work with the systems that they said it was. Well, that's on them, like doesn't integrate with the data the way that everyone wants.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Built a POC launched to production all this and operate in that space for months without engaging once with sales. It was trying to reach them, right? And I ended up being the one that talked to them like first because they reached out to support because there was something about their delivery that was off there, like missing a feature. And they did not want to talk to sales. They ended up talking to me. When I was in product marketing, I was like my first exposure, these people know I want to talk to sales. And then there's another one where like a giant retail company where like the engineering team signed up with their personal email addresses so they wouldn't get bombarded by sales. It was only like later that we found out.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Trying to build it, right? They'll come up for context and steps and such when they get stuck if they're motivated. So that's one thing. And then another one is just the aversion to talking to sales. And I think everybody can, you know, hearing out clouds like, oh, yeah, well, I hate sales too. And when I sign up and get bombarded by sales, that's the worst. I totally get that. But just developers are on this whole other level. There was a Fang company who signed up for Twilio.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“That makes them really different. And some of you have kind of some interesting stories. One is developers like almost two to one do not look at your marketing website at all. They go straight to your sign of flow. So what that means is all that beautiful context you're setting and product page and pricing and all that stuff, like very often they're skipping all of it context free and going straight to your signup. And so anything that is not that, you know, anytime you make an assumption like, oh, well, they probably know this coming in to sign up. Or like, well, we don't need to clue. That's on the marketing website. Like none of that's going to apply to this group of people. They are like, they're there. They're looking to do a describe them, like the analogy I have for this group is the IKEA buyers who package comes, they're not opening up the instruction manual and reading it and then starting to go through. They're in their Tear and open the bags and starting to like pull the pieces together.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“The first is that developers are just a very different audience from any others. I've seen so many people who have come in strong done growth really well or product really well with other audiences and like, oh, I'm going to take all those learnings off pivot into serving developers. And ends up being a very steep climb because developers are so different. And let me give you like a couple just fun facts.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“The problems they have, the psyche that they have coming to your space so that you can build something that is going to effectively make them feel like, oh, you can solve my problem, you get me and show them how your product's going to do that.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and it's right, and there's always the nice thing about it is it's in some ways it does feel like you're starting fresh because you do need to kind of start with the customer again and like, what's their problem? But in a lot of ways, you know, you can carry over a lot of the same knowledge. I mean, at that point, you kind of know what's working well. Like amplitude, for example, does have a ton of knowledge around what some of the best ways are to set up reports. There's a lot of things that they have momentum going. Sort of like, where do you choose that momentum and how do you put that and curate that in front of users and make sure that they're getting the right things. There was a ton of momentum already there. It's just a little bit on harnessing it, understanding that, yeah, where are the gaps because there are going to be gaps. But anchoring in a customer problem is, I think, the way that you start any new product, any new thing that you're releasing should always think about like the customer and the pain point. So no different than when you're doing, you know, PLG for the first time or cracking into it. You need to be thinking again, starting again with the problem.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Them kind of that powerful. But the thing that I think sticks across all companies, not just amplitude, making the shift is just that when you're doing this, do not think this is a copy-paste, but like chop it for parts thing. Don't start with your product when you're building out your strategy. Start with your customers, your users, your prospects, your people who are going to be coming into your self-surflow. Make sure you're understanding how their problems differ because they do from the people that you're addressing at the sales-led side. and then make sure that you're orienting your experience and product around those people.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Early stage companies have. And so when they're coming back to their psyche, what are they caring about? What are they thinking about when they're sending up for product analytics product or something? They're looking for something that's going to help them feel reassured they're going to be able to actually get to the bond the right metrics, create the reports that show things the right way. What's the best way to show churn? There's got to be a best way. Like so many people are doing it. Guess what? There are some really good ways to do it. And there are some really successful ways to set up dashboards for the board. People have done that too. Like there's a lot of that knowledge that exists and a lot of those templates and frameworks that exist. Benchmarking. Are these numbers even good? Right. And so one of the hypotheses that I have is that, you know, if we take that perspective and we understand that that is the problem, there's a number of things that we can do to really change the way that self-serve experience works to help convert people and show them how amplitude can make.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“And really give them a really startups and give them a space to land and to grow. You have to think, what are the challenges and unique problems that they have? Because we're going to be using our product to kind of sell to them. We need to meet them where they're at with the problems that they've got. And I think one of the things that I've observed from being all these startups and advising some startups is I very rarely, I don't think I've ever come across a startup where they have like the right number of analysts for their needs. In fact, a lot of them don't have any. And so what that means is that the CEO is, you know, being an analyst to create their dashboards for the board. And the product manager is like being an analyst to figure out what the heck's going on and creating reports for their product. And that's happening all over the place is that people are in their roles and they have to be an analyst too. And I think that that's a problem that especially younger companies and”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“And the focus is a lot around the product. What are we going to do with this product? How are we going to crack it open and shift it and give it to these people, these users, these visitors? And what it's missing, I think, is, and a lot of times it's easy to miss, is that when we're doing PLG and we're shifting from sales to PLG, we need to reset. We need to recognize that, you know, again, this is sales, sales via the product. What do you do when you're what is a good sales rep do when they're engaging? They understand what the problem is of the person in the space you're talking to. So we need the same thing here. What are the unique problems of people who are coming into our self-serve space? And I think when it comes to a company like Amplitude, a lot of the folks that will be kind of looking to address via the PLG motion, there's a number of things you want to achieve there. But one of the primary things is to kind of tap into this.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“I think when it comes to amplitude, I think there's a huge opportunity here. I think the key is and that the challenge for companies that have done like the sales thing and are trying to crack into the PLG thing is really comes down to how you fundamentally are approaching that space. And again, your users and where they're at and the psyche of where they're at. I think a lot of companies will say, well, okay, like, hey, we're going to do this PLG stuff. Let's take that sales enterprise, whatever offering that we have and let's, you know, chop it up a bit and like cut excess here and like cut out this feature here. And we're going to like slap this plan out. We're going to put a price on it. And we'll maybe have like hours of debates over whether it's like $10.99 or $104 or like $75. And eventually we'll like someone will win that battle and like slap it on because you'll have any discussion.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Between the SLG and PLG. I mean, yeah, for me, they're kind of two sides of the same coin. Like, you know, growth, growth, and sales, it's all to me very thematically the same stuff. The difference is that with growth, you are selling with your product. And with sales, you're selling with person, like one-to-one. And, you know, so companies need to be employing both those forces to optimally convert their audience. We're in a world where people are expecting both. They're expecting to be sold by your product and sold at the enterprise level in large companies by human beings. It's going to listen to their specific needs and really break it off them. And, you know, if you only have one, you're going to miss stuff. So absolutely, I think you want to, those two forces together working well. And obviously there's sort of, you know, different stages, you know, things work differently in different spaces.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Made talk about the psyche of people. We talked about how developers telecom until they can be intimidating. We'll talk about like this, the non-developer, like sometimes the buyers or like the people who are instantly buying decisions. Like for them, it was like, not only was it like telco, but it was like the developer stuff was inaccessible. But they still wanted to jump in and they wanted to have that experience. And so this was a way for us to give them momentum, give them comfortable. Jeez, if I can get this running my development team, definitely do it, right? And so it was a very powerful kind of, you know, moment where we could really address the psyche of those users, get them excited about Twilio, and then give sales kind of the ability to give something powerful to those non-engineering buyers and folks are talking to.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, ended up kind of being like, we had a lot of pet names, like nicknames for it. I think probably the one that most succinctly describes it as just it ended up being kind of a create your own demo experience, which, you know, made.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Ground, but man, if we can get them to do that, if I can get them to $1 and spend, I can get them to five and then five and get them $50. Like $10,000, then I can get them $100,000. This whole long journey, like, hey, Laura, if your team could just get them off the ground, man, we can do so much. So yeah, the journey is all about, okay, what were the things that were missing and the experience we were offering? And, you know, ultimately was they couldn't write code from scratch. That was real difficult. And also we're going to stand up a server. That was difficult. We ended up iteratively experimenting a way to validate those hypotheses and what's the right way to do this. And yeah, it was great. It's called Quick Deploy on CoExchange. Anyone can go there and deploy an app without having to write code and kind of get an aha moment there with Twilio.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“We saw that that little non-developer little dude ad was kind of growing, right? We were actually the number of people in the ecosystem who were identifying themselves as not a developer where we're in the space. But very interestingly, they were, as we kind of got more refined in our understanding of those folks, a lot of them wanted to build with Twilio. There was a hypothesis of like, oh, well, maybe they're lost. Maybe they just want pricing. Like maybe they sent a bite mistake. And I was like, nope, they're here to build. They want to build, you know, and they kind of struggle through the developer onboarding and someone would succeed and someone. But anyway, it was all about identifying like what did they need to succeed? Like, were they, if we were made them successful, could it contribute to dollars? You know, one of the core learnings I'd heard from sales at the time was, hey, it's very challenging for us to get the kind of folks when a developer is not involved yet to go from zero to one to get something off their.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Amazing. Good plug for reforge. What's the course called? Just if folks want to active interest in retention, I think it's live is my part. I think it's live right now. We can all get, I'll give you a link so we can. In the show notes. But yeah, the high level version is we, you know, this was kind of deeper into my journey at Twilio. This is fast forward a few years, kind of blow up this team and some cool things going on. But I was really looking for kind of what's like the next big thing for us to do, like what could that be? And I noticed, you know, remember that question very back in the day when I asked about like the kind of developer versus not developer folks in our...”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Like tens of millions of dollars in the pipeline was really, really powerful and took some time to kind of navigate and validate. Other times we did, like that onboarding stuff that's talking about like kind of like catching those things. Like that was, you know, could happen on a little bit of a faster clip, but still, you know, took some time to validate and understand. But yeah, over the course of a year, you should generally kind of be able to commit to movement, but help people understand the methods there so that they're not coming at you on a weekly basis being like, and what did you do these past couple days?”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, you know, I have more like, you know, hey, we've got a few bets that we have or a few core hypotheses. And if they take off, that's going to be our high bucket. Like, wow, like we think these things could be kind of lightning on a bottle here. But they could also be, you know, a bunch of vapor mist. But until we run it, we're not going to know. And if those bear out though, then yeah, that's our high. And hey, we've got a few things that we think are safer, you know, maybe it was sort of validated a bit in the previous year, what have you. you know, these looking at the metric system out. So it's helpful to give people though that construct right. It deviates from it very hard, deviates from this notion of like here's the single number that we're going to hit, right? Just things that help people kind of understand that space a little bit better and what to expect. And because of that, it can be a little bit lumpy. Like, you know, there were some things that you released. I mean, truly for the most number of years, it's been the easiest to talk about this construct here. But like, you know, there's one thing that we did that generated.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it's good, especially for newer teams, but even teams in general commit to something that you can kind of do over the course of a year and low, medium, high, right, is always helpful in that space. I mean, you know, a lot of times.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“It where I was after that, and I'm sure I'll spend some time at Amplitude as well, just helping folks kind of understand what is the healthiest ecosystem, the most powerful ecosystem for a growth team to operate in. And time and expectations over time is a big part of that.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Short timeline wins and improvement is always going to be dangerous. That's an environment that's conducive to vanity metric usage and massaging the data to fit. And ones that are more successful are ones that are porting over longer periods of time. Because any growth team given enough time to kind of fail enough times to learn the right thing to do is absolutely going to show success. Real success, right? Like not that, okay, we're going to make this data fit, but like real moving the metrics success. And so definitely educating out if you find yourself in a position where you are beholden to that, you know, share that 80% fail rate. Like just math, statistics, data. That is not, you know, you cannot be successful in an environment, but over time, you can be. And so that's one thing I definitely would draw on that they spent, I end up spending a decent amount of my pie chart at Twilio and then also at”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“The same, and gosh, we got everyone really excited about this thing that we all worked in really hard. Like, oh my God, what am I going to say in the QBR, you know, the monthly or whatever it is that sort of these things, you know, the results come to light. And to this, I'd say this, that it's incredibly important for growth teams to educate out and for folks outside of growth and leading growth, especially to understand that the best way for a growth team to succeed, the only way really for them to succeed is to embrace the fact that they're there to validate, to understand what the biggest opportunities are and to go after them. And that is not something that can be done on a weekly timeline, sometimes even a monthly, depending on the space you're in and what's known and unknown. And so any growth team that's beholden.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“I think we kind of got into this one a little bit, but I just really want an exclamation point underline it, which is that notion of kind of making the data wrap to fit like a concept, you know, I think a lot of teams feel and are under a lot of pressure to show progress, right? And what did you do this month? Where do the metrics move and like, you know, and it can cause folks to feel like they have to do that where it's like, oh gosh, this everyone's got this experience where you run an experiment and you're like looking at the data, refresh, refresh, refresh. Oh my gosh, and actually perform worse, you know, or it's like.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Appreciate that. It's funny, it's like on my general life to do is to start writing some of this down. I have three children, one of whom is five months old, two and four. And so sometimes I'll start to write. And then like one of them will crawl across the keyboard and, you know, like one of them, I mean, all of them multiple times. But eventually, yeah, I'll be very happy to do that if folks would be interested. I'm always happy to do whatever I can to help folks help empower folks with knowledge to do better because none of this is like secret sauce really. It just sort of learn from experience and it's always better to learn from others' experience on your own. It's faster. So yeah, I would definitely aspire to Lenny. I think that's the best that I can say. But eventually my kids will get older. I hear this and maybe I can do so.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“About why you're doing this. Again, this is why when you ask that question, identify it's a very data driven person. But I think some of the methods that I use can sound at the service level as more like, oh, I'm going by my gut. But again, very data driven is just embracing the reality of some of the hard data that I don't think we all embrace or are even aware of sometimes about that fail rate.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“In and of itself to not ship as much as you possibly could in a year. That is a huge risk given that very high fail rate. So, you know, I wouldn't, you know, to those data scientists and I've chatted with a few of my time. What I try to explain is that that article, that data, that the 80%, like that's hard data, right, about what a detriment it can be if you are, you know, if you don't run it off experiments, you can literally if you just run 10 in a year, odds are maybe two random impact, two of a course of an entire year if you kind of take that approach. So, you know, data scientists can understand, hey, if we do this, you know, if we move this down, we can like double or triple or whatever it is, the number of experiments we can run. And overall, net, that's going to result in more successes that will overall net us to a positive place. Like you can still tell a data story to the data scientist.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“That hypothesis was true. I want to be looking at the qualitative data from the ones where people were thrown into the existing flow and ones put into the docs that one of them felt more confident and more like this was really easy to get through. And like they felt better territory and things like that. And I'd be wanting to hear from the ones who were in the other one things like, oh, I kind of got stuck on that phone number like kind of figure this out, but it just, it feels like it's I'm out of my depth. Like I would want to be looking for other things to corroborate the hard data that I'm seeing, right? And yes, it opens the door to, you know, whenever you open the door to more risk acceptance, you are going to have some false successes there. But all of these things together can overall make it more likely that you're shipping more things that are going to positively influence the customer. And again, I can't say it enough. It is a huge risk.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“And so this is another area where you could absolutely fall into that trap. Let's hover on 80. I think it's good. That Lauren person said it was cool. So I think that's fine. That will always be a trap, right? So it needs to be very deliberately thought of in advance as a way of like, hey, here's what we're going to validate this and always, always, always if you're going to accept more risk of a false success or false positive, false negative, you want to then be really thinking about how you're going to harden your validation of hypothesis. For example, let's take that one we talked about with Trillia where we kicking people out and we're sending them to that, you know, that the pilot and hot dog experiment, right? And we're sending people to that experience to kind of hide the phone number. In that case, right, let's say that we were going to accept a lower confidence interval. I would very much want to see qualitative feedback to confirm.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“I'll say this is actually very critically important. You must have this game plan set before you run something. Failure mode that I've seen so many teams fall into is they'll run the experiment or whatever it is and then they'll make the data fit Hypothesis, right? Or sometimes alone with aldehypes and just be like, this is going to do better things for our metrics, but not like a core reason is to why or what, you know, what exactly is”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Lower confidence intervals and 95%, especially if that doubles the amount of experience that you can run in a year, right? End of day, these are all methods that we use to try to validate the hypotheses that we have. And if you have like, if you're doing a 95% confidence interval, you're still accepting a 5%, you know, a sum amount of false success, do that a little bit more, challenge you, do that a little bit more, and then run way more experience. If you look at the net of what your team is doing over the course of year, what you're doing over the course of a year, you will be positive.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“In that context can be very dangerous, for lack of a better word. You think of other pharmaceuticals. The 95% confidence rate belongs in some companies and some industries because the risk of the impact of a false success is very high. But those of us converting users and trying to upsell folks, we're very fortunate to not have that level of burden on us and we can take advantage of that, right? And so there are definitely times where I will advocate for and I will push for and I will myself use”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source
“Because of the way that I approach it, which is I am very comfortable and very comfortable in using qualitative responses and things like that and supplement to quantitative data to make a decision and that puts less of a burden on the quantitative to really make an assessment whether something was working or not. You know, one of the things I see, I think sometimes goes against whether folks feel that I'm seeing kind of things shift a little is that 95% confidence rate, right? I came from like my background in college, I was in a lab running experiments or really publishing to a journal and stuff. And we had to have that 95% confidence. We had to because the things that were coming out of the lab and being published were influencing things like how we do education and how we understand how bias works and when it shows up and therefore how we can combat it, things where we're wrong and sending a bunch of baloney. Like that can cause some significantly bad things like false positive, false nature.”
2023-03-09 · Lenny's Podcast · Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude) · IDENTIFIED FROM THE TRANSCRIPT · source