YouSaid · the spoken record

Ramesh Johari

lines on the record: 97
first: 2023-11-09
most recent: 2023-11-09
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“There's a couple lessons there about both trying a lot more stuff that's not all risk averse and not necessarily running everything for so long. So really getting velocity up. So you can see that there's a big incentive problem there, right? Because the culture that says it's okay to fail big actually requires changing the terminology of wins. This is one of the things I hate most in A-B testing, I have to say. I get where it comes from. Experimentation was never historically in science about winners and losers. It'd be weird if Ronald Fisher, who's kind of the father of experimentation with his agriculture experiments, talked about winners. I don't think that's necessarily how he talked about things. Experimentation was always very hypothesis-driven. It's about what are you learning? And that's really an important distinction because what it means is if I go with something big, risky and it quote unquote fails, meaning that doesn't win, I”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Now, what do I mean by these two things? So, what's interesting to me about this dynamic is experiments don't live in a vacuum. Companies have incentives. And in companies that really go all in on experimentation, one of the things that gets wrapped up in that is the incentives around experiments. Because if you go all in on experiments, a common thing you'll see is data scientists get judged based on how many wins they had that quarter. And right. How do you get more wins? It's easier to get wins when you're being incremental. And because it's important to have wins, you have to run them long enough to demonstrate that they're really wins. You're less willing to cut something off in exchange for trying something riskier. So the big lesson is Microsoft paper. It's called A-B testing with what's called fat tails, which in lay terms just means you're running a business where there's potentially big opportunities out there if you look at kind of the effects of the experiments that you run.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Ultimately What's getting built and tested are choices that are made through the organizational structure of the data scientists, the PMs, the engineers. Everybody's on the, you know, before we're running experiments, we're actually thinking about even what's worth experimenting. Like what designs are we commitment? So that's one. And the other big one is how long do we run these experiments? Okay. That's a big choice. And what I generally believe, and I think there's a paper we can link to later that I'll point your readers to as well, that not my paper from some folks at Microsoft. What I generally believe is that we're risk averse on both these two dimensions. That what people decide to test in a world that has promoted experimentation for everything tends to be more incremental by design, okay? And we'll come back to why actually let me answer the because in a second. So that's one. And two is people tend to run experiments for a long time and probably longer than they should.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, first of all, I'm really glad you broached the E word. I was dancing around it, and I'm really glad that we talked about experiments because, yeah, one of the big lessons of this recent conversation we've been having is just how could you possibly know that difference without doing something like experimenting? Okay, so yeah, I am a big believer in experiments. I mean, I'll just lay those cards on the table. I love working with businesses that think experiments are important to helping make good decisions. Now, all that said, I am also someone who feels pretty strongly about this exact issue that you're raising, which is you can't experiment your way out of everything. And one way, you know, that one frame I like to give people is that although you might say you're an experiment driven business, some businesses will proclaim we literally test everything, but that kind of leaves aside a little bit is there's a lot of degrees of freedom in what it means to test everything.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Right? Because in the end, while we might have these predictive algorithms to rank who you're going to hire, that's not the important question. Interestingly, the important question is actually to evaluate the quality of the match that's made. And we would do that through the next step of that flywheel. We'd ask ourselves, you know, what ratings did they give back to that freelancer? Do they hire that freelancer again? So you're comparing two different algorithms not through their ability to recreate the past, but their ability to make matches in the future that can And then rating systems, I think we could talk quite a bit about kind of a similar phenomenon there too.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“About two different ranking algorithms. I don't want to be only comparing them in terms of how well they recreate the choices people made in the past. The way I'm really going to evaluate those is in my market, does one of those lead to better matches or more matches than the other one, right? So Airbnb as a business, like what are the most obvious core metrics? It's bookings and revenue. So you're going to want to ask a very basic question. If I use the ranking algorithm Lenny just developed last night versus the ranking algorithm mesh developed last week, does Lenny's ranking algorithm lead to more bookings than Ramesh's ranking algorithm? And it's so important to put it that way starkly because that's so different a question than does Lenny's ranking algorithm do a better job of predicting over the last two years what bookings people made than Ramesh's ranking algorithm okay so that's that's I think like you know at that level then you know we talked a little bit about about ranking at the point of making a match and I think that's where this kind of hiring issue popped up”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Maybe the right way, the right frame of reference for this and the word that an academic would use is causal inference. So what we're changing from is machine learning to causal inference. So let's think that through in a couple of different use cases that are related to that marketplace data science flywheel I talked about earlier, finding matches, making matches, and then learning about matches. So finding matches, like you said, a core part of that is search and recommendation. And each of those relies on rankings. So I want to be able to rank order. Let's say I go do a search on Airbnb. On a rank order of the different listings in the marketplace, right? At some level, it's true that what I'm trying to do there is I'm trying to just predict what are you going to like the most, right? But I think there's an important piece of that also, which is that I want to think a little bit about the distinction between two different ranking algorithms. That's the real decision that's being made. And when I think about the distinction,”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“And so the first and most important thing that I feel very strongly about, and what would I get a data scientist to do is no matter who they are, even if it was that person in the weeds thinking about building this prediction model for hiring, get them to be thinking in the back of their mind always that their goal is to help the business make decisions and that the distinction between causation and correlation matters a lot. We can talk a lot more about how does that play out in terms of their day-to-day work, but at least at a starting point, you have to recognize that the first step is always recognition, that prediction isn't the same thing as making decisions.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Predicting is about picking up patterns, but making decisions is about thinking about these differences. Now, why is that important? Because we learn in high school. Correlation is not causation. That's a phrase everybody has heard all over the place. What does it have to do with this? Well, when we teach people to build machine learning models, we're asking them to make predictions. We're asking them to find correlations. Prediction is inherently about correlation. But when we ask people to make decisions, we're asking them to think about causation. If I make this decision, then will I actually increase the net value of my business, right? Will I have by sending the promotion, increase the likelihood that this person is going to spend more on my platform?”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“And when you look at it that way, what you realize can happen is picking up on patterns because of good predictions, right? Finding the people that have high LTV because you predicted that is very different than making good decisions, which is about saying the difference in their LTV is going to be higher because I sent them this promotion. I love this example because I taught a class here at Stanford. It was like an executive education class. We had all the executives from a company in the room. And one of the people in the room was the chief marketing officer. And I just asked this question like, hey, okay, let's see you got this great LTV model. Who would you send the promotions to? It's like, definitely the highest LTV people. And there's a CMO in the room. And so, you know, it's like it's a little bit of a delicate situation like pushing back a little bit, right? I do want to be clear. There's reputational reasons you might do that anyway. I mean, I'm not trying to get away from that, but just to make the narrow point that”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“But the problem with that way of thinking is actually predicting what their lifetime value is isn't really the question. The question is, how much more are they going to spend on my platform because I sent them that promotion? That's a very different thing. It's a differential rather than an absolute. I'm not interested in their absolute LTV. I'm interested in the difference in their LTB because I sent them this promotion.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“If you think about it a little bit, you realize what that algorithm is doing, it's really just picking up on patterns in past data. So, yeah, that's great. This person is likely to be hired. But what we really want is something different. We're trying to add value by ranking people. So, you know, to give another example that's similar to this, when you're a marketing manager and you've got a crack data science team that's built a long-term value lifetime value model for you, you're not going to get in trouble with anyone if you send your highest value promotions to the highest LTV customers, right? Who's going to blame you for that? Because you're like, oh, yeah, you know, this person's worth a lot. And I set them this promotion. You know, say that in your monthly report, nobody's going to give you a hard time.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Sounds pretty natural, right? And then you think about it a little bit. And this to me is really, it's such a passion project to get people to understand that this is why the humans in the loop that help us in businesses and making sense of data are so critical is the following problem.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, you know, we're working on this problem. Great. And then I kind of poked my head up a little bit. I'm like, why are we working on what is this going to do? Well, it turns out the reason these kinds of things are important is they get used to make decisions. So what kind of decision do you make with that? Well, one thing you do is you say, oh, well, if I could predict who's most likely to be hired, then I should just rank people based on that. And that would be a good matching algorithm, right? That would be a good way to sort and triage people applicants for employers when they're screening trying to figure out who to interview, who to hire. Great.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Predict which of these workers is most likely to be hired on that job. That was the narrow question. And so why is that a good question? Because we have a whole awesome set of tools now to solve that kind of a problem exactly. How do we do it? Take a lot of past data of past jobs, past applicants, past hires that were made. And then we ask these crazy big black box algorithms, all right, do the best job you can predicting who's going to get hired on this job with these applicants. And we use that data to test how well these applicants, these algorithms are doing. That's like machine learning in 30 seconds, basically.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, in a lot of companies today, especially a main thing that you ask data scientists to do is build what's called a machine learning model. Machine learning model even already can mean a lot of things to a lot of different people. I'm going to focus on something very concrete. You're asking them to predict something When I started at ODESC, this is in 2012, one of the funny things about me is I started ODES because I had a academic career up to that point in about 10 years just building mathematical models. What I expected would happen is I go to industry and I'd be told, hey, look how important data is, you know, and definitely my eyes were opened. And one of the first things I was asked to think about is, well, okay, someone comes to ODAS, posts a job. Workers apply to that job.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, one answer to your question is if I'm in a place like Uber, Lyft, DoorDash, I want to have good data scientists thinking about pricing, because that seems like something which should be heavily dependent on the instantaneous state of supply and demand in my marketplace, right? So that's one type of answer is, well, do I need people, data scientists, working on pricing? Do I need data scientists working on search? Why search? Because maybe in my marketplace, finding the needle in the haystack is really the biggest highest friction problem. So maybe I need a lot more data science to say about search. That's what I'm going to evade, okay? I'm going to focus more on something completely different, which is just a more philosophical point about what a data scientist does.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“This is an incredible question, right? Because I think I could answer it in a number of different ways. One question I think there that's kind of basic is just what should this person be doing? And I'm going to actually kind of evade that question a little bit. I'm going to give some examples of what they could do, but I feel like that's one where context matters a lot. So as an example, you know, at ride sharing or grocery delivery marketplaces, pricing means actually what do you pay for that ride or what do you pay for that delivery, right? So that's actually the price that's set at the moment you actually place it. Just to be clear, by the way, if you order from DoorDash, I don't mean the price to the restaurant. I mean, what do you pay to DoorDash, right? What's that? What's that fee? Is there a surcharge? Because it's surge or whatever, right? So, okay, so that's a thing, right? But that's not really a thing in a marketplace where the platform's not setting the prices. So at Airbnb really hosts are the ones who are charged of setting prices for their listings.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think one of the great things about that experience is it felt magical to have someone who kind of got to know you, right? But that depends on a relationship that doesn't feel like a freelance relationship every single time you're going back. Another example that I would pull out is pretty much any healthcare platform. For example, for physical therapy, it'd be weird if every time you went to a physical therapy platform, you just got randomly matched to whoever happened to be available then. So I think there's some curation that needs to happen of that relationship. Does that mean full employee? Maybe not, but it does mean you have to think a little bit about exactly as you brought up, you know, what's the nature of curation of your labor pool?”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“I think one of the things that's cool about Stitchfix is the experience that people had early on with stylists at Stitchfix.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, that's a great point. One of the things I think that's useful for people to think about here that you're raising. At some level, it's kind of tied up, I think, with that question of whether I should have employees or kind of contract or freelance work on one side of the marketplace. And that's actually a pretty old question in economics, the way we talk about it often is a distinction between a market or a firm. And kind of one of the interesting puzzles in economics, Ronald Coase is a famous economist who thought about this, is, well, you know, if markets are so efficient, why do we need firms? Because if markets are efficient at matching labor up with things that need to get done, why would you ever need a firm? And that's one of the earliest recognitions that transaction costs are a real thing. And that's kind of one of the things that firms are solving for. And I love what you're saying because what it's recognizing is, hey, for your frictions, the best resolution to that might not be to have a marketplace. It might actually be to have very tightly controlled labor. A good example of this actually, you know, stitch fix.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“On one side and use it to attract the other side. If you don't have either side, don't worry about it. Don't worry about being a marketplace. Worry about scaling one side. And in that world, it's opened up your visibility up completely into the advice of many, many startup advisors. People who have advice not so much about scaling a marketplace, but about scaling a startup. And I really, I want to say you got to let the ego go at that point, right? Like it's fine to articulate to people that your vision of the future is to be a platform or marketplace. As I said, virtually every business is going to have that option at some point in the modern tech enabled economy anyway. So you're not saying something people don't already know when you tell an advisor or an investor that. But I do think you need to be humble enough at the starting point to recognize that there's no sense in talking about a marketplace if you don't have scaling on either side yet.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Get more buyers. That's one option. There's no shame in not being a marketplace. Scaling a business is scaling a business. If that's the way to do it, do it. If you decide you want to be a marketplace, then at that moment when you've got a lot of buyers, but not a lot of sellers or a lot of sellers, not a lot of buyers, the choice you're facing is how do I take advantage of having that one side scaled to attract the other side? We can talk more about that, but there's a lot of ways to kind of hack that, to think about how, so to take Uber as an example, right? They would walk into New City. And one thing that Uber was kind of commonly known for doing this was back in the days when really Uber Black was the only service is they just hand out coupons for free rides at kind of events, parties, things like that to take people home. And that was a way of saying, hey, we're subsidizing the drivers in the city. That's our scaled side. Now we're going to use that subsidized driver base to attract riders. So that's like how do you get that flywheel going? And again, many people have written about how to take liquidity, scale liquidity.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Let's go back to kind of thinking about this concept of a marketplace of reducing friction, right? So the litmus test I like to give to someone who claims to me that they're building a marketplace business or their marketplace founder is do you have what I would call scaled liquidity on both sides of your platform? What is scaled liquidity mean? What it means in lay terms, and that's, by the way, I am a data scientist and I love to think about these quantitatively, but fundamentally, like if it doesn't pass the smell test, then you don't have to keep going with the data science. The smell test is scale liquidity asks, do I have a lot of buyers and a lot of sellers on my platform? Or do I only have one of these two or do I have neither? If you don't have both, you could call yourself whatever you want to call yourself. But at this moment in time, you're not a marketplace. All right. If you have one, congratulations. You've won the game on one side of the market. And now you can, if you want, you have a choice point. You can lean into growth on the side that you're doing well with, right? You got a ton of users, a ton of buyers. Great. Lean into it.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“From the perspective of those sellers, that's a breaking of a social contract that's been developed over a very long time. And so, you know, I love the substack example because that's like, hey, let me amplify or social contract, right? But I think for every one of those, there's an eBay warning sign that you can also trap yourself a little bit.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“That's like a really positive story, right? Where they managed to actually expand the frontier of their business by enabling that network For every one of those, there's unfortunately a lot of negative stories. I mean, one that I think is very painful is how eBay sort of had a lot of challenges with its seller community as it introduced more and more fine-grained kind of sources of fees. And I think a lot of that, I mean, there's many, many treatises at this point written on eBay and their history and how they got to the point that they're at. But I think one kind of simple thing I do want people to think about there is that the sellers on eBay who had matured with the platform, who had grown with it, had come to develop certain expectations about what their lives on that platform would look like. And it's understandable because a lot of these businesses, they had built their livelihood on that platform. That was their entire business. So when you now reach in and you say, I'm going to completely change the rules of the game in which your business model operates.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Longer term relationships made disintermediate? Does that mean you need a pricing plan that actually takes that into account? So early commitments in this case to like a particular pricing scheme, particular monetization can really tie your hands as you then realize later you actually are a platform.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“And that meant that the longer that goes on, the less value the platform is adding into that relationship. But you're still pulling 10% of all the dollars. So what does that lead to? A word that most marketplace CEOs know well is disintermediation, which is where you were intermediating between the two parties and now they disintermediation means that essentially they're like, hey, we don't need you anymore. My favorite example is we had some stuff delivered from IKEA by a thumb tech worker once. And my wife is like, oh, thanks a lot. You're so reliable. He's like, hey, great. Here's my business card. Ever need me again? Just call the number on the back, right? And that was it. Like Thumbtech got there one lead gen and then, you know, we didn't need the platform anymore. And I think this issue for ODES meant that after they merged with ELANC and became Upwork, they had to think a little bit about, okay, what's the monetization strategy we want to use? How do we address this issue that...”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Future awaits you. There may be choices you're making early on that are tying your hands later. A great example of this is when ODA started, it was because the tools they were providing were for ongoing monitoring of work. It's a very natural thing to say, you know, we'll just take a constant fraction of the dollars that cross the platform. That all works well and good until after you become mature, some of these relationships between worker and employer last a long time. And most of the value is generated now, not so much because they're able to track each other, because the trust is now there, but because they found each other, because they're able to build that relationship throughout Esk.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“But open AI is a marketplace now. They may not want to call themselves a marketplace, but they have plugins. The plugins are flooding that platform. I don't know. If people have played with it, it's not an easy thing to find the plugin you need for what you want to do. And that really is a two-sided thing now. There's the plugin creators and there's the users. And they may believe it. They may not believe it, but they are a marketplace. So I think a different way to think about it is every founder is a marketplace founder. It's going to be a choice they want to make for themselves of whether they want to become that platform. That's, I think, one. And two is because that's the case. I think one of the other challenges I find founders struggle with is you never want to, you don't want to overcommit your future. And what I mean by that is that you're building up trust and you're building up a sense of what kind of business you are in your early days. If you believe that this kind of platform future awaits you or market platform.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“You know, one thing you said was that's what you tell marketplace founders. I mean, something I've actually pressed hard on in my own way of thinking about this is that maybe we shouldn't talk about the concept of a marketplace founder. Really, there's founders. And I think every entrepreneur, I mean, one way to think about it, right? It's very hard to think about a human business endeavor that has not been disrupted by the potential for transactions to take place online. And if that's the case, it means literally any founder is a marketplace founder. It'll be a choice they make after they grow as to whether they want to build a platform. I mean, to take a very hot recent example, no one in their right mind would have thought of open AI as a marketplace.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“That people in this space are facing that I can deal with when I'm not a scaled marketplace. Again, with the cleaning industry, I can comment on that from personal experience, but otherwise I think that's the way I would think about it. It's almost never about building a marketplace when you're building a marketplace.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“You know, that's bespoke. It means different things. And in the case of Odesque, where I started, that initial thing was that remote work is a weird thing because basically you've somehow got to know that this person who you're not next to is doing what you're asking them to do. And so the initial value proposition of ODESC was to provide tools for workers to verify they were working the hours and doing the things that they said they were doing, you know, screenshots and various kinds of tracking. And then in return for that, to be able to provide guarantees on both sides, right? So now the workers could say, hey, I worked what I said. So I should get paid. And the employers could say, oh, I actually see that you worked what you said. And so, you know, I feel comfortable that I got what I paid for. That was the initial value proposition. It was resolving a trust issue at a remote scale, right? At that point, liquidity isn't the game. It's asking what's a problem.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“That wasn't the key thing anymore, right? So what's the moral there? The moral is a marketplace business never starts as a marketplace business because what we think of as a marketplace business is something which at scale is removing the friction of the two sides finding each other. But when you start, you don't have that scale. So when you start, you had better be thinking, what's my value proposition in a world in which I don't have that scaled liquidity?”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“And once they overcame that first thing to get some liquidity onto their platform, they could move towards asking, how do I solve for these frictions that I talked about earlier? How do I solve for helping people find potential matches? How do I solve for people making those matches, right? You can't do that when you don't have liquidity on your platform. It's silly to tell someone, hey, I'm really going to help you find all those drivers out there, even though I only have three drivers on my platform. That's not a friction you're solving for. In their example, As they evolve, they actually shifted their monetization model away from billing specifically for this friction of allowing you to pay with credit cards. Instead, to now billing for how you were interviewing and contacting sitters. They had kind of a two-part plan for that, you know, one with like a pay-as-you-go menu, one with a more of like a subscription option. But the key thing was, either way, what you were paying for now was finding potential babysitters, not paying them with a credit card.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“High school students or something, they want to get paid. They're not going to take your IOU that you'll send them some check in the mail the next day. And unfortunately, you often don't have cash. They don't take credit card. They're high school students. That was an incredible friction to address, which is literally just, we accept credit card payments for babysitting. That's it, right? Now, from there, what happened is they took advantage of Facebook networks between parents and babysitters to build trusted introductions. So like, let's say MySitter wasn't available. I get to know sitters in the Facebook network of that sitter, right?”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“That they think too much about a marketplace before they're a marketplace. That, in my view, is the biggest failure mode. You mentioned specific things, cleaners. I wonder about that, right? Is it about something about the cleaning industry? It possibly is. I don't claim to be an expert on the microeconomics of the cleaning industry. But often it's not that. It's that I thought I was building a marketplace from the beginning. And that's not the way the world works. So I'm going to give you one vignette of this that I really like. And that's Urban Sitter. So an urban sitter is a babysitting marketplace. Okay. We can talk about kind of their whole life story, but I think what's most interesting is really the early days. And in the early days, what I found interesting, the way I found out about them actually, is that we were stuck looking for some help. And I found out about this new platform where kind of the clever thing was, you know, when you used to hire a babysitter, it's just like pre-Venmo days, you need a cash on hand. Because when the babysitter is done at the end of the day, they're usually like, you know.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“That is such a fantastic question. And I want to preface what I say with a couple of comments. So, one of them is that I've worked with a lot of different marketplace companies, but anything I say is pertaining to something more sensitive. I may not name the company just over the course of the podcast. But the other more important thing I want to say is that I'm a professor at Stanford. And there's a reason I'm not like a successful scaled entrepreneur of marketplaces. And that's because I probably haven't unlocked the key to exactly the question you asked. But nevertheless, I have some thoughts on it. The most important one is this. What I found talking to people who want to start what they think is a marketplace.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Finding potential matches, making matches, and then learning about those matches, and then cycling back again, that is the data science and marketplaces. And I feel like every marketplace that you could think of in any vertical has those three problems to deal with and relies on algorithms and data science to help them solve it. And in turn, that is the, I think, really the underpinning of taking those frictions away.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Hire? Who should I interview? It's a common problem we face in the real world, but now it's all remote. I don't meet these people in person. All I've got is this application they submitted to me. I need help triaging that. So that's helping make a match out of possible partners you can match with. And then finally, we make matches. Well, what did matches tell us, right? I mean, if you stay somewhere in Airbnb, you learn something about the host, you learn something about the listing. The host learns about you too. And that's all information. that the marketplace should feed back in. So this is where we get to rating systems and feedback systems, even passive data collection, right? Did you leave your booking before you were supposed to leave? Well, maybe that's a sign that something didn't quite work out the way you wanted to work out. So that's passive data collection. Did you leave five stars? That's active data collection. Get all this back in. And what does that do? Well, that lets us do a better job finding potential matches and make potential matches in the future. Every single thing I just said.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Of commerce now. And it means we can architect and re architect the marketplace kind of on the fly. And we really are doing it all the time. And these frictions that are getting taken away, they're getting taken away because of data and data science. So I really want to highlight kind of three pieces of this for people, which I kind of want you to think of them as a cycle, but to start with, let's just lay them out one at a time. Okay, one of them is finding people to match with. So that's the problem of I want to stay somewhere, who is out there who's willing to let me stay with them on a given timeframe. And then if I'm a host, I have a listing, who is out there, who's willing to stay at my place when I have it available. So that's finding matches. Then there's making the match. And so here, you know, going back to my time at Odesque, a big problem that we dealt with there was if I've got multiple applicants to my job, who should I?”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Think this concept that we're making money by taking transaction costs away is such a fundamental idea that's misunderstood around marketplaces that when you're an entrepreneur starting a marketplace or thinking about your business model, I think you can be wildly off if you forget that that's the thing that's fundamentally your value proposition. And then you asked about the role of data and more broadly data science in marketplaces. It's an interesting thing, right? The example I always love to give are the ancient agoras, you know, in Greece or like Trajan's market in Rome. I mean, look at pictures of these things. What really stands out to me is the rock. I mean, these things are made of stone. It's not like you were going to move a booth from one place to another place without moving a lot of rock from one place to another place. So you flash forward to 2023, and here we are with technology undergirding pretty much every kind.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Is who's willing to give me their room? I mean, in principle, there's people who are willing to let me stay in their living room, but I don't know who they are. So those are frictions. And what the marketplaces are selling you is taking the friction away. That's what you're paying them for. And it's an important observation because what that means is the marketplace is customers aren't just the people buying the rides or buying the listings. Actually, the hosts are Airbnb's customers. And the drivers are also Uber's customers, right? So both sides of the marketplace are the customers of the platform. Both sides depend on the platform to help the platform take that friction away. Because just like”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Uber and Airbnb are selling you something, they're taking away of something, which is a weird thing to think about. What they're taking away is the friction of finding a place to stay. They're taking away the friction of finding a driver. In economics, we call those things transaction costs. You know, when you take Econ 1, you learn about markets and how supply meets demand and we get prices out of that. But what you don't learn until like Econ 201 is that markets don't always work. And one of the reasons markets don't always work is because we have what are called market failures due to the presence of these kinds of friction. So like what's a market failure? It's that, you know, Lenny wants to get from Palo Alto Alto to Burlingame and he can't do it. Why can't he do it? Because he doesn't have anyone to drive it. Well, why doesn't he just call someone to drive in? Well, whose light he's supposed to call? Who are those people? Are they out there? Are they willing to drive him right now, right at like, you know, 10 a.m. on a Friday? Are they willing to take him somewhere? When I want to stay somewhere, when I'm traveling, a frick.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“It's interesting when people sit down and think about, say, Airbnb. What does Airbnb sell? Average versus like, oh, that's pretty obvious. Airbnb sells rooms. I go there to book a room I want to stay at, right? Other people say, oh, what does Uber sell? Uber sells me rides. I use a room and I need to get a ride from somewhere to somewhere else. And in some sense, you're not wrong. I mean, you go there, that's a platform to get these things, but that's not what the platform is selling. That's a really important distinction. There are people on the platform that are selling that to you. The hosts on Airbnb are selling you listings. The drivers on Uber are selling you rides.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“I really am. Yeah. And I actually want to thank Riley too. I got to know Riley when I was at Odesque, first as a research scientist and then I directed their data science team. This is like way back in 2012. And I was looking around for people who are experts on how we think about data in marketplaces. And Riley Newman came up. And so I invited him to come talk to us at ODESC. And we stayed in touch since then. You know, those were early days of where this industry was. And I've had a kind of lengthy career now thinking about those kinds of problems. So I'm pretty excited to talk about it with you.”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source
“Marketplaces are a little bit like a game of whack a mole. One example that I came across with one of the companies I worked with that I love is our new supply side was having a pretty bad experience. So what we decided to do is build some custom bespoke features that were really going to direct them to more experienced folks on the other side of the market. Good. And then, yeah, lo and behold, pretty soon those metrics start to look better. But then we're looking at we're like, wait a second, now, you know, the existing folks on the other side are having a worse experience. So you kind of whiplash around. You're like, oh, wait a second, we better do something about that. So we take them, we try to match them up with the more experienced folks. And now suddenly, month after that, you're like, you know, wait a second. And your metrics just keep moving around. And that's because the whack-a-mole game here is ultimately a lot of marketplace management is moving attention and inventory around many of the changes that are most consequential create winners and losers and rolling with those changes is about recognizing whether the winners you've created are more important”
2023-11-09 · Lenny's Podcast · Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor) · IDENTIFIED FROM THE TRANSCRIPT · source