YouSaid · the spoken record

Itamar Gilad

lines on the record: 90
first: 2023-09-21
most recent: 2023-09-21
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“A lot of your viewers are wondering where the, oh, because there's another variant of this called Rice, where there's a rich as well. Prefer ice because I prefer to fold the rich into the eye for various reasons, but both are valid. Both are equivalent in a sense.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, for the people who don't know him, Sean is amazing. He's like one of the fathers of the growth movement. He coined the term growth hacking. And he popularized the concept of product market fit. Created ice, he created a bunch of things that we use in product that we don't even know”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Is basically the opposite of effort, how easy or hard it's going to be. But both of those are guesstimates. Both of those are things we need to estimate. I would argue that just by breaking the question to these two questions, we usually have a slightly better discussion than just my idea is better than yours. But then there's the third element, which is confidence, which tries to assess how sure are we or should we be about our first guesstimates, about the impact and the”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, impact confidence is, which is basically a way to assign three values to each idea. The impact tries to assess how much impact it will have on the goals. And that's why it's so important that we have very clear goals and not many. How we're measuring the ideas on the North Star metric, on the top business KPI, on the local metric of the team, whatever it is, let's be clear about it and then let's evaluate the ideas against this thing.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Say this idea is going to be the best. But we do say this because we have cognitive biases that kind of convince us that this idea is far superior to anything else and it's definitely the right choice. In order to avoid this, what we want to do is to evaluate the ideas in a much more objective and consistent and transparent way. In the book, I suggest using ice impact confidence in ease. I think I have a slide coming on this.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, next, we need to go to the ideas layer. And the ideas layer is there to help us sort through the many ideas we might encounter. And they may come from, as you said, the founders, the managers, the stakeholders, from the team, from research, from competitors. We're flooded with ideas. And what usually happens inside the organization is some sort of battle of opinions or some sort of politics sometimes or higher-paid person's opinion. Hippo, you had Ronnie Cujavio invented this term in your show. What doesn't happen is very rational logical decisions. These are the best ideas because it's really, really hard to predict, honestly. There's so much uncertainty in the needs of the users, in the changes in the market, in our technology, in our product, in our own organization. It's almost impossible to”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Another thing that happens in a lot of organizations the team topology reflects the structure of the software or some hierarchical model where we want to organize the organization in a particular way. But if you start with a metrics tree, you can try to arrange the topology around goals. And sometimes you need to readjust. It's not a constant reorg, but from time to time you will realize the goals of change and we need to reorganize. So the tree helps visualize that as well.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“This is a very important one, and we will talk about impact assessment shortly. This helps with it, it also helps with alignment because the entire organization is trying to move these two metrics. Most cure results, if you like, the topmost things. So when you go and work with another team and you say, hey, why don't you work on my project? They might say, you know, this project, this idea actually might move the nostril metric more than your idea. And that helps you guys align. And I've seen cases where team B put aside their own ideas to jump on the ideas of team A because of this model. It also creates an opportunity to give some submetrics to teams to own on an ongoing basis. So it creates a little sense of ownership as well and mission within the tree.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, the North Sum metric is measuring how much value we're creating for the user, the core value that they're getting. In this case, this is some productivity suite. So this is number of documents created per month, for example. Of course, we think that every document created, maybe it's a small document, I don't know, AI is in fashion now, is a little incremental value. So that's the number we're trying to grow. The top KPI is what we expect to get. It should be revenue or profit.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Once you have these two, you can further break them down into what I call metrics trees. So there's a metric tree for the north-style metric, and there's the metric tree for the top KPI, the top business metric, which you see here on the left side in blue. And usually they overlap. So you might find in the middle some metrics that are super, super important, because moving them actually moves the needle on everything else.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“And there are examples like this in amplitude, for example. They measure active learning users or weekly active learning users, which are users that found in the tool some insight. Was so important that they shared it with at least two other users and they consume it. So it's a very powerful thing to point at this metric this is the most important metric combined with a value metric that we want to capture, revenue, market share, whatever it is.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Super interesting. I would argue that what they told you is what is the most important metrics we measure? What is the number one metric for us? But it's not what I call the north-stone metric. The North Star metric measures how much value we create for the market. Example, let's take WhatsApp, WhatsApp for Very Messages Sent, because every message sent is a little incremental of value for the sender, the receiver is free, it's rich media, you can send it from anywhere in the world. Compared to SMS, that's huge value. So if in year one we have a billion messages being sent and year two, two billion, probably we double the amount of value. In Airbnb, I think one of our key metrics or the real Nosta metric was Knights booked. I don't know if it was still the case while you were”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“And I would argue that in evidence guided companies, and you worked for a few, so probably you've seen this, they use models in order to construct overrouting goals for the entire organization. One of the models I show in the chapter about goals is the value exchange loop, where basically the organization is trying to deliver as much value as it can to the market and to capture as much value back. And by creating a feedback loop between these two, you are actually able to grow very fast. Now, I would argue that you want to measure both of these and to protometric on each. And the metric we usually use to measure value delivered is called the North Star metric. I know you wrote an article, a very good article about it. And in it, you listed dozens and dozens of companies, like leading companies. And what they considered an off-star metric.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“That's a whole different discussion. You had Christina, you had a real expert on over there, so I doubt I can add more to that. But it's true. OKR is all part of it. But let's start with ghosts. What's our goal supposed to be? Goals are supposed to paint the end state to define where we want to end up. And the evidence will not guide you unless you know where you want to go. And in many companies, what you have is goals at the top for revenue or market share, whatever it is, and then a bunch of siloed goals for each department, there's engineering goals, there's design goals, there's marketing goals, et cetera. And that actually pushes people into different vectors, and it's really hard to decide.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“But we can talk about in each level what's actually being done. So when people say, I have goals, usually they Detect the gauze layer and use it as a planning session. They talk about what shall we build, by when, what are the resources. And that's actually not goals at all. That's planning work.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“That's a great question. So, there's this whole strategic context that is outside of GIS. GIS is not trying to tackle that. It assumes it's in place. There's another huge blob, which is research. GIST is not about research. It's more about kind of discovery and delivery. strategy is extremely important and you can use some of the tools we will talk about to develop your strategy as well. In many companies, the strategy is just a roadmap on steroids. It's more plan and execute just on a grand scale Google Plus, again, was a strategic choice, actually, if you think about it. So in the book, there is a chapter where I touch on strategy and I explain how the same evidence-guided methods are being used by companies to develop their strategy as well.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“These are four areas that you need to look at and ask. In each one, you may need to change or even transform. And as I go and explain each one of those, I'll give you basically three things. In each chapter in the book, I try to touch on three things. The principles behind them, the frameworks or models that implement the principles, and then process. And the process honestly is the most brittle part and the one that you wouldn't need to change and adapt to your company because not two companies are exactly the same. And it's very tempting when you write a book not to give any process, but that's the part that people actually want the most. So it's included as well, but just be aware that you will have to change this process.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“And just listening to this, a lot of this will sound familiar to you because Gist is not a brand new invention. It's a meta framework that puts in place a lot of existing methodologies. It's based on Lyn startup, on design thinking, product discovery, growth. There's a lot of all of these things here. It just tries to put them all into one framework or one model.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“You're seeing this. So, this is the JS model goals, ideas, steps, and tasks. And essentially it tries to break the change, which is a really big change for a lot of companies into four slightly more manageable parts. They're still big, but each one you can tackle on its own. And that's kind of the reason I kind of split it. And goals are about defining what we're trying to achieve. ideas or hypothetical ways to achieve the goals. steps are ways to implement the idea and validate it at the same time so essentially build measure learn loops and tasks are the things we manage you know in kanban and jira and all these good tools these are the things that your development team is usually very focused on”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Then as you go down, you see there's not a lot of experimentation, and if there is experimentation, there's not a lot of learning. And finally, another tell sign is that the team is disengaged. So the engineers are kind of getting the signal that what they need to do is deliver. They're focused on output. That's what they're measured on. So they kind of disengaged. They're disengaged from the users, from the business. They don't care that much. That's usually a sign of usually something that you can fix by adopting a more evidence-guided system”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“I think there's a few telltale signs that I look for. First, the goals are very unclear. Either there are many or they're very kind of obscure and vague or they're about output this misalignment. So the goals part is not there. Usually this goes hand in hand with metrics, missing metrics or just using revenue and business metrics, but there's no user facing metrics. So that's another telltale sign. Then there is a lot of time and effort spent on planning, especially on road mapping, creating the perfect roadmap, which really can consume a lot of time of the top management and PNs, etc.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Think if you come with data, if you run a secret experiment and you come back and you show them, you usually get one of two results. Either they get extremely mad at you and they tell you to get back to work and to do what you were told. And in that case, probably you need to start polishing your resume and look for another place, either inside the organization or outside it, because that person is not being reasonable, to be honest. But the more common case is they're pleasantly surprised. And that's what happened with Steve Jobs as well. He was against phones, but then people showed him also to evidence that Apple can make a phone. He was against multi-touch initially, but then he changed his mind. There was a lot of back and forth. So even Steve Jobs given evidence was willing to flip.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Things are not actually panning out the way you guys are all expecting. What can we do? Should we continue? Should we pivot this? I think the discussion would have done better. Now, I'm doing a huge disservice. I was not in all the discussions. I know probably in Google Plus there were very serious discussions happening along these lines. But it just as a general trend, I find that evidence is very empowering for us smaller people in the organization or mid-level managers to be empowered to challenge the opinions.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, another nice thing about Google is that it's a very open culture. And people are not shy to tell even Sergei and Larry that they are wrong. And they do this all the time. In certain forms, right? You need to know the right channels, but there was a very big discussion about Google Plus and whether it's the right thing to create a clone of Facebook. There was very public internal discussion. I think what I would change. Is not have this discussion based on opinions because when you have the discussion, you come with your own opinions. Usually the most senior person's opinions will win. That's just the way it is. If we had come with data, hard data, and we said, listen.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“And by the way, the example you give that's a classic example Steve Jobs, he just brainstormed in his, I don't know, kitchen, the iPhone, and then just told the team to build it. That's the story Steve Jobs told, but it's not the real story at all. Now we know what actually happened. And iPhone has actually a story of discovery, of trial and error, multiple projects to do with multi-touch with phones. Most of them failed. Steve Job was the architect. He kind of managed to connect the dots and eventually come up with this perfect device. But it wasn't actually the creator. It wasn't his grandchild. It was actually against it for a while. But over time, as he saw the evidence, as he saw what this thing can do, as he saw the demos, he was able to piece together something that was very useful.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“I think the founders are very important, especially in the startup and scale up space. They come up with many of the most important ideas. And it's super important that they have the space to express and to push the organization to look at those. However, it's not about Shutting them down, it's about looking at them critically. You need to create the environment in your organization where the leader comes and says, you know what? I talk to these three customers.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So this is a book for people like us, product people, who want to bring evidence-guided thinking or modern product management, if you like, into the organizations. There's a lot of challenges. It's not simple. We all read the books. We all know the theory. We all know some parts of the system. It tries to give you a system how to do that. It's a meta-framework that kind of helps you lift your organization in the direction of evidence guidance if that's what you want to do.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, in hindsight, I was asking myself, why was this project so different? And I think the reason is that we didn't have that much confidence in our opinions. We had opinions, we had ideas, but we didn't just go all in and just let's build it. We actually used an evidence-guided system. And I think that's not unique just to Google. I think every successful product company out there that you look at, Amazon, Airbnb, anyone you will check, at least in their best periods, they found a way to balance human judgment with evidence. They didn't try to obliterate human judgment and opinion, just to supercharge them with evidence. And they came up with very different models. Apple is another example. But the principle still holds in all these companies.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“That turned out to be very successful for a lot of these passive users. And this was a surprise to a lot of people because most of my colleagues and most of the people I talk with actually know how to manage their inbox. So for them, that solution makes complete nonsense, like splitting promotions and social to the side sounds like the stupidest idea. But there's about 85% of the population, 85 to 88% that absolutely love it. And today, Gmail has about 1.8 billion active users, according to Gmail. Most of these users are using this feature. So it was a pretty high impact feature as well.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“And I came up with an idea how to fix this. I was sure it was great. I wanted to push it, you know, plan and execute. But my colleagues were like, hold on. We actually tried this. We have a bunch of ideas to help people organize their inbox. They're not using it. Why is your idea good? So that sent us kind of, and me and my team, into searching, into researching these users, into establishing goal that was much more user-centric, and then thinking of other ideas. And then we started testing them much more rigorously. And basically, we started testing on our own inboxes, and then we recruited other dog fooders, other Googlers to test the same inbox. Then we put it outside for external testers with a visibility studies, we did data, we built a whole data mining team and a whole machine learning team to build the right categorization. And we ended up with a solution.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“But I think inside Google, the DNA still existed. So inside JML, the next project after Google+ was the tabbed inbox. So it started with the reverse of Google+. It started as a very small idea that no one believed in. And we started looking what's behind this idea, what's the goal, what's the problem actually we're trying to solve. It turned out that a lot of people were receiving social notifications and promotions, etc. And most of them were very passive. They weren't clearing their inbox. They were just living in this world of clutter.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“And so you had to kill your project or pivot it seriously if it didn't work out. And I think had we kept fail fast, it would really have helped Google+. If we had this mentality. But for some reason, with Google+, Google put this playbook aside and used a different playbook, which I call plan and execute, essentially.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“That's right. So Google is a very successful company. It's not for me to criticize it or to, in hindsight, kind of say you guys need to be better. And some of the people that were behind Google Plus were some of the smallest leaders. And I still think there are, despite this story, if you look back at the history of Google, how things started in the first decade or so, Google was what I call an evidence-guided company. So essentially it put a high premium on focusing on customers, on coming up with a lot of ideas, on looking at the data, looking at how these ideas actually worked out. They weren't shy about launching betas and things that were very rough and incomplete and learning from that. And then they expected people to take action based on the results. So fail fast is a very famous paradigm.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, it's true. But at the end of the day, neither Google's advertising revenue was affected. Neither was Facebook affected. So it turned out this idea was not that necessary after all.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So there was a tremendous earthquake inside Google to create the Google Plus team. Teams and entire divisions were kind of thrown apart and reformatted. And I think at its peak, it was about 1,000 people inside my It was a division the size of Android and Docs. A really sizable thing, they're on their own buildings. It was, yeah, it's taken from the playbook of Steve Jobs, you know, create this whole secretive project inside and just run like”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“And for me, this story kind of was the epitome of what I call today opinion-based development. We come up with an idea, we believe in it, all the indications show it's good. Maybe the early tests show it's good. Then we just go all in and we try to implement it. And I made this very mistake many times as a product manager. I was the guy pushing for the ideas. So for me, this was kind of a turning point. I felt we need to adopt a different system.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, putting aside all the tremendous waste that went into this, all the millions of person hours and personal weeks, in hindsight, not only did Google bet on the wrong thing, it missed much easier opportunities. So just not far from Google's headquarters, there was WhatsApp, not very famous in the US, but they actually created massive impact, hundreds of millions of people were using their stuff, and they became a threat to Facebook much more than Google was. So Google missed the opportunity of social mobile apps like WhatsApp, like Snapchat, etc.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“And we went on and we launched a series of features in Gmail for a couple of years, honestly. And Google Plus itself became this massive project, very feature-rich, and with a lot of redesigns and iterations. And none of it worked. Turned out people actually didn't need another social network. People didn't love it. People didn't use it. Eventually in Gmail, we rolled back all the Google+ integration a few years later. And Google Plus itself was shut down in 2019.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Google was my first experience at Gmail. I joined Gmail in August 2011. And the first thing they asked me is let's connect Gmail with Google+. If you're hazy about the story back then Facebook was massive. It's still massive, but then it was growing like mushrooms. People were spending hours that really freaked out Google. And the solution, the obvious solution was to launch a social network of Google called Google+. And we all believe in this thing. It really caught on very well initially. We all used it. We all believed in it. So our mission was to build this thing. And Google really cut no costs. It created a whole new division within Google and it created a whole strategy around Google+. And we had to connect Gmail and YouTube and search to Google+ to make them more personalized in a sense and more social. So that was the idea.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“You fake it. You do a fake door test. You do smoke test, wizard of the voice test. We use a lot of those in the tab the inbox, by the way. One of the first early versions was actually we showed the tabbed inbox working to people, but it wasn't really Gmail. It was just a facade of HTML. And behind the scenes, and according to the permissions that the users gave us, some of us moved just the subject and the sender into the right place. So initially the interviewer kind of distracted them and then showed them their inbox and in it the top 50 messages were sorted to the right place, more or less, if we got it right. And people were like, wow, this is actually very cool. But it gave us some evidence to go and say, hey, we should try and build this thing.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source