YouSaid · the spoken record

Itamar Gilad

lines on the record: 90
first: 2023-09-21
most recent: 2023-09-21
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Find it, you can go to itamargula.com or to evidenceguided.com, and you will find the book and you'll find me. Best value to me. Try it out. Just take some of these ideas, bring them back to your office, talk with your colleagues, say, what do you think we should do about this? Just give it a go and reach back to me. Tell me, I'm easy to find in my website. Tell me what happened. I'm really interested.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Whenever I arrive in Israel, I usually go for Shawama, which is like donor kebab, if you know it, just better. So if you're in Israel, if you go visit Haifa, which is the city where I grew up, definitely check up the Shawarma.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“I think both of them, in their own way, did relatively modest jobs, you know, teaching or doing other things, but they always strived, again, to be the best they can and to deliver the most value they can. So it's very connected somehow. Maybe I'm seeing the world through this lens. But they kind of taught me to strive to be the best I can at what I do.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“That's a big one. Albert Einstein, I think, said, strive not to be a success, but to be of value. And I think that's a great motto for people and for companies. It's something that kind of guides me. And this whole concept of the value exchange, et cetera, is kind of loosely connected to that.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“It's a cliche, but it's AI. There's a company called Eleven Labs that do voices and like the best voices, synthetic voices you heard, but they can also replicate your own voice. So you can create a voice signature. If you're American and you can use their kind of default free version or chip version to replicate your own voice, and that could be pretty useful if you need to, I don't know, narrate an audiobook or do some online course. So I'm finding this service very interesting.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“I like to ask them to design something for a niche audience, so a navigation system for elderly people or some sort of laptop for people with vision impairment, etc. So those are good questions to see their customer empathy, their creativity, their ability to evaluate multiple ideas, the ability to find flaws in their own ideas. So there's a lot of room to dig in there and kind of see how this person is thinking as a product person.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm not really a big TV or movie buff. I'm discovering that YouTube is actually becoming one of my sources of information entertainment. I'm learning a lot of Spanish recently, so I discovered this channel called Dreaming Spanish, which is if you're learning Spanish, it's incredible. So that's my recommendation.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I'm going to cheat. I'm going to recommend a series of books, two series one is Right, cool. And those are obvious ones. One is the series published by SVPG, Silicon Valley Propel. So inspired, empowered, now I think transform has come out. I haven't read it yet, but I'm sure it's amazing. So this is Multi Kagan and his colleagues. They write some tremendous books in every product manager should read them. The other series, a bit older, this is the Lin series. The Lin Start up Lin Enterprise Lin Analytics, there's gold in all these books, LinUX, really, really important books, and I think they're not as appreciated as they should. Running Lin, that's another example.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Hopefully on Amazon, you can search for it. You can go to my site, itmargila.com, and it will be presented prominently there. And there's also the book landings page where you'll find everything you need to know about the book, evidenceguided.com”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So it might not be the full board because you're missing some of the pieces, maybe your goals are not as good or your idea prioritization isn't as good. But if your team is very, very delivery focused and sometimes it's also the opposite. The managers are telling them how to build and you want to break this kind of dynamic, you want to create a step backlog. So instead of a product backlog, let's create a backlog of steps which are just validation steps, betas and previews, etc. And that changes the dynamic pretty strongly.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, the metrics tree is plus your mission plus the individual missions of the teams give you most of what you need to populate your OKRs. Of course, a process of alignment top-down, bottom up, side to side, which I talk a little bit about as well. OKRs is a very rich topic. Those things are usually the core. There's usually some other OKRs about the health of the company, the health of the product, et cetera. Those are called supplementary OKRs. I talk about those as well. Yeah, I think OKRs are a helpful tool if you like them.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“That's one way. I have articles, I have resources on my site, but I try to condense much of what we just discussed and much more, a lot more nuance in the book. If you're interested in that, I would give it a go.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, this is not a roadmap. This is just a tool for the team to manage the project. But I have a proposal for outcome roadmaps inside the book.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“How we achieve this, sometimes we know we have a concrete idea that is high confidence that we already tested, we switch into delivery, then we can put it on the roadmap and say, yeah, we're going to build this thing and we'll aim for October. But otherwise, you want to keep it open. And the roadmaps can kind of suffocate this process if you decide up front with low confidence that this particular idea must be launched.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“I would say that release roadmaps where you are just saying by Q3, we want to launch this or by October we have to launch that, they're kind of competing with this. If you're doing that and people know that the goal is to launch that thing by October, forget about learning. Forget about evidence guided. Recommend using outcome roadmaps saying by October we want to achieve this outcome. By Q4 we want to launch another three countries or we want to grow our usage in India by that much. By this time we need to tackle the problem of churn.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. And another interesting thing that happens every time you run a step, if it's successful, you have evidence. And you can go back to the managers and tell them and share and say, you know, with this idea, we thought it was great, but we got this result. What do you think that means? And sometimes that manager that proposed it would say, you know, I think the test failed. Let's rerun it. Or sometimes they will say, you know, maybe it's not as strong as I thought. The discussion just becomes that much more nuanced and objective, if you like.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, basically. And you can alter this as you go along. Sometimes you can run multiple steps in parallel. It's not always sequential. But that's basically the process, yeah.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Exactly. It's not an engineering milestone or a design milestone, it's a learning milestone. So we build something. And along the way, we actually grow the scope of what we build. We are building the product in the process and we learn. So the two have to come hand in hand.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Then the team together needs to develop which steps should we run? How can we validate this? Some of the steps will be done by the PM, some by the data analyst, some by user researcher, but some will involve the team. There'll be some coding, there'll be some running of experiments. And so there's some ownership around the steps. A sub-team owns each one of these steps. And we will change the board very actively. So if an idea turns out to be bad, we will take it off the board and put another idea in its place. Or maybe we achieve...”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, exactly. You can write the objectives alongside that if you want to remind people what are we trying to achieve, but the cure is also the thing we show here. You pick some ideas, the ones that look most promising as unintuitive as it sounds or counterintuitive as this sounds, I would recommend that you let the team pick these ideas. The manager of the stakeholders can propose the ideas, everyone can propose, but the team should use the ICE process to kind of, and especially product manager is very important here, to choose which ideas to test first.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“The way I say this is at the beginning of the quarter, the team defines its goals. The leads of the team define the goals, but they review it with the team, they review it with the managers, of course, with the stakeholders. Everyone's in agreement. These are all the maximum four key results and the one or two objectives you guys need to work on. Teams cannot deliver on more than that. You copy these key results into the gist Then you start looking at your idea bank, or you start generating ideas and say, How can we achieve these key results?”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Today, because most of the discussion happens at the roadmap level, and then there's a lot of discussion at the task level, but this middle layer of what actually are you trying to achieve and how well are we doing on it doesn't exist. If you do have this, you create a lot more context in the minds of your team, and then they need to ask you fewer questions. You need to tell them less what to do. They know what success and they are able to actually do a lot more on their own.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Let's let them do more than just develop. Let's let them discover as well. And one of the tools I suggest, and again, this is a process, is what I call the GIS board. So it's basically the top three layers of GIST. The goals are on the right. These are just the key results. And usually per team, I suggest not more than four. So you create a GIS board per team. Then the ideas we're working on right now, sometimes with our iScores, and then the next few steps that we might want to pursue in order to validate these ideas. And this is a very dynamic thing. It changes all the time. The team leads need to update it. And the team needs to meet around it at least once every other week to sync, to talk about what's going on. Are we still following the right ideas? How are we doing on the goals? What are the next steps? What's blocking us from completing the most important steps? This is a discussion that has not happened.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Like a project manager, feed the agile machine with perfectly prioritized product backlogs and stories. And it just doesn't work, honestly. And the PMs I made are very tired and they have to spend so much time in planifications and roadmap discussions. They're very busy. They don't have time to do research or to test ideas. I suggest changing this and bringing the developers a little bit out of their kind of agile cage, if you like. And no disrespect to agile. It's a great thing, but...”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“All right, awesome. So in many organizations, there's these two worlds, there's the planning world where basically you have the managers, the stakeholders, some of the PMs really sit and think about what we need to launch. And that's where we create the strategies and the roadmaps and the projects. But guess who's not invited to the party? The people who are actually doing the work. They live in agile world. They're very focused on moving tickets to the don state, on completing burning story points, you know, pushing stuff into production. And there's a big gap between these two worlds. They don't understand each other. They don't see eye to eye. There's a lot of mistrust being built sometimes against the plans or the manager feels that the teams are just not being very effective. We've seen all of this. And the solution kind of the stopgap is to put a PM in the middle. The PM is supposed to make all of this work, deliver on the road.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“You further validate your assumption. Sometimes you need to wall back and change things, but it's another opportunity to learn. So, the key point is you don't have to start at the right hand side, which is expensive, you can start early on, and that leads to parking a lot of ideas very quickly. You realize they're not as good as you thought, and then you can invest more effort into the good ideas. If they generate positive evidence, you can go further and further until that point where you feel you're ready for delivery.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So the next stage is to actually build a kind of more complete version of this, and then you can dogfood it, and you can give this to your users internally. When I joined Microsoft many years ago, the first thing I noticed was that Outlook was very buggy. And I asked people what's going on. And they told me, we were all dog-fooding the next version of Outlook that hasn't come out yet. And that's a very common practice in Silicon Valley. You can do previews, you can do betas, you can do labs. So those are tests. Now, there's a special class of tests which are experiments because they have a control element, so A-B tests, multivariate tests, those are all experiments. I'm using the word experiment the way data scientists use it. Although people tend to call experiments to everything that you see here. And finally, even the release, you can do stage release, you can do percent launches, you can do holdbacks, all of these things help.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Think it's a googly thing, but some people told me that they use fish food as well in their company, the name. So I'm using it. I don't know if there's a better name for it.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“But it gave us some evidence to go and say, hey, we should try and build this So, initially you fake it mid-level tests are about building a rough version of it. It's not complete, it's not polished, it's not scalable, but it's good enough to give to users to start using. So those are early adopter programs, alphas, longitudinal user studies, and fish food. Fish food is testing on your own team.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“But until now, we didn't build anything. Now you're ready to start testing, building versions of the product and putting them in front of users and measuring the results. But initially, you don't build anything. You fake it. You do a fake door test. You do smoke tests, wizard divorce test, concierge test, usability test. We used a lot of those in the tab, the inbox, by the way. One of the first early versions was actually we showed the tabbed inbox working to people, but it wasn't really Gmail. It was just a facade of HTML. And behind the scenes, and according to the permissions that the users gave us, some of us moved just the subject and the sender into the right place. So initially the interviewer kind of distracted them and then showed them their inbox. And in it, the top 50 messages were sorted to the right place, more or less, if we got it right. And people were like, wow, this is.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Obviously, these last two are pretty expensive, so it's often good not to wait until you have the idea and then start doing your research. It's best to keep doing your research ongoing, and then you have some sort of data to rely on and to compare your idea against.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Things that don't require a lot of work. You check if it aligns with the goals. This idea that you have in your hand, you do maybe some business modeling, you do ice analysis, you do assumption mapping, which is a great tool by David J. Bland, or you talk to your stakeholders one-on-one just to see if there are any risks, et cetera. These are usually not expensive things and they can teach you an awful lot about the impact and the ease of UDRD. The next step is to dig data, and usually that goes hand in hand with this, so you can find data in your data analysis, through surveys, through competitive analysis, through user interviews, and through field research, observing your users.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“All right, so steps. Steps are about kind of helping us learn and build at the same time as we said. And one of the patterns I see is that organizations don't know that they can actually learn at a much lower cost. They believe they need to build this elaborate MVP, which is not minimal in any way, and then launch it, and then they will discover. And basically, it's what we used to call beta 20 years ago, but just with a different name. What I'm trying to do here, the steps layer, is to help companies realize there's a gamut of ways to validate your ideas or more specifically to validate the assumptions in your idea. And I created a little model for this. It's called After Assessment, Fact Finding Tests, Experiments, and Release Results. But again, it's just putting together things that much smarter people invented. So in assessment, you have very easy things.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“You're building too much and you're not learning enough, start adopting the steps layer, which we haven't seen yet. And if your team is very disengaged, you have one of these teams where the developers are very into agile, very into quality, very into launching things, start working on the tasks”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“I recommend that they discuss internally where is the biggest problem that they're facing. If the goals are unclear, there's misalignment, we keep chasing the wrong things, started the gold slayer, try to establish an all-style metric, your top business metric, your metrics trees, start assigning teams with their own area of responsibility. If you're spending a lot of time in debates and you're constantly fighting and changing your mind, start with the ideas layer and establish impact is confidence or whatever prioritization model you like, but involve evidence in it. I think the confidence meter is a good tool to use irrespective.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Absolutely. I would recommend that you don't try because if the transformation is way too big, you will get fatigued and you will just create a lot of process for a lot of people and you will not see the results and after a quarter you will give up. So exactly what you suggested is the right approach.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“One thing I discovered, by the way, there's two types of companies that really benefit from this technique. One is those companies that are kind of emerging into modern product development. They have product teams, they have product managers, they have OKRs. They're starting to do agile, but they're starting to do experimentation, but they're struggling to put it all together. Every CPO is building their own little framework. And the other type is those companies that used to be evidence-guided and they regressed. And that happens way too often. Change of management, change of culture. And then all of a sudden, they need to rediscover, to rekindle that spirit that was lost, Google+. So some of the people that actually respond to the strongest are actually surprisingly in these companies.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Evaluating ideas in a very systematic way. In a company like Netflix, by the way, I don't know if they need this specific method.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Absolutely. I think the concept we talked about, the nostal metric, the value created versus the value captured is very important in every company. Building your entire metric trees maybe overkill, doing heavyweighted OKRs maybe overkill for early stage. Early stage companies even don't know how they create value. So they need to iterate and their goals is really to find product market fit. Beyond that What happens is that you need to start building your business model. So that's your goal and you iterate towards that and you need to put metrics on that. And then when you move into scale, you need to try to create order because when you scale up and all of this is covered in the book, there's a special chapter just about these questions. When you scale up, you get a lot of people and a lot of money and everything is happening at the same time. So there you need the order of...”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“And I would argue that the evidence guided method is far more impactful, is far faster, is far more resource efficient than the opinion-based method. Because opinion-based methods tend to waste a lot more of your resources building the wrong things or discovering, learning too late while evidence guided helps you learn earlier. Plus it is a fallacy that if you learn, you don't build. Good teams know how to do both at the same time. And that's actually what the steps layer is meant to teach you or to help you do.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So there's always a trade off between Speed of delivery and speed of discovery. And that actually leads to the next layer of how do we combine the two? Because people tend to think it's an either or either we are building very fast or we're learning and then we're building very slow. But I think we're using the wrong metric. The metric is not how fast can we get the bits into production. There's a lot of uncertainty, and we all face uncertainty, and startupists especially, it's not about getting the bits to production. It's about getting the right bits to production. It's about creating the outcomes that you need, the impact. And so it's about time to outcomes.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“This is a real world usage that I hear about a lot. Use this to kind of do an objective way to say no and gently or to say we will think about it, but look at these other ideas we have and how their impact is and confidence stack up.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, you can also tie your investment into the idea based on the level of confidence you had found, essentially. So early on, you want to do the cheap stuff just to gain more confidence, and then you can go and invest more. If it's a really cheap idea, you can jump to a high confidence idea, a test. You can do an AB experiment, early adopter program, whatever it is, and then launch it. Some ideas you don't need to test. Sometimes expert opinion is enough. If you're just changing the order of the settings, no one sees this or no one will be impacted. The risk is low. You can launch it without testing. So part of the trick is also knowing when to stop, not just trying to force your all the way up when you don't have to.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Finally, to gain medium and high confidence, you really need to build your idea and test it. And that's where the red area is. So there's various forms of test we will talk about them if we have time. And they give you various levels of confidence.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Next, we're moving to data, and data could be anecdotal, so you find a few data points across dotted across your data, or you talk to a handful of customers, or maybe one competitor has that same idea. In many companies I meet, if the leading competitor has this feature and we think it's a good idea, validation is done. Let's launch it. That's it. It's a great idea. We need to do it. Never works, honestly. You should not assume that your competitor actually knows what they're doing any more than you do. Data could be also what I call market data that comes from surveys, from assessing a lot of your data by doing a deep competitive analysis. And there are other methods where you create a larger data set and you contrast your idea against it.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Then we move into slightly harder tests. One is reviewing it with your colleagues, your managers, your stakeholders. The idea, they don't know either. They don't have a crystal ball. They're usually not the users. They cannot predict, but they can evaluate it in a slightly more objective way and maybe find flaws in your idea. On the other hand, groups tend to have biases too politics, groupthink. So groups can actually arrive sometimes at worst decisions than individuals. There is some research to that. Next, our estimates and plans, so you may do some sort of back of an envelope calculation, or your colleagues might go out and try to evaluate the ease a little bit better. That gives you a little bit more confidence, but still we're at the level of guesswork at this point.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“Maybe you created a shiny pitch deck or a six page document that explains in detail why this is a great idea, slightly harder to do, but still very low confidence. Maybe you connected it to some theme. You know, it's about the blockchain. Well, sorry, the blockchain is out of fashion. What's hot right now? Exactly. AI. It's about AI. That makes it a good idea. Absolutely not. Or the strategy of the company. That's another thematic support. Thousands and thousands of terrible ideas are being implemented right now as we speak based on these themes. So all these things combined can give you a maximum of 0.1 out of 10 according to the tool if you follow it.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“All right, awesome. So basically, I constructed it a bit like a thermometer. It goes from very low confidence, which is the blue area or the upper right, all the way to high confidence, which is the red area. And you can see the numbers going from zero to 10, where zero is very low confidence. We don't know basically anything. We're just guessing in the dark. And 10 is full confidence. You know for sure this thing is a success, no doubt about it. And across the circle, I put various classes of evidence you might find along the way. So for example, starting at the top right, all of this blue area is about opinions. It could be your own self-confidence in the idea, your self-conviction. You feel it's a great idea. Guess what? Behind every terrible idea that was ever, someone thought it was great. That gives you 0.01 out of 10.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“What happens is that people tend to just go with God's instinct and then give themselves a high confidence. They say it's an eight, and I'm pretty convinced, so it's eight for confidence. And I found this a bit disturbing because it kind of subverts the whole system. So, I wanted to help people realize when they have strong evidence in support of their guesses and when it's weak evidence. How to calculate confidence in a sense. And for that, I created a tool called the confidence meter, which you can see here, this colorful thing. Shall I go and explain it?”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source
“So, yeah, you're actually leading me to my next point. So, when you come to estimate impact, you will realize it's the hardest part. So sometimes it's just a gut feeling and it's a guess. And sometimes it's based on some spreadsheet or some analysis and back-of-the-envelope calculation you've done. And I think that's legitimate. Sometimes these things do show you some things you didn't think of. And sometimes the best case is based on tests. You actually tested it. You interviewed 12 customers, you showed them the thing. And out of those, only one actually liked it. You should reduce your impact based on that, usually. Or you do other types of tests. So we will talk about tests in a second.”
2023-09-21 · Lenny's Podcast · Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft) · IDENTIFIED FROM THE TRANSCRIPT · source