YouSaid · the spoken record

Gary Klein

lines on the record: 89
first: 2022-08-09
most recent: 2022-08-09
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“That's why we use these knowledge shields and scientists use the knowledge shield. Scientists don't want to have anything challenge their theories. And so you bring up a data point that's going to make my pet theory look a little bit sketchy. And I'm going to say, well, how did you collect those data? Or I'm not really sure that you've interpreted the data correctly. I'm going to find all kinds of ways of dismissing it. Yes. It leads to fixation because one of our strengths when we encounter a situation is we quickly make sense of it. We use the patterns that we've built up. And it usually works, but it doesn't always work. And so some people say we should discourage people from coming up with immediate reactions. But that's ridiculous.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“So, what you want to have is a way to harness the power of curiosity when something didn't happen the way it was supposed to. And our natural tendency is to dismiss it, to ignore it, and to explain it away. And you think, well, why can't people think like scientists? Because scientists are careful. But the researchers who have investigated this find that scientists do the same thing. They have something that's called knowledge shields the way all of us do, that a variety of techniques for explaining away inconvenient data and anomalies.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“There's got to be some sort of mystery about something happened that wasn't expected. And then the people listening to the story want to know how it wound up. Second is the storyteller should be somebody who likes to tell stories. And I've encountered some people who don't like to tell stories. And I'll ask them about stories and they say, I can't remember any stories. So either they're avoiding the issue or their brain isn't wired to be able to access stories. Or some people think they're telling a story when they just give me a narrative. This happened and that happened and we go through what happened, but there's no story. It doesn't move. It doesn't evolve. There's no transformation. And the good stories...”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“Stories are doing all of those things and they're doing it in a In a way that's really engaging. Like here's what happened and I didn't know what to do about it. And then you want to know what did I do about it? How did it end? And I know when people have... recounted stories in a couple of groups that we studied when we were doing this project, we'd say okay, so what's the takeaway from the story? And everybody had a different takeaway. So that's the richness of stories is what have you learned from the story? What are the implications? And then we, as a small team, just compare notes and realize that because we have different mental models, we have different interpretations of the story and the importance of the story.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“Where did you come up with this abstraction? What happened? Tell me what went on that changed your mind. And then I can ask you about that event. So you were in this situation and something happened that surprised you. What happened? And why were you surprised? And how did you notice it? What sense did you make of it? Stories are not hampered by the limitations of language because now we have an incident account that we can drive. And many organizations don't take advantage of stories. And we've just finished a project for a petrochemical group about using stories to try to get at these kinds of expertise and communicate them.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“That feels reasonable. I think to do that kind of learning, you're sort of exposing a problem where I get your abstraction, but I haven't lived your experience. So abstraction, I might draw the wrong lesson from your abstraction. I don't know where it's coming from. And I could ask you to explain it. And language is a means for communication. But it's also a means for miscommunication because there's all kinds of ambiguity in the words that we use. And people use the same words and mean different things and then believe that they mean the same thing and can get themselves in trouble that way. So that's a limitation of having you explain your abstraction. A way around that limitation is to ask you about stories.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“What kinds of mistakes they might make, what kinds of flawed beliefs they might have so that you need to be more careful as you've described what they're supposed to do. So my idea of a mental model started out with how something works and now it includes all four of these components, how it works, what its limitations are, how to work around them, and how to anticipate confusion on the part of people who are going to be working with.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“Where can it go wrong? And people who are experienced are well aware of these boundary conditions. And so that's part of the richness of their mental model. And then we found another component, which is what are the workarounds that I can use if we run into one of those limitations? How can I recover from it? And then we found that there was a fourth component, we have actually a matrix of this two by two matrix. The last component is somebody with a good mental model can also anticipate where people might get confused.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“This could be another rabbit hole. But a few years ago, my colleague Joseph Borders and Ron Vizion and I did a study of methyl models in petrochemical plants. And going into this study, I thought that a mental model was a set of beliefs about how something works, whether it's a piece of equipment, a piece of machinery, it's an organization. Here's how it works. Here's what happens. Here's the kinds of components and how they fit together. So that was my belief going in. And we spent a week at a petrochemical plant running people through a challenging scenario to capture their mental model. And that was part of their mental model, but only part. We found that there was another part of the mental model, not just how does it work, but what are its limitations, what are its boundary conditions.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“Right. Both of those are important issues. The act of journaling is a way of helping you inquire better and be curious about things that you hadn't thought about. Yes, all of those are all of the above. I would like to see the decision journal when we're finished here.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“I think the ideal decision journal is a great idea. And I hadn't heard of that before. I would like to know when the employee is making a decision, what is the decision? What are the goals that the employee wants to achieve? The primary goal, but there may be other goals that the employee is aware of. What are the prime pieces of information the employee is using to make the decision? Who are the other people or teams that are going to be affected by this decision? Those are the things that I'd like to examine. And then in retrospect, we can see their cues that I should have been paying attention to than I wasn't. Are there goals that I should have been thinking about that I wasn't? But you have it in the decision journal. you have a record of what was the person thinking back at the time. I think it's a great idea.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“And I come to the meeting with what I think were the decisions. The employee comes with their decisions. Then we compare notes. And then we look at the decisions and say, which ones worked and which ones didn't. Now, some decisions may have been good decisions that didn't work through no fault of the employee. And some may have been bad decisions that worked because the employee got lucky. So you can't just look at the outcome, but you have to look at what was the person thinking about when they made the decision and what can we learn whether it was a success or a failure. And my experience is that this decision scorecard method of evaluating people is much less stressful. And the employees and I enjoy it because we're all Learning a lot as we go through it.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“You're exactly right. So what you're saying is that we need to go beyond a simple surface explanation of why people did things. My colleagues and I talk about as cognitive interviews to get beyond the surface explanation to find out how are you sizing this situation up? What were you noticing? And what inferences were you drawing? And we can be asking people those kinds of questions to get into their head more deeply rather than I figured the market was due for a correction or something like that. I thought about an idea called a decision scorecard for an employee. I sit down with the employee and actually I've used it several times and I say let's go back in a previous year what were some of the major decisions. you make.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“And some of them are going to get it right by chance. And people assume it's because they know what they're doing. And in fact, they're just the lucky ones. So you can't rely on performance. And you can't rely on years of experience.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“How do you evaluate their decision making? You run them through a scenario. see what choices they make find out what their rationale is find out the reasons that they're picking one thing over another and use that to determine if if you think that their mental model is rich enough another i told you is ask people tell me tell me about the last mistake you made or or you can you can and should use years of experience just don't take it all that seriously so there's a variety of criteria that you can use you can use their track record have they had a track record of successes but none of these are foolproof i may have um i may be an investor and i have a track record of calling the market right the last five times and people say wow he really knows what he's talking about but there are lots of people who are trying to predict which way the market is going to go and”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“And they weren't intending the consequences to be used to evaluate people, but because they used it in that way, nobody wanted to do any more of that training.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“really did get it. He understood what the relationships were and the different causal factors and how they work together. Now, the story has a bad ending because they decided the first person, despite all their enthusiasm for him, he wasn't ready. He wasn't going to be safe operating a panel of a petrochemical plant that could blow up. I mean, these are volatile compounds and lots of heat. And it's a dangerous environment. They said he's not ready. He's going to have to continue outside for another year before he can use him on the panel. And the second person was somebody that they were ready to use. And after they did that, nobody wanted to engage with the decision-making exercise. They realized if you don't do well, it can have consequences.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“side checking valves and climbing ladders and things like that now they're inside working the panels so they they've been promoted and there were two people who had just been promoted and he said let's get these people up to speed more quickly and let's have them do these these decision making exercises and one of the people with somebody everybody felt good about he just carried himself with great confidence and people were excited about about working with him and the other one was was sort of tentative about things and they they worried about him and they ran them through the decision making exercise and what did they find the first person didn't really get it he didn't really understand critical dynamics of the plant and the second person for all of his caveats”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“And if I express reservations and I use qualifiers, I think this is the case, but I'm not sure. I'm essentially telling people I'm not entirely an expert here. And so I'm disqualifying myself. I'll give you an example. A friend of mine was excited about a scenario-based technique for training using decision-making exercises. And that's a precursor for the shadow box method that we're using now. And so he had some exercises that he had developed. He worked in a petrochemical plant. And he had about, oh, 30 years of experience there. And he said, this would be great. Let's use this for people who have been working at.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“Right, that's a really important question. And I think most people do a mediocre job of determining who they can rely on. And a lot of people go on surface characteristics. So if I carry myself, I stand up strange than usual. I carry myself with confidence. People are going to listen to me.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“To me that means this person may be competent, but it certainly is not an expert. Experts are well aware of their mistakes and their mistakes eat at them until they can sort of imagine what I should have done is this and figure out a way around it. So experts are highly aware of mistakes, but people who are journeymen, many of them stay as journeymen because they want to move on and forget about their mistakes.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“Right, there is a difference. And so one of the things I've examined is how do you know who's an expert? And I identified about seven or eight criteria. And the discouraging part is none of them are foolproof. There is no gold standard for who's an expert. But years of experience certainly contribute to expertise, but it isn't the same thing because there are people who just don't reflect on what happened. They don't learn about what happened. One of the other criteria I use if I'm going to find gave somebody's expertise is I'll ask them, tell me the last mistake you made. Let's talk about that. If the person says, I can't think of any mistakes.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“Another issue in organizations, the people at the top who are calling the shots don't want to rock the boat because they're getting ready to retire, they're getting ready to draw a pension. My sense, I don't have data on this, is they tend to be risk averse. They want to keep things going smoothly until they reach the end of their career at the organization.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“Right. So experience, I still found that most of the insights in the sample that I studied would not have arisen unless people had experience. I mean, Darwin didn't just sit in his study and imagine he was on the bego. He was watching these various species. He was looking at the variation. He was building his experience base. So experience is important. There are other things there are other factors that are operating and experience is not infallible here. And the people who are calling the shots are risk averse.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“You know, we tried that and it didn't work. This may be the time for it to work. Maybe the technology is mature. Maybe the world situation has changed. Who knows? But the people who are calling the shots can remember times when this was tried in the past and it failed. And often that's a good enough reason to squelch the idea.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“some part of the organization that could re-examine, re-examine the idea. Because let's take an organization that wants to come up with new ideas for whatever it is that they're doing. Who's running the organization? People with lots of experience. And I found in my research, that experience is essential for coming up with insights. However, people with experience have lots of scar tissue, of things that got tried before and failed. And if you look at innovations, usually there's a bad track record. People trying it and failing and then somebody else trying it and failing until eventually it succeeds. But until you get to the point where it succeeds, you're going to have people, the experienced people, the senior people in the organization, what are they going to say? They're going to say,”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“And you bring it to your boss and it goes up the chain. It only takes one person in that chain to say, now we better not do it. And then the idea is rejected. And good ideas are fragile and precious and easily discarded. So they could give you an option for a review if I think the idea has been rejected prematurely. I can not really appeal. I don't want to make trouble, but there could be somebody, there could be”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“can you do as an organization they are unless enthusiastic if you have an idea that's a bad idea and you make a mistake that's the down arrow everybody can see it right it's public So there's a lot of cost for making a mistake. If you fail to make an insight, nobody will know. So the reward structure always favors the down arrow. Organizations, if they're really serious about this, Can try to create a mechanism so that right now, if I have an idea, I bring it to my boss. I bring it to you.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“You can't be curious about everything because then you just waste tons of time, but you can be a little curious for a few seconds and say, I wonder what that means. And think about it and imagine it and just get yourself into that kind of a mindset of wondering and speculating rather than recoiling at something that's unfamiliar. So that's what you could do as an individual.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“You can be deliberate like you're describing now. You can say, I want to have more insights, which means taking an insight stance rather than a stance that requires at something new and unexpected and jarring, you can become curious. You can start to celebrate your own insights because if you go through the day, we often have very small insights, things we didn't know, and we make these discoveries. And we just sort of, you know, cruise right past them and use them. But if we make a mistake, we beat ourselves up, right? At least I know I beat myself up. How could I have been so stupid? But I'm less likely to say, gee, that was really clever of me or of people that I'm working with. So you can start attuning yourself to the insights you have and the insights people around you have so you're more sensitive to that.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“Yes, they like predictability. To be a good manager, you want things to run smoothly. And insights are not ways of running smoothly. Insights are disorganizing and disruptive. And so that's a major reason that organizations, without even intending to block the insights that come their way.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“And they want innovation and they really don't. But they're being serious. They think they want insights. But insights are disorganizing. As you point down, insights make you change the way you think, make you change all kinds of things, and they may not be right. And so most organizations actually inhibit insights when people bring up new ideas, they tend to stifle them. And so that's a big problem”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“And that's the problem and that's what's hanging you up. And that's the kind of insight most researchers study because it's easy to study in a laboratory. And the way we get past that flawed belief, we found about 27 cases of that kind of pathway. And usually what happened was there was a hint. Something happened that made them investigate further. And so instead of dismissing the anomaly, which is what we usually do, they became curious about it. So in terms of insight, in terms of changing our beliefs, that's the secret sauce for this kind of pathway, or even practically for any of the pathways, is to become curious about things that don't make sense. Now, what blocks insight in organizations And this was the really discouraging part of my research on insight. What blocks insight is that organizations say they want insight.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“He says, who flicks the ashes in a brand new BMW? That doesn't make sense. Something is off here. So they light them up and pull the car over. Sure enough it was a stolen car. So that's a second type of insight, which is a contradiction insight, where something happens that you didn't expect. Now, this didn't force anybody to revise their mental model or their thinking. It just allowed them to investigate further. Now your question is, where is the resistance coming from? And that's the third type of pathway that we identified, the correction pathway where you're stuck because you have a flawed belief.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“And there you do have to change what you believe or wonder what's going on. And the example I used there is a story I heard from a police officer who was driving around with a partner who is in his first year. And they're stuck in traffic. There's a red light. And the partner, this young guy, looks at the car ahead, which is a new BMW. And he sees the driver take a deep drag on his cigarette. and then flick the ashes.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“Increase them. And I try to find a common theme, and I couldn't because it turned out that there were three different pathways. One pathway is a connection pathway where we put things together. So there, we're not contradicting anything. I'm going back to your original question. We're not upsetting our mental model. We're just putting different ideas together. Darwin, knowing that there was a change in species, but what's driving the change? He reads malthus' book about population growth and competition for scarce resources. He read the sixth edition. And he puts that together and he said, that's what's driving evolution is a competition for scarce resources. So that's a connection type of insight. A second type of insight is a contradiction insight where something happens. That doesn't make sense”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“But then they would ask me the question, what do you know about insights? And I would answer, I don't know anything about insights. I just have this one slide. I haven't studied it. And once I used a slide in a talk I gave in Singapore, and I got asked that question, and then I flew home afterwards. And it's a 17-hour flight. I had a direct flight. That's a long time to be stewing over the fact that I felt like an idiot not being able to tell them what insights they're about. So I decided to do a study. So I collected 120 examples of insights to see what's going on and hopefully find out how to improve them.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“An insight definitely is going to be incompatible with beliefs we hold. That's why we're surprised. I did a study of 120 examples of insights. I wondered where insights came from. And I did this study out of frustration because I would give talks and I would say, if you want to improve performance, there's two things you can do. And there would be like a down arrow, what you want to reduce and an up arrow what you want to increase. And to improve performance, what you want to reduce is errors. And that's what most organizations do. They try to cut down on errors. But they're missing the up arrow of improving insights. And so most organizations only focus on reducing errors. So I would present this slide, and people in the audience would say, yes, that makes sense. That's my organization. All they care about is reducing errors.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT
“One of the things I've examined is how do you know who's an expert? And I identified about seven or eight criteria. One of the criteria I use from somebody's expertise is I'll ask them, tell me the last mistake you made. Let's talk about that. If the person says, I can't think of any mistakes. To me, that means this person may be competent, but it certainly is not an expert. Experts are well aware of their mistakes and their mistakes eat at them. But people who are journeymen, many of them stay as journeymen because they want to move on and forget about their mistakes.”
2022-08-09 · The Knowledge Project with Shane Parrish · Gary Klein: Insights For Making Better Decisions · IDENTIFIED FROM THE TRANSCRIPT