YouSaid · the spoken record

Noam Brown

lines on the record
196
first
2022-12-06
most recent
2022-12-06
sittings or episodes
1
sources
podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

  1. “So, one of the things I'm really excited about with diplomacy, there's never really been a good domain to investigate these kinds of questions. And diplomacy gives us a domain where trust is really at the center of it. And it's not just like you've hired a bunch of mechanical turkers that are being paid and trying to get through the task as quickly as possible. You have these people that are really invested in the outcome of the game and they're really trying to do the best that they can. And so I'm really excited that we're able to, we actually have put together this, we're open sourcing all of our models. We're open sourcing all of the code and we're making the data that we've used available to researchers so that they can investigate these kinds of questions.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  2. “Yeah, I mean, I think when people play diplomacy for the first time, they approach it as a game of deception and lying and they ultimately, if you talk to top diplomacy players, what they'll tell you is that diplomacy is a game about trust and being able to build trust in an environment that encourages people to not trust anyone. So that's the ultimate tension in diplomacy. How can this AI reason about whether you are being honest in your communication? And how can the AI persuade you that it is being honest when it is telling you that, hey, I'm actually going to support you this turn?”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  3. “We actually found that lying would make the bot perform worse in the long run. It would end up with a lower score because once the bot lies, people would never trust it again. And trust is a huge aspect of the game of diplomacy.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  4. “Yeah, and we're filtering for several things. We're filtering is this a sensible message? So sometimes language models will generate messages that are just like totally nonsense. And we try to filter those out. We also try to filter out messages that are basically lies. Diplomacy has this reputation as a game that's really about deception and lying. But we try to actually minimize the amount that the bot would lie. This was actually mostly a...”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  5. “That's a neural network. So we're well, it's a combination. It's a neural network, but it's also using planning. It's trying to compute what is the policy that the other players are going to play given that this message has been sent. And then is that better than not sending the message or not?”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  6. “I mean, there are a lot of ways that this could fail. So, for example, I mean Could have a situation where you're basically we don't tell the language model like here are the pieces of our action or the other person's action that you should be communicating and so like let's say you're about to attack somebody you probably don't want to tell them that you're going to attack them but there's nothing in the language like the language model is not very smart at the end of the day so it doesn't really have a way of knowing like well what should I be talking about should I tell this person that I'm about to attack them or not So we have to like develop a lot of other techniques that deal with that. Like one of the things we do, for example, is we try to calculate if I'm going to send this message, what would I expect the other person to do in response? So if it's a message like, hey, I'm going to attack you this turn, they're probably going to, you know, attack us or defend against that attack. And so we have a way of recognizing like, hey, sending this message is a negative expected value action and we should not send this message.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  7. “Basically, like one message at a time. So we'll feed into the dialog model here are the actions that you should be discussing. Here's the content of the message that we would like you to send. And then it will actually generate a message that corresponds to that.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  8. “The way that RL and planning is done is actually not using language. We're coming up with this plan for the action that we're going to play and the other person is going to play, and then we feed that action into the dialog model that will then send a message according to those plans.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  9. “A combination of reinforcement learning and planning, actually very similar to how you approached poker and how people approached chess and Go as well. We're using self-play and search to try to figure out what is an optimal move for us and what is a desirable move that we would like this other player to play. Now the difference between the way that we approached reinforcement learning and search in this game versus those previous games is that we have to keep it human compatible. We have to understand how the other person is likely to play rather than just assuming that they're going to play like a machine.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  10. “So that's really the crux of the problem. How do we leverage the benefits of self play that have been so successful in all these other previous games while keeping the strategy as human compatible as possible? And so what we did is we first trained a language model and then we made that language model controllable on a set of intents, what we call intents, which are basically like an action that we want to play and an action that we would like the other player to play. And so this gives us a way to generate dialogue that's not just trying to imitate the human style, whatever a human would say in the situation, but to actually give it an intent, a purpose in its communication. We can talk about a specific move or we can make a specific request. And the determination of what that move is that we're discussing comes from strategic reasoning model that use.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  11. “Yeah, and honestly, when you look at what makes a good human diplomacy player, obviously they're able to handle themselves in games with other expert humans, but where they really shine is when they're playing with these weak players. And they know how to take advantage of the fact that they're a weak player, that they won't be able to pull off a stab as well, or that they have certain tendencies and they can take them under their wing and persuade them to do things that might not even be in their interest. The really good diplomacy players are able to take advantage of the fact that there are some weak players in the game.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  12. “That's a great question. And so I think this kind of goes into how do you measure the performance in diplomacy? And I would argue that when you're measuring performance in a game like this, you don't actually want to measure it in games with all expert players. It's kind of like if you're developing a self-driving car, you don't want to measure that car on the road with a bunch of expert stunt drivers. You want to put it on a road of an actual American city and see is this car crashing less often than an expert driver would So that's the metric that we've used. We're saying we're going to stick this game. We're going to stick this bot in games with a wide variety of skill levels. And then are we doing better than a strong or expert human player would in the same situation?”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  13. “Ultimately, the problem is simple to quantify, right? Like you're going to play this game with humans and you want your score on average to be as high as possible. If you can say, I am winning more than any human alive, then you're a champion diplomacy player. Now, ultimately, we didn't reach that. With real humans online, the bot came in second out of all players that played five or more games. And so not like number one, but way, way higher than one.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  14. “And so honestly, I was kind of reluctant to work on it at first because I thought it was too far out of the realm of possibility. But I was talking to a co-worker of mine, Adam Lehr, and he was basically saying like, eh, why not aim for it? We'll learn some interesting things along the way. And maybe it'll be possible. And so we decided to go for it. And I think it was the right choice considering just how much progress there was in AI and that progress has continued in the year since.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  15. “Look at a game like chess or a game like Go, they took decades for researchers to ultimately reach superhuman performance at. I mean, like chess took 40 years of AI research. Go took another 20 years. And we thought that diplomacy would be this incredibly difficult challenge that could easily take a decade to make an AI that could play competently. But we felt like that was a goal worth aiming for.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  16. “Before I get to that, one thing I should explain why we decided to work on diplomacy. So basically, what happened is in 2019, I was wrapping up the work on six player poker on Pluribus and was trying to think about what to work on next. And I had been seeing like all these other breakthroughs happening in AI. I mean, like 2019, you have StarCraft, you have Alpha Star beating humans in StarCraft. You've got the Doda 2 stuff happening at OpenAI. You have GPT-2 or GPT-3. I think it was GPT-2 at the time. And it became clear that AI was progressing really, really rapidly. And people were throwing out these other games about what should be the next challenge for multi-agent AI. And I just felt like we had to aim bigger.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  17. “Think that's true that if you're going to play in this human game, you have to somehow adapt to the human surroundings and the human play style.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  18. “Think it's different from the Turing test, like the way that the Turing test is formulated, it's about trying to distinguish a human from a machine and seeing, oh, could the machine successfully pass as a human in this adversarial setting where the player is trying to figure out whether it's a machine or a human? Whereas in diplomacy, it's not about trying to figure out whether this player is a human or a machine. It's ultimately about whether I can work with this player, regardless of whether they are a human or machine. And can the machine do that better than a human can?”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  19. “At least you have to understand how humans play the game so that you can understand how to work with them. If they have certain expectations about what does it mean to be a good ally, what does it mean to have like a reciprocal relationship where we're working together, you have to abide by those conventions. And if you don't, they're just going to work with somebody else instead.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  20. “So, what you have to do is incorporate human data. And to kind of give you some intuition for why this is the case, imagine you're playing a negotiation game like diplomacy, but you're training completely from scratch without any human data. The AI is not going to suddenly figure out how to communicate in English. It's going to figure out some weird robot language that only it will understand. And then when you stick that in a game with six other humans, they're going to think this person's talking gibberish and they're just going to ally with each other and team up against the bot or not even team up against the bot, but just not work with the bot. And so in order to be able to play this game with humans, it has to understand the human way of playing the game, not this machine way of playing the game.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  21. “Yeah, and I mean, there's the natural language aspect. And then even besides the natural language aspect, you also have the cooperative elements of the game. And I think this is actually something that I find really interesting. If you look at all of the previous game AI breakthroughs, they've all happened in these purely adversarial games where you don't actually need to understand how humans play the game. It's all just AI versus AI, right? Like you look at checkers, chess, Go, poker, StarCraft, Dota 2. Like in some of those cases, they leverage human data, but they never needed to. They were always just trying to have a scalable algorithm that then they could throw a lot of computational resources at, a lot of memory at, and then eventually it would converge to an approximation of a Nash equilibrium, this perfect strategy that in the two players there are some game guarantees that they're going to be able to not lose to any opponent.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  22. “Right. I mean, there's the complexity and understanding when somebody is saying this to me, what does that mean? And then there's also the complexity of like, should I be telling this person this? Like I've overheard these whispers. Should I be telling this person that like, hey, you might be getting attacked by this other power?”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  23. “If you look at the next line, so the person saying, like, I've heard bits about a Lepanto and an octopus opening and basically telling Austria like, hey, just a heads up, you know, I've heard these whispers about what might be going on behind your back.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  24. “Yeah, that's just kind of like the generic readings at the beginning of the game. I think the meat comes a little bit later when you're starting to talk about like. Specific strategy and stuff.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  25. “Simplified language to make this complexity smaller, but ultimately we thought the most impactful way of doing this research would be to address the natural language component head on and just try to go for the full game up front.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  26. “The real difficulty is the breadth of things that you can talk about. You can have natural language in other games. Settlers of Katan, for example, you could have a natural language settlers of Katan AI. But the things that you're going to talk about are basically like, am I trading you too sheep for a wood or three sheep for a wood? Whereas in a game like diplomacy, the breadth of conversations that you're going to have are like, you know, am I going to support you? Are you going to support me in return? Which units are going to do what? What did this other person promise you they're lying because they told this other person that they're going to do this instead? If you help me out this turn, then in the future I'll do these things that will help you out. The depth and breadth of these conversations is really complicated. And it's all being done in natural language. Now you could approach it, and we actually consider doing this.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  27. “So there's a lot of aspects in diplomacy that make it a huge challenge. First of all, you have the natural language component. And I think this really is what makes it really the most difficult game among the major benchmarks. The fact that you have to, it's not about moving pieces on the board. Your action space is basically all the different sentences that you could communicate to somebody else in this game.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  28. “Yeah, so people have been working on AI for diplomacy since about the 80s. There was some really exciting research back then, but the approach that was taken was very different from what we see today. I mean, the research in the 80s was a very rule-based approach, kind of a heuristic approach. It was very in line with the kind of research that was being done in the 80s. Basically trying to encode human knowledge into the strategy of the AI. And it's understandable. I mean, the game is so incredibly different and so much more complicated than the kinds of games that people were working on, like Chess and Go and Poker, that it was honestly even hard to like start making any progress in diplomacy.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  29. “So, if somebody is able to actually gain control of a majority of the map, then they've won the game. And that is a solo victory, as it's called. Now that pretty rarely happens, especially with strong players, because like I said, the game is designed to incentivize the other players to put a stop to that and all work together to stop the superpower. Usually what ends up happening is that all the players agree to a draw and then the score, the win is divided among the remaining players. There's a lot of different scoring systems. The one that we used in our research, basically gives a score relative to how much control you have of the map. So the more that you control, the higher you score.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  30. “Usually, the game lasts, I would say about 15 or 20 turns. There's in theory no limit. It could last longer. But at some point, I mean, if you're playing a house game with friends, at some point you just get tired and you all agree like, okay, we're going to end the game here and call it a draw. If you're playing online, there's usually set limits on when the game will actually end.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  31. “True, yeah, so Russia starts off with four units while all the other players start with three, but Russia is also in a much more vulnerable position because they have to have a lot more neighbors as well.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  32. “So I think the general consensus is that France is the strongest power in the game. But the beautiful thing about diplomacy is that it's self-balancing, right? So the fact that France has an inherent advantage from the beginning means that the other players are less likely to work with it.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  33. “So, the story that I've heard for the creation of the game is it was created by somebody that had looked at the history of the 20th century and they saw World War I as a failure of diplomacy. They saw the fact that this war broke out as the diplomats of all these countries really failed to prevent a war. And he wanted to create a game that would basically teach people about diplomacy. And it's really fascinating that in his ideal version of the Game of Diplomacy, nobody actually wins the game because the whole point is that if somebody is about to win, then the other players should be able to work together to stop that person from winning. And so the ideal version of the game is just one where nobody actually wins. And it kind of has a nice wholesome take-home message then that, you know, war is ultimately futile.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  34. “So, the game was created in the 50s And from what I understand, it was JFK's, it was played in like the JFK White House. Henry Kissinger's favorite game. I don't know if it's true, but that's definitely what I've heard.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  35. “Tension is absolutely core to the game. The fact that you can make all sorts of promises, but you have to reason about the fact that like, hey, they might not trust you if you say you're going to do something, or they might be lying to you when they say they're going to support you”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  36. “Could be, they could be. And this is where the interesting aspect of the game comes in because Can support your own units into territory, but you can also support other people's units into territories. And so that's what the negotiations really revolve around.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  37. “Support is like it's an action that you can issue in the game. So you can say this unit, you write down this unit is supporting this other unit into this territory.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  38. “Kind of like risk where the map is divided up into 50 different territories. Now you can enter a territory if you're moving into that territory with more supports than the person that's in there or the person that's trying to move in there. So if you're moving in and there's somebody already there, then if neither of you have support, it's a one versus one and you'll bounce back and neither of you will make progress. If you have a unit that's supporting that move into the territory, then it's a two versus one and you'll kick them out and they'll have to retreat somewhere.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  39. “Yeah, so on every turn, you got a bunch of different units that you start out with. So you start out controlling just a few units. And the object of the game is to gain control of a majority of the map. If you're able to do that, then you've won the game. But like I said, the only way that you're able to do that is by working with other players. So on every turn, you can issue a move order. So for each of your units, you can move them to an adjacent territory, or you can keep them where they are, or you can support a move or hold of a different unit.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  40. “Yeah, so some players do go this route where they just like kind of view it as a strategy game, but also a role playing game where they can act out like, what would I be like if I was a leader of France in 1900?”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  41. “That's right, you can make all sorts of, you can talk about anything, you could say, like, hey, let's have a long term alliance against this guy. You can say, like, hey, can you support me this turn in return? I'll do this other thing for you next turn. Or, you know, yeah, just you can talk about what you talked about with somebody else and gossip about what they're planning. The way that I would describe the game is that it's kind of like a mix between risk, poker, and the TV show Survivor. There's like this big element of trying to social element. And the best way that I would describe the game is that it's really a game about people rather than the pieces.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  42. “So, there's different ways to play the game. You know, you can play it in person, and in that case, it's all natural language, freeform communication. There's no constraints on the kinds of deals that you can make, the kinds of things that you can discuss. It can also play it online. So you can send along emails back and forth. You can play it live online or over voice chat. But the important thing to understand is that this is unstructured communication. You can say whatever you want. You can make any sorts of deals that you want. and everything is done privately. So it's not like you're all around the board together having a conversation. You're grabbing somebody going off into a corner and conspiring behind everybody else's back about what you're planning.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  43. “Right, yeah. So, the way the game works is on each turn you spend about five to 15 minutes talking to the other players in privates and you make all sorts of deals with them. You say like, hey, let's work together, you know, let's team up against this other player because the only way that you can make progress is by working with somebody else against the others. And then after that negotiation period is done, all the players simultaneously submit their moves and they're all executed at the same time. And so you can tell people like, hey, I'm going to support you this turn, but then you don't follow through with it. And they're only going to figure that out once they see the moves being read off.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  44. “So I talked a lot about two player zero sum games. And what's interesting about diplomacy is that it's very different from these like adversarial games like chess, Go, Poker, even StarCraft and Dota. Diplomacy has a much bigger cooperative element to it. It's a seven-player game. It was actually created in the 50s, and it takes place before World War I. It's like a map of Europe with seven great powers, and they're all trying to form alliances with each other. There's a lot of negotiation going on. And so the whole focus of the game is on Forming alliances with the other players to take on the other players.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  45. “Exactly. And I think a lot of the old school poker players have just kind of given up on that aspect. And I got to give Dan Negroni credit for keeping up with all the developments that are happening in the sport.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  46. “And I'm gonna give him credit because he is one of the few old school really strong players that have kept up with the development of AI.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  47. “So, my feeling is that it's a difficult question because just like in chess, where you can't really compare Magnus Carlson today to Gary Kasparov because the game has evolved so much. The poker players today are so far beyond the skills of people that were playing even 10 or 20 years ago. So you look at the kinds of all-stars that were on ESPN at the height of the poker boom. Pretty much all those players are actually not that good at the game today. At least the strategy aspect. I mean, they might still be good at reading the player at the other side of the table and trying to figure out, are they bluffing or not? But in terms of the actual computational strategy of the game, a lot of them have really struggled to keep up with that development. Now, so for that reason, I'll give an answer and I'm going to say Daniel LeGranio, who you actually had on the podcast recently. I saw it was great.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  48. “A very noisy signal. Now, there is a signal there, and so you could do this calculation, it would just be much harder. But the same way that AIs have now taken over chess and all the top professional chess players train with AIs, the same is true for poker. The game has become a very computational people train with AIs to try to find out where they're making mistakes, try to learn from the AIs to improve their strategy. Now, yeah, so the game has been revolutionized in the past five years by the development of AI in this sport.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  49. “Think it is much harder to estimate something like elo rating in poker. I think it's doable. The problem is that the game is very high variance. So you could play, you could be profitable in poker for a year and you could actually be a bad player just because the variance is so high. I mean, you've got top professional poker players that would lose for a year just because they're on a really bad streak.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source

  50. “Yeah, so the way the value functions work in, like the latest and greatest poker AIs, they do use neural nets for the value function. The way it's done is very different from how it's done in a game like chess or Go because in poker you have to reason about beliefs. And so the value of a state depends on the beliefs that players have about what the different cards are. Like if you have pocket aces, then whether that's a really, really good hand or just an okay hand depends on whether you know I have pocket aces. If you know that I have pocket aces, then if I bet you're going to fold immediately. But if you think that I have a really bad hand, then I could bet with pocket aces and make a ton of money. Value function in poker these days takes the beliefs as an input, which is very different from how chess and GoAIs work.”

    2022-12-06 · Lex Fridman Podcast · #344 – Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation · IDENTIFIED FROM THE TRANSCRIPT · source