YouSaid · the spoken record

Jimmy Soni

lines on the record

69

first

2024-06-26

most recent

2024-06-26

sittings or episodes

1

sources

podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

1. “There you go. Maybe somebody working on anthropic nose. And they're like, no, no, no. It's actually Jean-Claude Van Damme. They just thought that was really cool.”

   2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

2. “Yeah, I mean, I'm not that active on X. I mean, I'm as active as I need you, but I love hearing from readers, and you can just find me. I have a website, Jimisony.com, but just find me on X and DM. I love hearing from folks who are in this space. And I would just say, I really appreciate your enthusiasm for Shannon. That's actually one thing that unites the two worlds. The people I interviewed knew who Claude Shannon was. Like Elon references Claude Shannon Onyx from time to time because these ideas still last decades later. And there is something really inspiring about that. I mean, like ultimately, all of us want to do work that stands the test of time. And so it's interesting to me that Shannon has had kind of a second life. The only other thing I would say is I am quite curious if Anthropics Claude is named after Claude Shannon. And nobody's been able to pin that one down. And so I'd be really like, if any of your listeners or you or somebody put a signal into the universe and figure that one out, I would be really grateful. That'd be cool.”

   2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

3. “A lot of the people writing about Elon are not engineers. And so the problem is they're writing about things, but they don't have any kind of understanding of how those things work. And so it's easy to look at it something like, I'm going to lay off 80% of Twitter and say, that's terrible. It's another thing for somebody who has been the CEO of at that point two internet companies, Zip2 and Pip, to look at a situation and say, well, here are the things that matter in this business. Here are the things that don't matter. And I need to cut costs because that's part of the issue here is that it's just things are deeply inefficient. I can't be, I don't know enough about that situation to know what was right and what was wrong and what was executed well and what was executed poorly. But I think that one caution I would give to listeners who are reading things about Elon is to remember that most of the people writing about him don't have an engineering background. It's actually a real deficit when you're writing about things like rocket engineering or car design or anything like that.”

   2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

4. “And it's hard, right? Because he's one of the most covered subjects in the world right now. And by the way, like my book is not, it doesn't like a book where I'm trying to praise him or trying to criticize him. I just went where the facts took me. So I interviewed a lot of people. I had a bunch of email. I had about six gigabytes of email. That era that someone shared with me. So I had a day by. So, for people who were listening, maybe the reason to read the book is because you can read Elon's resignation note. You don't even have to take my word for it. Like it's all there. It's all just facts. But I would say that one of the things that happens is that people underestimate his willingness to do whatever it takes to have his companies be successful. and the way that that kind of focus inspires engineers to do their best work too.”

   2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

5. “Respect is the difference between respecting somebody and giving them a $15 million check for their rocket company. And so you're right on the broad outline, which may have been some temporary misgivings about how things happened, but over time, there's a partnership there that has actually withstood a lot.”

   2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

6. “And it comes from Founders Fund. And it is one of the parts of the story that people miss, which is they did have a feud. They did have a moment where Peter replaced Elon as CEO. There was a lot of bitterness on both sides. If you were to talk to Peter and Max, they thought they were doing the right thing. If you talk to Elon, they thought they were doing the wrong thing. But it is the case that Peter invests in Elon and more than that, like invest in Elon and will often Companies and industries that Elon is in because I know what it's like to compete against Elon and you're going to lose relentless. And so it's interesting because it isn't just that they're”

   2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

7. “Of the And it was actually when SpaceX, not at a billion, it was, I think it was $14 million or $20 million or something.”

   2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

8. “The most surprising thing about PayPal is this company should have gone under. None of this should have happened and it stayed alive. And that's the cool part of the story is how do you keep a company alive when you're under siege and the pressure's on and everything is like chaos. And there's not low interest rates and there's not easy money and there's not hundred million dollar rounds that are being thrown at people. To my mind, that's actually like one of the cool things about this. And it's one of the things that this team walked away with. Is they built something successful, but it came during a period of real hardship And it meant that every subsequent thing they did, YouTube, Yelp, Palantir, SpaceX, Tesla, LinkedIn, that they were accustomed to this difficulty. And that's probably a good way to preview that.”

   2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

9. “Coming, they get sued, they get hacked, people try to defraud them. They're in this epic battle with eBay. They are in one of the worst markets for IPOs after 9-11. There wasn't an IPO for three or four months because, I mean, 9-11 is like the financial services industry is in rubble, literal rubble. You have two competitors, x.com, which is Elon's company and Confinity, which is Peter Thiel and Max Legends Company. They fight for three months, then have a merger that's basically organized within four or five weeks. And this is all just before the bubble burst. There's no financing left in venture for this kind of company. So I would just say that the surprising thing about PayPal is the mere fact that it survived.”

   2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

10. “The other thing I would say is the fact that this company, PayPal, it's created like as the dot-com bubble is just getting to its peak and months after its creation, March of 2000, the bubble bursts. And all of a sudden, like the Nasdaq loses 86% of its value that year. 86% of its value. Like Barrons is writing like they had this famous cover story. It's like DOT.bom, right? And it was about, oh, Amazon.bom. It was like Amazon.bom. And the idea was like Amazon's share price at the time was $9. And everyone's like, oh, this is it. The internet's like a total hype machine. All nonsense, right? It's all foolishness. September 11th happens during the early creation years of PayPal 9-11. And it massively changes things like KYC rules, know your customer rules and financial regulation because the government suddenly wants to correct down on terrorist financing. There are all of these things that happen.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

11. “And it's this really interesting thing. And I think at the end of chapter 14 is when he leaves the story, because he gets thrown out as CEO. But I think people would really benefit from reading that, especially people who kind of throw stones and don't realize that there was a period in which it wasn't a glide path. There wasn't a $56 billion payday. You need to have a little bit of, I think, humility about the people in our business life who become business celebrities and to appreciate that there's a humanity there, there's some challenge there, there's some struggle there. So that's one story. It's a little surprising.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

12. “He almost dies because he contracts malaria and meningitis while on a trip abroad. And he loses a child. And people don't really know because he's not going to talk about it all the time. But I went pretty deep on the history of that little moment and I wrote about it at the end. And you think about what someone's been through and it looks like these days his life is a glide path. He can say whatever he wants, et cetera. There's this real profound four-year period of difficulty. in which he struggles with the hardest thing that anybody i think could struggle with as a parent myself i can't fathom losing my kid And he has an infant died, a sudden infant, whatever the synchrony is.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

13. “I mean, there's so many. But let me pick a couple. One that people may know, but they should really think a little bit more about, which is there is this temptation these days that I think if Elon ate a piece of fruit, there would be a bunch of hate about that piece of fruit. I feel like if he suddenly got into like raspberries, there'd be like an anti-raspberry coalition or some nonsense like that. And I think that one of the nice things about writing about a very specific and defined period of his life is that I wasn't asking him any questions about SpaceX, Tesla, Twitter, none of that. I was studying the years 1998 to 2002. And one way to think about the years 1998 to 2002 in his life is that it's during this period that he gets kicked out, he gets overthrown at PayPal as the CEO.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

14. “A lot of poker players. There's a lot of that kind of puzzle solving game playing in the ether as well. But it's interesting. It's a fascinating, to me, it's a fascinating story because it's like, who are these people before they become relentless objects of attention today? I find the most interesting thing to me about writing biographies is figuring out who people are before they become rich, famous, or successful. Because to me, I want to know who they are when they're closer to me. I'm trying to solve these Rubik's cues for myself. I don't actually find most of the contemporary coverage on these people all that interesting. Because the truth is, when you're at that level, it's not your life becomes a little bit more boxed in. But when you're in your 20s and nobody knows who you are, the world is a blank page.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

15. “Then he had zip two. I think he had an arcade when he was living in South Africa. And then he had PayPal. So it was like his fourth, maybe his fifth. And so you just have people who are serial entrepreneurs, but minus the word entrepreneur because that sounds, it makes it sound fancier than it is. They were just building businesses and trying to figure it out along the way. And so I would say that's a common thread. I would say the other common thread is there's a lot of game playing, puzzle playing, chess playing. There's a lot of like that in the air at PayPal. There's a bunch of stories in the book about that, but obviously Peter Thiel was a well-known chess player. But I would say a lot of chess Titans within this group.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

16. “You didn't do the internet. So, all these people are kind of tinkering, they're playing with computers, they're doing things, they all move out west, the boom starts. A lot of them study computer science, engineering, not all of them. There are a bunch of people on the business side who are more traditional humanities tracks. But I would say that like one common theme is I noticed that many of them had built businesses before they ever joined PayPal. So for Max Levin, who's one of the co founders, one of the best known co-founders, he is now the CEO and founder of a firm. PayPal was his fifth business and then four failures in college while he was like moonlighting as a student. He was also building businesses.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

17. “There's a lot of people at the heart of the story. So there's a few common threads, but I don't want to paint with too broad a brush because it's like a little tough. I would say that a lot of them were building things on computers when it wasn't cool to do so or when it wasn't lucrative to do so. So remember, the internet economy doesn't really become an economy until the mid to late 90s when Netscape goes public. Before then, the internet is like this thing that academics use and it's the thing that nerds use. It's not a way to become a billionaire. In the 80s in particular, if you wanted to become very, very wealthy, you went into financial services. You didn't go into technology. And if you went into technology, you did hardware, not software.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

18. “And at first, the people who were on the product team were like horrified because they thought this would nuke new signups. And it turned into an innovation that is literally all over the internet. I mean, you can't use most websites these days without having some kind of financial services website, certainly, without having some kind of gate like that. I could go on and on. That group created things in financial services that still affect our lives today. And then I thought it was just interesting to know like before they flew private and ran the world, what were the reasons, who were these people? Who were these people when they were in their early 20s? How did they figure life out? How did they get to Silicon Valley?”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

19. “probably use without knowing it. I'll give you one example CAPTHA tests is these tests where you have to identify that you are a human and not a robot Those were commercialized at PayPal. And the reason they were commercialized at PayPal is because when PayPal designed its system, a bunch of people started creating bots to create fake PayPal accounts so they could get the sign-on bonuses. The company needed to devise a way to separate Preston, flesh and blood human user from like 10 robots Preston created to collect $10 bonuses, right? They did that with the first application of the CAPCHA tool. Surprise. They did it with surprise.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

20. “And I got the sense that there was a story there that hadn't been told because what happened is that these people became so larger than life that the focus of everyone's attention is kind of like, what are you doing today? Not what did you do 20 years ago? So I, through Hook and Crook and a lot of intros and everything else, I interviewed around 300 people and I wrote the book that's called The Founders. And it's about the story of PayPal from 1998 to 2002. So it is the story of creating this payments architecture and this startup during the dot-com bubble bursting and during all these other events happened. But basically it's this crazy four-year sprint to build this company. And it has everything in it from like what you'd expect, like kind of like, how did Elon get the idea to do this company? Why was it called x.com at one point? All of those things. In addition to like there are a lot of things that were invented at PayPal that you”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

21. “And it also has given rise to a group known as the PayPal Mafia that includes people like Elon Musk, Peter Thiel, Reed Hoffman, David Sachs, the founders of Yelp, YouTube, LinkedIn, Yammer, Palantir. I mean, every significant company to come out of Silicon Valley, arguably, for 20 years has some tie back to this ecosystem, whether they're an investor or something or whatever. Many years ago, I think at this point, five or six years ago, maybe actually longer. I got this idea in my head. I was like, everyone's talking about these people, but how come no one's ever gone back and asked them like how they all ended up at the same company at the same time? That seems kind of weird, right? Like, I literally, the question was just like, wait, hold on. Elon and Peter, and all these people worked at the same company. What was in the water? I mean, it was literally the question I was asking. One of my sentences in the introduction to the book, what was in the water there? Because if you think about it,”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

22. “And I'm happy to come back on and chat about it because it's going to be a much longer conversation I suspect in some ways your audience is more familiar with the people who are at the heart of that book than they are with Claude Shannon. Maybe. PayPal is a company that has existed at this point for over 20 years. If you're listening, you've probably used one of their products because they own Venmo now. Obviously, the Core PayPal ecosystem, everything else.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

23. “The information is contained in the surprise, meaning if I can predict it, it's actually not informative by Shannon's definition of information. It's not informative. Information is contained in the surprise. That's one little bit of a very simplified way to think about information.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

24. “Performance at school, and I guess he was going to do a solo performance and some kind of competition or something, and he was going to play the trumpet. And Shannon said, well, would it be interesting if your trumpet breathed fire? And sometimes like, yeah, it would. So Shannon constructed a flamethrowing trumpet that Andrew used at school to perform. Again, with no practical purpose, just because he thought it'd be funny and kind of clever and interesting. And this is, we talked a lot about information theory and about the details and the arcana of the science. It's worth appreciating that a lot of the fun of doing this project was just figuring out kind of the zaniness of Claude Shannon. He was just like somebody who had an idea and did it. There's a part of it that's more endearing because we adult to take ourselves super seriously, right? We all take ourselves a little too seriously. Shannon never did. It was one of those things. I still remain inspired by it.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

25. “The other piece of it I'll tell you is like a bunch of my favorite Shannon stories ironically are not scientific stories, they're parenting stories. One more example. I think it's made it into the book, but I don't know if it did. There was an oversized tree on his property at one point. It was like a big tree. They had a build of land. And the natural thing to do was like it was an aging tree and you got to take it down because it's unsafe or whatever, or there was an eyesore. I don't even remember why they were going to cut it, but they needed to cut it. And Shannon had this thought, instead of cutting it down, what we should do is we should hire some people to basically file the tree down so that it's sort of like a flagpole that's growing in the ground. And then we should carve a skull at the very top of the tree and hoist a pirate flag up this carved tree with the skull on top. And that's exactly what they did. So they built they like fashion the biologist tree down, like got a skull on top, hoisted a jolly roger up the flagpole. And the whole reason they did it, there's no purpose. They just did it because Shannon thought it was funny and kind of interesting. I'll give you another example. One morning, I guess Andrew, whose son woke up, said he's got a”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

26. “that it is information we were trying to conceal from people is what gives us the ability to conceal nothing from people.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

27. “Well, we have to actually figure out a way to do this so that we're not transmitting a ton of data so that it can avoid detection. You have to figure out little tips and tricks and tools to like take a certain word and turn it into something entirely different. Alan Turing was doing this work. Obviously with like ciphers and other things in the UK, Turing and Shannon, funny enough, became friends. There's a part of this that is about encoding and a part of it that interestingly runs through a legacy of code breaking and code making, which I always found was one of the more interesting parts of this, like the idea of secret codes, things that we were trying to hide ultimately lead to a world in which we don't have to hide anything. Anything about yourself, right? It's a kind of cool thing. It's a very interesting historical irony.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

28. “Of what you're saying. And you used the word code as did I because there are codes. There are error correcting codes that are used to communicate digital messages. It's an interesting word, though. And the reason it's interesting is because the other cool part of Shannon's life is what happens right before the information theory paper. He is a code breaker during World War II. And so part of what he's studying, like here's an example. There was a phone system that Churchill used to communicate with FDR. It's called Sig Sally. SIG Sally was designed to develop at Bell Laboratories. I think it was a pretty top secret project. A lot of information has emerged about it after the fact. But Shannon was one of the engineers working on these code breaking devices. He was also somebody who was an important job during the war, was figuring out how can President Franklin Donald Roosevelt send a message to Winston Churchill that cannot be intercepted by the Nazis. And so part of what Shannon has to do is think about”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

29. “There's a lot of promising stuff happening in this world, and I'm not at all an expert in that. But my sense is that ambitious, smart people are trying to figure that out. And they're given a college try, it seems pretty serious.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

30. “You know, computers are mechanical devices. They're not digital devices yet. The digital world is still super far off. And so he eventually becomes an enthusiast. He obviously builds some of these things. He gets an Apple II that was delivered from Steve Jobs. He embraces all of that. But the technologies are very primitive in 1948. So it's a conversation that's a little bit, it's years away. But it is why with the subtitle and with the way that the book kind of was embraced, it's like we really do live in Shannon's world. Everything that is going on today is Shannon's world. And I agree with you, by the way, that I think the information demands based on these technologies are going to be significant. Ironically, I also think there's a world in which these technologies help us figure out that problem, right?”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

31. “Research that for the next 60, 70, and still going on today would create the algorithms that we use to transmit information. So the short answer is not really. And he wasn't really thinking in practical terms about what energy demands would be required. Because remember at the time he's doing this writing, 1948, computers are still like these room size things.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

32. “He had a hugely groundbreaking insight in that this is possible we can statistically do this, but he laid out the theoretical architecture for it. A good analogy or a good way to think about it is when Galileo was basically challenging the church and saying, no, the earth revolves around the sun. The sun doesn't revolve around the earth. This is heliocentrism versus geocentrism. Galileo didn't have to do everything that came after the acceptance of geocentrism, meaning there's a bunch of stuff that we've discovered about the fact that the earth revolves around the sun after Galileo. He wasn't doing all of that, but it was important that the theory of geocentrism be broadly accepted as the truth so that we could do all of these other things in the same way, once Shannon identified that it was possible to do this, it sent the rest of his field off on the”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

33. “And what that did is it was like a starter gun. It basically sent the rest of that domain off on a search to find the codes that would most efficiently compress information that would reach the Shannon limit, that would enable the technologies of today. Shannon didn't do that.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

34. “It's a great question, and in some ways it's a very simple answer, which is Shannon was in the following great position or privileged position. He pushed information theory, not the application of information theory. And so he had this great, there's a bunch of great lines about this from other people in the field. And they basically said they were like, Shannon didn't identify what the error correcting codes were going to be or what the algorithms were actually going to be that did this. All he did, his genius insight, was identifying that this was a statistical possibility, that it was actually possible for engineers to do this.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

35. “because they are predicting things that used to be done by us, even on image creation, right? On image creation, they can predict that this shade of blue should be Based on statistical probabilities. Probabilistically speaking, this shade of blue should be next to this shade of blue on a picture of planet Earth. That's a probabilistic determination. They're making a best guess at what something could be. But it turns out the machines are pretty good at guessing. They've gotten better.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

36. “But these things are intimately tied together because the whole thing, the whole sort of genius insight for Shannon was we actually don't need most of the English language. And that applies across different fields. We actually don't need this, this, this, and this in order to communicate a message. And it allows for more efficient communication. So it doesn't surprise me that the world of AI and LLMs has embraced this to make compression better. And I actually suspect that this is going to herald an era in which technologies like 6G are come to market much more quickly. Like I think there's going to be a bunch of changes because of this because honestly, it's part of the magic of playing around with some of these tools is seeing exactly how they know what to do”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

37. “And there was a time when we needed to convey every single bit. And now because of these predictive technologies, we can communicate with fewer bits. And by the way, we're at the 10,000 foot level on this. We're also at the limits of my understanding about how LLMs actually do this.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

38. “The basics of what you said are correct. Two things. One is these compression algorithms have gotten better over time. And it doesn't surprise me that large language models, which are studying language broadly defined, that they have found ways to compress files further and make the amount of data that is transmitted smaller. Because what they're probably doing is they probably have better ways of predicting what the next word or what the next bit or what the next thing will be. And when you can get that prediction improved, you don't need as much data transmitted. And so if you didn't need a specific word or bit and you cut it out, you've taken, let's say, 10 units of data and you've made it into eight units of data.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

39. “Yeah, and people are going to have to forgive me because it's been a number of years since I wrote this work and since the book's been out. And I think we have to be careful because data and entropy are really specific terms within this field. So I want to be careful about not sort of going over my skis or getting things wrong.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

40. “Year legacy of something that started with Claude Shannon, which is how do I get a text on my phone to be transmitted from planet Earth to a satellite and over to Preston without losing any of the meaning? And that is the theoretical work that Claude Shannon did.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

41. “That is actually from an information theory perspective. That is more information, meaning this new word is surprising and ergo is new information because we have to do more to convey it. We have to do more to communicate it. It's a little abstract, but I think people can get the idea that the basic idea is there's a ton of redundancy within the English language. And you can eliminate whole words and whole vowels and people can still understand something. And so you can use that redundancy to your advantage because what you can do, and this is where Shannon applies the information theory, you can encode messages with certain kinds of redundancy so that my message to you is conveyed not perfectly, but almost perfectly. And by the way, I don't know if people have ever watched the show Silicon Valley, but when they're talking about lossless compression algorithms, the lossless compression algorithm idea traces directly back to this paper. And so for people who are working in modern communications, you stand in a 70.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

42. “Yeah. So, what he's saying is that all information can basically be abstracted into a mathematical formula, that there's no difference between me saying the fox jumps over the dog mathematically. There's no difference between me saying the fox jumps over the dog and the dog jumps over the fox. They're actually identical. So meaning doesn't matter. The actual bits mathematically, they're equivalent. He also points out that because the English language has so much redundancy in it, it's surprise that is the thing you're trying to control for. Like if you have a word that's unexpected after another word, that's new, quote unquote, new information. So good example of this. The letter U always follows the letter Q. So if you see a letter Q, I think it's like a 99.999999% chance that the next letter after Q is you. But if all of a sudden, let's say next year, Miriam Webster, there's some word that goes viral and it's Q and it doesn't have a U after it.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

43. “Out in a little drawing, and if any of your listeners are curious, they can just Google information theory and they'll see a little drawing. And he basically outlines that there is a sender, there's noise, there's a channel, and there's a recipient. And he shows you different ways to reduce noise in that channel. And so that's what we're trying. That's the basic guts of information theory is how do we communicate. Now, a number of surprising things emerge from this. One of the surprising things that emerges, Shannon posits that the actual value of information is not the meaning, it's the surprise. So that's a really interesting thing. What do you mean by that?”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

44. “And he'll show how even if you eliminated most of the vowels or other letters within that sentence, someone reading it could actually still make sense of it. And that compression is one of the ways that you and I can have this exchange without it taking seven years. And the reason I'm doing that is because I want to connect the theory to the technology, which is this is effectively the theoretical plumbing for all of modern life. It affects everything from the phone calls we make to this exchange to a video or a song if you're using Spotify, you're in some meaningful way using Claude Shannon's information theory. The idea is how do you most efficiently and in the most compressed format possible communicate information with high fidelity? And what Shannon outlines is he outlines a theoretical limit for some of these. So it's called the Shannon limit. And since he laid down that limit, engineers have basically been trying to get us technologies that communicate up to that limit. And that's why it was such an earthquake. Someone finally figured out that there was a model for this. And he lays it.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

45. “That emerges in the paper. There are a bunch of different ways to convey a piece of information without losing meaning. You can shout louder. You can repeat yourself. You can do many things to communicate information better. But some of those ways of communicating information better will just add to the amount of time or the amount of bits that you need to communicate information. So the real trick in information theory is how do I tell Preston a sentence in this video and how is it communicated as efficiently as possible? There's a number of components to this, but one of the things that Shannon explains to people in variation theory is there's a lot of redundancy in the English language, right? So he'll take a sentence like the fox jumped over the something.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

46. “This is one of those where I'm going to do my best. And I translated it and spent a lot of time thinking about that chapter of the book when we were writing it. If you feel like my explanation is inadequate, find me on Twitter, shoot me DM or shoot me an email, and I'll just send you that chapter of the book because it'll do a much better job than I can in 30 seconds of explaining it. But here's the visual that I would use that helps best explain what information theory actually does and why it matters. Right now, you and I are able to communicate over Zoom or I can like pick up my phone and download a YouTube video, right? And watch a YouTube video. And the reason that that process can happen is because of the transmission of information, but it's compressed information. And what information theory does is it provides a model for the compression and transmission of information without loss of meaning. So I'll give you a kind of example.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

47. “Information being coded into our genes. Shannon actually did work on genetics before he published the information theory paper. And so there are some useful applications. I think what was happening is that people were not treating the work with sufficient seriousness in 1949 and 1950 and 1951. And a lot of those people were within Shannon's field. And that's why he had to kind of throw the brushback pitch of that bandwagon paper.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

48. “And one of the reasons is because those systems do is reduce noise in financial transactions, right? They reduce latency. And well, noise is not latency, but they reduce latency. And for a separate project, when I was interviewing Elon for the founders book, which is about PayPal, and we were talking about the role of money, like what is money? As this view, he said money is just an information system. It's a way for us to avoid having to barter. That's almost an exact quote from something he said to me. It's like a way that we don't have to barter. It's an information system. And he says, because it's an information system, you could see how a universe of digital currencies is actually a much more efficient information system by Shannon terms than something like handing you pressing a dollar, right? Because there's a lot of friction involved there. And so there are some interesting applications of information theory around the world and useful ones in the case of Sinclair.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

49. “We traded some really nice notes. Now, what's interesting is he is carefully and rigorously applying the principles that are in information theory. Talk to discuss his field. And so there is this, like I think what Shannon was saying was like let's electrical engineers try to puff this up too much in our domain, but the metaphor or some of the principles that are in information theory apply very well in other domains. I'll give you another example that's probably more relevant to your audience and actually have some crossover with some of the other work I've done. One of the things that people have mentioned is that there's a fair amount of overlap in the world of Bitcoin and cryptocurrency and the work of information theory.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT

50. “By the way, he takes it very seriously, Dr. Saglen, we've actually corresponded about Shannon. He's a huge screen fan. And I wrote him when his book came out because he had mentioned Shannon a couple of weeks. I said, hey, and he had already read a mind of play.”

    2024-06-26 · We Study Billionaires · BTC188: Claude Shannon and Information Theory with Jimmy Soni (Bitcoin Podcast) · IDENTIFIED FROM THE TRANSCRIPT