YouSaid · the spoken record

Joseph Carlsmith

lines on the record

86

first

2022-08-03

most recent

2022-08-03

sittings or episodes

1

sources

podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

1. “Yep, so my blog is handsandies.com, and my Twitter handle is JK Carl Smith. Those are good places to reach me. And then my personal website is josepharlsmith.com.”

   2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

2. “Yeah, thanks for having me. Oh, I'll also say, you know, everything I've said here is just purely my personal opinion. I'm not speaking for my employer, not speaking for anyone else, just myself. So just keeping that in mind.”

   2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

3. “Convey is the ideas that matter most to me or that have had close to the biggest impact in my own life. Other books, I love the play Angels in America. I think it's just epic and amazing. And that's not quite a book, but you can read it. I actually recommend watching the HBO mini series. But that's something I recommend. And then I don't know, last year I read this book, Housekeeping by Marilyn Robinson. And it had this sort of numinous quality that I think a lot of her writing does. And so I really like that and recommend it to people. That's also a piece of fiction. If you're looking for philosophy, I don't know. A lot of my work is in dialogue with Nick Fostrom and his overall kind of corpus. And I think that's. Really, really valuable to engage with.”

   2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

4. “Probably my primary recommendation that this is somewhat self serving because I helped with this project is to book The Precipice by Toby Ord, may be familiar to many of your listeners, but I think it's a book that really.”

   2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

5. “Yeah, and I think actually the blog, I started the blog partly as an exercise in just getting stuff out there. I think I had had the idea that I would one day write up a bunch of stuff that I've been thinking about. But it was somehow, and I would write it up in this grand, you know, I would finally write it up and it'd be this beautiful thing. And I would, you know, take all this time. And then I had ended up for various reasons feeling like I was approaching some aspects of my life with too much perfectionism or too much and I needed to just like get stuff out there faster. And so the blog was an exercise. And I think has paid off in ways and that I don't know, I don't think I would have done it otherwise.”

   2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

6. “On my kind of overall output. And so I do think there's a balance there. And to the extent you're one of these people who you can just stream of consciousness and that's like close to what you would get out of editing, which I'm not sure I am, you know, all the better. It's sort of like you're lucky.”

   2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

7. “Yeah, I wonder. I mean, my feeling is that my writing is quite a bit better if I have a chance to edit it. And it's just a time thing. But I do think people vary quite a bit. And it's interesting. I don't know if I was recently reading this book, George Saunders, who I think a writer I really admire has this book about fiction writing called Swim in the Pond in the Rain. And the vibe, he tries to convey. And I think this is relatively common amongst writer types, is like this obsessive focus on even at a sentence-by-sentence level, really thinking about where is the reader's mind right now? How are they engaging? Are they interested? Are they surprised? Am I losing them? And, you know, his writing is really, really engaging in ways that it's like not even obvious. You just sort of start reading along and you're like, oh, wow, I'm really into this. But it's also quite a daunting picture of the level of attentiveness required. And it's like, wow, if I'm going to write everything like that, it's like, that's going to cut down a lot.”

   2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

8. “Think it's something like I would have written you a long letter, but I would have, I didn't have time to write you a short letter, so I wrote your long letter or something like that. Yeah, exactly. I have a friend who says, like, the actual thing it should be, I didn't have time to write you a short letter, so I wrote you a bad letter. And, you know, I'm like, I hope it's not that bad, but I do, you know, I do think if I had more time for these posts, I would try to kind of cut them down. And so that's one time saving for better or worse.”

   2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

9. “Should say a few of my recent, my most recent posts, which were especially long. I had taken some time off from work and I was working on those partly in an academic context. But the first year and a half or so of the blog was just on the side and I've gone back to having it be on the side now. I think one thing that helps is my blog posts are too long. And so there's, you know, I have dreams of writing these, you know, taking my long blog posts and then really crunching them down and making it into this like pithy, elegant statement that's really concise and condensed. But that would be more, so one way I sort of increase my output is by not doing that editing. And I feel bad about that. But that's one thing at least.”

   2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

10. “Whatever, and it says you're wrong immediately. That's not how it's going to be. And so you don't have the ability to really hone in on concrete details that are actually true. And so in some sense, you need to, there's this back and forth where you need to sort of imagine a concrete thing and then be like, okay, that's wrong, but there will then take the flavor of concreteness that you”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

11. “Yeah, I mean, I think so, I think that's a great example of, or specifically this sort of image of the difference between relating to history as this sort of how is the border changing versus the concreteness of these people. And, you know, often I think engaging with history is horrifying in this respect, is when you really bring to mind the lived reality of all these events, it's a really different experience. And I think to some extent, one of the reasons that concreteness might be often lacking from futurism is that you can't any attempt to specify the thing will be wrong. So, you know, we can, you can, you might be right about some abstract thing. Like you might be like, oh, we will, you know, we will have the ability to manipulate matter at like blah, you know, blah level scale. But if you try to dig in and then you're like, and here's what it's like to wake up in the future, you know, and then, you know, you're eating the.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

12. “Zone is inflected with kind of social dynamics, or it's you know, it's kind of like an intellectual game or you're enjoying it for its own sake, or it's like a sort of there's sort of status dimensions and the way people talk and other things that I think start to move our discourse in directions that aren't about like ah, we're talking about the real world right now. Let's actually get it right. And I think that happens with futurism and maybe more so because it can feel like I think some people there's sort of topics that they treat as like, ah, that's a real serious topic that's about real stuff. And then there are other topics where it's like, this is the chance to kind of make stuff up. And my experience is sometimes people relate to futurism that way. There are other topics where people move into a zone of like, one can just say stuff here and there are kind of no constraints. And I think that's actually wrong. And with futurism, I think there are important constraints and important things we can say. But I think that vibe can seep in nonetheless.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

13. “I think it comes partly because imagination is just quite a limited tool. And it's just easy. You know, when you're talking about the whole world, like the future is a big thing to try to model with this tiny mind. And so of necessity, you need to use these extremely lossy abstractions. And so it puts you in a mode of having these like really sketchy and gappy maps that you're trying to manipulate. I think that's one dimension. And then I think there's also a way in which this isn't all that unique to futurism insofar as just in general, I think it's hard sometimes to keep our intellectual engagement kind of rooted and grounded in the kind of real world. And I think it's just easy to kind of move into a zone and especially if that's.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

14. “But, you know, my own take is kind of what happens here we hit this infinity, you hit infinite universes fairly fast, and then they kind of break your anthropics in analogous ways to how they break your ethics. And that's kind of where I'm currently at. And I'm hoping to understand better how to do anthropics with infinities and some of my work on the universal distribution, which is a sort of, I have a couple blog posts on that, was attempting to go a little bit in that direction, though it has its own giant problems.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

15. “I don't know, maybe we're in a simulation and people are obsessed with simulating us or something like that. And then I think this is actually more important and worrying is I think the way I see this dialectic is first SIA, I mean, so a big problem with SIA is it immediately becomes certain naively that you live in an infinite universe or a universe with an infinite number of people. And then it breaks and it doesn't know how to compare kind of infinite universes. Now, to be fair, SSA also isn't great at comparing infinite universes. And they both have some, you can do things that are actually quite analogous to things you can try to do in infinite ethics where you have like expanding spheres of spacetime and you count, you know, you have some fraction or some density of people in those spheres. And there's this general problem in cosmology of like trying to understand what it means to have like a fraction or a density of different types of observers.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

16. “My main claim is that SIA is better than SSA. And I think it's just a horrible situation with anthropics. And I think overall, SIA is an update towards bigger, more populated universes. I think the most salient populated universes don't involve hidden people on other planets, but they're probably.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

17. “With the power of SSA have become extremely confident about how it's going to land. And so that's another argument. There's a number of other, I think, really, really bad problems for SSA.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

18. “And you get so that's the sort of exotic example you get similar analogs even in the Gods Coin Toss case where like naively it doesn't actually matter whether God has tossed the coin yet right so suppose yeah so like let's say let's say you wake up and learn that you're in room one right but God hasn't tossed the coin it's like he created room one first before he tossed and then he's gonna toss and that's gonna determine whether or not he creates all the rooms in the future If you on SSA, once you wake up and learn that you're in room one, you think it's incredibly unlikely that there's going to be these future people. So now you say before it's a fair coin, God's going to toss it in front of you, you're still going to say, I'm sorry, God, it's a one in a million chance that this coin lands tails and or sorry, one in a million, something like a very small number. I forget exactly. And that's very weird. That's a fair coin. It hasn't been tossed.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

19. “So you make the following resolution. You say if this boulder does not jump out of the way of this puppy, like the boulder leaps in some very weird, very unlikely way, then I will press this button and I will create zillions and zillions of people, all of whom are in a different epistemic state than me, but let's assume they were in my reference class. SSA thinks that it's sufficiently unlikely that you would be in a world with zillions of those people, but you at the very beginning with different colored clothes because that was a tiny fraction of the reference class if those people get created. That SSA thinks it's actually more likely once you've made that commitment that the boulder will jump out of the way. And that looks weird, right? It just seems like that's not going to work. You can't just make that commitment and then expect the boulder to jump.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

20. “Yeah, so an easy problem, or sort of one of the most dramatic problems, is that SSA predicts that it's possible to have a kind of telekinetic influence on the world. So imagine that there's a puppy. You're, you wake up and you're in an empty universe, except for this puppy and you and this boulder that's rolling towards the puppy, right? And the boulder is inexorably going to kill the puppy. It's very large boulder. It's basically guaranteed that the puppy is dead meat. But you have the power to make binding pre-commitments that you will in fact execute. And you have also to your right a button that would allow you to create tons of people like zillions and zillions and zillions of people, all of whom are wearing different clothes from you. So they would be in a different epistemic state than you if you created them. Now, SSA.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

21. “To come about more often if there are more arrangements of that type rather than sort of assuming that no matter what existence will include you.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

22. “So I think the SIA response to that would be, or at least I think the way of making it intuitive would be to say that you didn't know that you were going to wake up, right? So in the if we go back to that Jess So story where God is grabbing you out of the heaven, you know, it's not at all, it's actually incredibly unlikely that he grabs you. There are so many, so many people. I mean, there's a different thing where SIA is in general very surprised to exist. And in fact, that's the So you could make the same arguments like SIA says you shouldn't exist. Isn't that weird that you exist? And I actually think that's a good argument. But once you're in that headspace, then I think the way to think about it is that it's not a guarantee that you God is not dead set uncreating you. You are a particular contingent arrangement of the world and so that you should expect that arrangement to.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

23. “I think there's an intuition there, which is sort of like is SIA making a bad prediction? So you could kind of update against SIA because SIA would have predicted that you're in the future. I think there's something there. And I think there's a few other analogs. Like, for example, I think SIA naively predicts that you should find yourself in a situation where there are just tons of people that a situation obsessed with creating people with your evidence. And this is one of the problems with SIA. So you should expect to find, you know, in every nook and cranny, a simulation of you as soon as you like, you know, you open the door, it's actually this giant bank of simulations of you in your previous epistemic state. And then you don't see that. You might be like, well, I should update against the anthropic theory that predicted that I would see that. And I think there are arguments in that vein.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

24. “The sense in which SIA leaves you back at normality with the Doomsday argument is once you update on being in the 21st century, which admittedly should be surprising. Like if you didn't know what you were in the 21st century and then you learned that you were, you should be like, wow, that's really unexpected. And fair. So, and that's true. But I think once you do that, you're back at, you know, whatever your prior was about extinction.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

25. “Yes, so in a specific sense. So it's true that on SIA say that That we don't know what room you're in first, right? So you wake up in the white room and you're wondering, am I in room one or am I in rooms two through a million, right? And on SIA, what you did first, so you woke up and you don't know what room you're in, but there's a lot more people in the world with lots of rooms. And so you become very, very confident that you're in that world, right? So you're very, very confident on tails. And then you're right that conditional on tails, you think it's much more like you sort of split your credence evenly between all these rooms. So you are very confident that you're in one of the sort of two through a million rooms and not room one. But that's before you've seen your room number. Once you see your room number, it's true that you should be quite surprised about your room number.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

26. “And then he, and say there's like a big line of souls in heaven. And he goes and grabs them kind of randomly out of heaven and puts them into the world, right? And so in that case, if there are more people in the world, then you've got more shots and you're one of these souls. You're sort of sitting in heaven hoping to get created. On SIA, God has more chances to grab you out of heaven and put you into the world if there are more people like you in that world. And so you should expect to be in a world where there was sort of, there are more such people. And that's kind of SIA's vibe.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

27. “Yeah, so I think a way to think about SIAs. Kind of story. So I gave this story about SSA, which is it's sort of like this it's like first God creates a world. This is SSA. First He creates a world and then he takes and he's dead set on putting you into this world. So he's got your soul, right? And he really wants and your soul is going in there no matter what, right? But the way he's going to insert your soul into the world is by throwing you randomly into some set of people, the reference class. And so if you wake, so you should expect to end up in the world where the kind of person you end up as sort of a more likely result of that throwing process is a sort of larger fraction of the total people you could have been. What SSA or what SIA thinks is different, the way the story that I'll use for SIA though, it doesn't assist the only gloss is God decides he's going to create a world.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

28. “First creates world, and then he inserts you randomly into like some group. It's much more likely that you would find yourself in the 21st century if humanity goes extinct soon than if there are tons of people in the future. If God randomly inserted you into these tons of people in the future, then it's like really, it's a tiny fraction of them are in the 21st century. So SSA in other contexts actually has these important implications, namely that in this case you update very, very hard towards the future being short. And that matters a lot for long-termism because long-termism is all about the future being big in expectation.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

29. “In this case, in both cases, everyone has your evidence. And so the fraction is the same. And so you stick with the one half prior. But that's not true. So SSA in other contexts, not everyone has their evidence. And so it updates towards worlds where it's a larger fraction. So famously, SSA leads to what's known as the doomsday argument where you imagine that there are two possibilities either humanity will go extinct very soon or we won't go extinct very soon and there will be tons of people in the future. And in the former case, and then you imagine everyone is sort of ranked in terms of when they're born. In the former case, people born at roughly this time are a much larger percentage of all the people who ever lived. And so if you imagine, you know, God.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

30. “Your probability that the coin came up heads should be approximately one in a million. And that's because SIA thinks it's more likely that you exist in worlds where there are more people in your epistemic situation or more people who have your evidence, which in this case is just waking up in this white room. And that's, and so that can be a weird conclusion and go to weird places. But I think it's a better conclusion than the alternative. SSA, which is the main alternative I consider in that post, which is that the self-sampling assumption says that you should, you think it more likely that you exist in worlds where people with your evidence are a larger fraction of something called your reference class, where it's quite opaque what a reference class is supposed to be, but broadly speaking, a reference class is the sort of set of people you could have been, or that's kind of how it functions in SSA's discourse.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

31. “Sure. So I think one way to start to get into this debate is by thinking about the following case. So you wake up in a white room and there's a message written on the wall and let's say you're going to believe this message. And the message says, I God, it's from God. I God created, I flipped a coin. And if it was heads, I created one person in a white room. And if it was tails, I created a million people, all in white rooms. And now you are asked to assign probabilities to the coin having come up heads versus tails. And so one approach to this question, which is the approach I favor, or at least think is better than the other, is the self-indication assumption. These names are terrible, but so it goes. So SIA says that”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

32. “I guess I'd want to really hear the empirical case. I think it's true, you know, there are a lot of insects. But I think it's easy. I think if you want to say, taking seriously sort of the idea that there's some reason to not squash a bug if you see it leads immediately to kind of jane-like behavior absent long-termism or something like that. I really feel like I want to hear the empirical case about exactly what impact you're having and how, and I'm not at all persuaded that that's the practical upshot. And if it is, if that's a really strong case, then I think that's an interesting kind of implication of this view and worth concern. But I wouldn't jump, it feels to me like it's easy to jump to that almost out of a desire to.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

33. “Is recognizing that this is a real trade off. There's uncertainty here and taking responsibility for how you're responding to that.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

34. “Kind of schizophrenic about what cases we view as sort of morally relevant and which not. We have pet treatment laws and then we have factory farms and stuff like that. I don't see it as a radical position that ants matter somewhat. I think there's a further question of what your overall practical response should to that should be. And I think the main, and I do think the kind of costs, as in a lot of ethical life, there are trade-offs and you have to make a call about what sort of constraints you're going to put on yourself at the cost of other goals. And in the case of insects, it's not my current moral focus and I don't pay a lot of costs to kind of lower my impact on animals. And I don't sweep the sidewalk or anything, or sorry, on ants in particular. And so I think, and I think that's, you know, that's my best guess response. And that has to do with other ethical priorities in my life. But I think there's a middle ground between I shall ignore this completely and I shall be a Jain.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

35. “I have significant uncertainty about exactly, and I think this is the appropriate position about exactly how much kind of consciousness or suffering or other forms of moral other ways, other kind of properties that we associate with moral patienthood, how much those apply to different types of insects. I think it's a strange view to be extremely confident that what happens with insects is totally morally neutral. And I think it actually doesn't fit with our common sense. So let's say you see if you see a child like frying ants with a magnifying glass, I think there is some, you could say, ah, well, that just indicates that they're going to be cruel to other things that matter, but I don't think so. I think, you know, and you see the ants like, you know, and they're twitching around. And so I think we aren't, you know, as in many cases with animal ethics, I think we're a bit.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

36. “Know if this doesn't seem right to you or if it seems too crazy, why and really kind of processing, you know, it's a reason to pay a lot of attention. But if you've paid a lot of attention at the end of the day, you're like, well, I guess at an abstract level, that sort of makes sense, but it just doesn't feel to me like the real world. It just doesn't feel to me like wisdom or like a healthy way of living or whatever, then I'm like, well, maybe you shouldn't do it, right? I mean, and I think some people will do that wrong and they will end up bouncing off of ideas that are in fact good. you know, I think overall these are sort of sufficiently intense and difficult issues that kind of being actually persuaded and not just sort of chopping off the rest of your epistemology for the sake of some like version of the abstraction is it seems to me important and it's and it's a sort of a healthier way to relate.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

37. “So, I don't think there's a principled reason. And I should say, I don't think of my attitude towards infinite ethics as solely, oh, this has gotten too far down the crazy path. I'm out. This thing about the wisdom in the future is pretty important to me as a mode of orientation. A first pass cut that I use is when do you feel like it's real? If you feel like a thing is real as opposed to a kind of abstract fun argument, then that's important or that's a real signal. And I think so, and I generally encourage people if the sort of mode that I don't know, I'm drawn to is something like if there's an idea that seems compelling intellectually, that's a reason to investigate it a lot and think about it and really grapple with.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

38. “That it's focused less on, and the main event will be what happens to future people. And it's more about getting to the point where we are wise enough to understand and reorient in a better way.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

39. “The way I resolve it is, I think for many of these issues, the right path forward, or at least a path that looks pretty good is to survive long enough for our civilization to become much wiser. And then to use that position of wisdom and empowerment to act better with respect to these issues. And so, and that's what I say in the end of the Infinite Ethics Post is that, you know, I think future civilization, if all goes well, will be much better equipped to deal with this. And we are at square one in kind of really understanding how these issues play out and how to respond. And so I think both at an empirical level and at a kind of philosophical level. And so it looks convergently pretty good to me to survive, become wiser, keep your options open, and then act from there. And that looks, that ends up pretty similar to a lot of long-termism and existential risk. It's just.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

40. “Yeah, so I think there's a real tension there in that I think many, you know, how seriously should we take these ideas, at what point should you be making what sorts of changes for your life on the basis of different things that you're thinking and believing? You know, it's a real art, right? And I think some people go, you know, they grab the first idea they see and they start doing crazy stuff and in an unwise way. And some people are too kind of sluggish and they're not willing to take ideas seriously and not willing to reorient their life on the basis of changes in what seems true. But I think, you know, nevertheless, I think especially things that involve like, ah, turns out it's fine to, you know, do terrible things or there's no reason to eat your lunch or whatever, like things that, you know, sort of really, really holistically breaking of your ethics views, I think one should tread very cautiously with. So that's one aspect at a philosophical level.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

41. “Yeah, so I think there's two dimensions there. One is that I think it's good practice to not totally upend your life and do and if you encounter some destabilizing philosophical idea, especially one that's sort of difficult and you don't totally have a grip on it to come back.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

42. “So, I mean, there's a general, I think there's a couple dimensions here. So, one is. Trying to understand actually what sort of difference does it make if you're in this sort of infinite situation and you're thinking about a causal influence, what even did you change at a sort of empirical level before you talk about how to value that? And I think that's a pretty gnarly question. Even if we settled that question though in terms of like the empirical a causal impact, there is a further question of how do you rank that or how do you deal with the sort of the normative dimension here. And there, you know, so that's the sort of ethical question and there things get really gnarly very fast. And in fact, there are kind of impossibility results that show that even very basic constraints that you really would have thought that we could get at the same time in our ethical theories, you can't get them at the same time when you come when it comes to infinite universities.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

43. “Though that's not a mainstream view. So it's fairly, I think it comes in fairly directly and substantively if you have that combination of views. But then I also think it comes in. I think everyone should have non-zero credence in all sorts of different infinity involving hypotheses. And so infinite ethics gets a grip regardless.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

44. “So I haven't dug into the cosmology a lot, but my understanding is that it's at the very least a very live hypothesis that the universe is infinite in the sense that there are, you know, it's a sort of infinite in extent and there are suitably far away, there are copies of us having just this conversation and then even further away there are copies of us having this conversation but wearing raccoons for hats and and all the rest which is itself something to wonder about and sit with. But I, you know, my understanding is this is this is just a live hypothesis and more broadly kind of infinity is infinite universes are just sort of a part of mainstream cosmology at this point. And so yeah, I think I don't think it's just a thought experiment. I think infinite universes are live and then I think these sort of non-causal decision theories are actually my sort of best guess decision theories.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

45. “It's a little complicated because he's too far away, but just in general, like if I raise my hand or if I want to write stuff on my whiteboard, or if I'm going to, you know, there's, let's say I have to.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

46. “Sure. So a common decision theory, though it's not, I think the mainstream decision theory is a contender in the literature is evidential decision theory where you should act such that you would be, roughly speaking, happiest to learn that you had acted that way for that reason. And so the reason this allows you kind of a causal influence, so a way of thinking about it is suppose that you are a deterministic simulation and there's a copy of you being run sort of too far away for you to ever causally interact with it, right? But you know that it's a sort of deterministic copy. And so it'll do exactly what you do, absence some sort of computer malfunction. And now you're deciding whether to give, you know, you have two options. You can send a million dollars to that.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

47. “But actually, if you have even a tiny pretence that you can do an infinite thing, you know, either that dominates or it breaks. And then if you have tiny credences on doing different types of infinite things and you need to compare them, you need to know how to do it. And so I just think this is actually, you know, it's actually a part of our epistemology now, though it's, I think we often don't treat it that way because we're often not doing EV reasoning or really thinking about that. These are questions that just apply to us.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

48. “Of ways that I think it's just we should have non zero credence that we can have infinite influence in our actions now. So our kind of causal influence, the limitations there could be wrong. It may be that there are ways in the future we'll be able to do infinite things. And then I also think somewhat more exotically that there's sort of ways of having a causal influence on an infinite universe, even if you are limited in your causal influence. And that comes from some additional work I've done on decision theory. And so if you try to incorporate that, if you're a sort of expected value reasoner, it just very quickly starts to dominate or at least break your expected value calculation. So, you know, you mentioned long-termism earlier and a natural reason, a natural argument for getting interest in long-termism is, oh, in the future, there could be all these people, their lives are incredibly important. So if you do the EV calculation, sort of your effect on them is what dominates.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

49. “Often in ethics, we expect our ethical principles to extend to kind of ranking scenarios or sort of acting in hypothetical scenarios or overall kind of all possible situations rather than just our actual situation. I think infinities come in there. But then I think maybe more importantly, I think it's an issue with practical relevance and a way to see that is that I think we should have non-zero credence that we live in an infinite world. And it's a very live physical hypothesis that the universe is infinite, even if I think the mainstream view is that our causal influence on that universe is finite in virtue of things like entropy and light speed and stuff like that. But the universe itself may well be infinite in possibly infinite in a number of different ways that sort of Max Tegmark has some work on all the different kind of like large ways the universe can be really very large.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source

50. “Sure. So infinite ethics is ethics that tries to grapple with how we should act with respect to kind of infinite worlds and how should we rank them? How should they enter into our expected utility calculations or our attitudes towards risk? And I think this is important for both kind of theoretical and practical reasons. So I think at a theoretical level, when you try to do this with a lot of common ethical theories and constraints and principles, they just break on infinite worlds. And I think that's an important clue as to their viability because I think infinite worlds are at the very least possible, even if our world is finite and even if our causal influence is finite or our influence overall is finite, it's possible to have infinite worlds and we have opinions about them, you know, like an infinite heaven is better than an infinite hell and so I think.”

    2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source