YouSaid · the spoken record

Joe Carlsmith

lines on the record: 176
first: 2024-08-22
most recent: 2024-08-22
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Feature of how we have, in fact, structured a lot of our political institutions and norms and stuff like that. So that's the thing I'm getting at quote.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Taking that seriously and thinking about what it is it to do that in a way that's like genuinely kind of legitimate and kind of a project that is sort of a kind of just incorporation of these beings into our civilization such that they can kind of all, or sorry, there's like the justice part and there's also the kind of, is it like kind of Compatible with people, you know, is it a good deal? Is it a good bargain for people? And I think this is often how, you know, to the extent we're kind of very concerned about AI's like kind of rebelling or something like that. It's like, well, there's like a lot of A thing you can do is make civilization better for some, right? So it's like, and I think that's an important.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“You might think because part of who we are is been made by a kind of nature's way. And so that is in us. Now, I don't think that's enough necessarily for us to beat the gray goo. We have some amount of power built into our values, but that doesn't mean it's kind of going to be such that it's kind of arbitrarily competitive. But I think it's still important to keep in mind that this is, and I think it's important to keep in mind in the context of integrating AIs into our society that I think we've been talking a lot about the ethics of this, but I think there's also there are like instrumental and kind of practical reasons to want to have forms of social harmony and cooperation with AIs with different values. I think we need to be.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Sometimes the way in the context of the series where I'm talking about deep atheism and our sort of relationship, the relationship between what we're pushing for and what nature is pushing for, what sort of pure power we'll push for. And it's easy to say, well, there's like paperclips, which is just one way, place you can steer and pleasure is like another place you can steer or something. And these are just sort of arbitrary directions. Whereas I think like some of our other values are much more structured around cooperation and things that also are kind of effective and functional and powerful. And so that's what I mean there is I think there's a way in which we're sort of nature is a little bit more on our side.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Kind of shaping what we now, at least in certain contexts, also treat as a kind of intrinsic or terminal value. So some of these values that have kind of instrumental functions in our society also get kind of reified. In our cognition, as kind of intrinsic values in themselves. And I think that's okay. I don't think that's a debunking. All your values are kind of like something that kind of stuck and got kind of treated as a terminally important. But I think that means that”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“And, you know, I talk for a while about the sort of ethical virtues of these norms, but it's pretty clear that also why do we have these norms? Well, one important feature of these norms is that they're kind of effective and powerful, like liberal societies are secure boundaries, save resources wasted on conflict, right? And liberal societies are often more like they're better to live in. They're better to immigrate to. They're more productive, like all sorts of things. Nice people, they're better to interact with, they're better to trade with, all sorts of things, right? And I think it's pretty clear if you look at both like why at a political level do we have like various political institutions? And if you look kind of more deeply into our evolutionary past and like how our moral cognition is structured, it seems like pretty clear that various kind of forms of cooperation and game theoretic dynamics and other things went into”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“So, the context on that post is I'm talking about this hazy cluster, which I call in the essay niceness slash liberalism slash boundaries, which is this sort of like somewhat more minimal set of cooperative norms involved in respecting the boundaries of others and kind of cooperation and peace amongst differences and tolerance and stuff like that, as opposed to your favored structure of matter, which is sort of sometimes the paradigm of values that people use in the context of AI risk.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“You have to find some way of differentiating between the sort of development processes that preserve what you care about and the development processes that don't. And that in itself is this like fraught. Question, which itself requires taking some stand on what you care about and what sorts of metaprocesses you endorse and all sorts of things. But you definitely shouldn't just be like, it is not a sufficient criteria that the thing at the end thinks it got it right. Right. Because that's compatible with having gone wildly off the rails.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, you definitely don't want to be like, you know, if you transform me into a paper clipper gradually, right? Then I will eventually be like, and then I saw the light, you know, I saw the true paperclips. Right. But that's part of what's complicated about this thing about reflection.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“It's a good question. I mean, I think. It's like some guess about if there's no part of me that recognizes it recognizes it as good. I think Good, according to me, in some sense. So, yeah, I mean, it's a question of what it takes for it to be the case that a part of you recognizes it is good. But I think if there's really none of that, then I'm not sure. It's a reflection of my values at all. There's a sort of taut”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm inclined to think that Utopia, however weird, would also be, in a certain sense, recognizable, that if we really understood and experienced it, we would see in it the same thing that made us sit bolt upright long ago when we first touched love, joy, beauty. We would feel in front of the bonfire the heat of the ember from which it was lit. There would be, I think, a kind of remembering.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“There's a little bit of a loopiness if you're. I think there are probably like fixed points in that where you could be like, yep, I'm gonna do that. And then you're right. But I think it's. At least have a question of are we, you know, when people imagine the kind of completion of knowledge, you know, exactly how well does that work? I'm not sure.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“At the least, I think you don't get full knowledge in some sense because there's always like, what are you going to do? Like, there's some way in which you're part of the system. So it's not clear that the knowledge itself is part of the system and sort of like, I don't know, like if you imagine you're like, ah, you try to have full knowledge of what the future of the universe will be like. Well, I don't know. I'm not totally sure that's true.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, but this isn't coming from my dad. This is like there's a blog post, I think Sean Carroll or something. And he's like, we really understand a lot of the physics that governs the everyday world. Like a lot of it, we're really good at it. And I'm like, I think I'm generally pretty impressed by physics as a discipline. I think that could well be right. And so on the other hand, like, Really, you know, had a few centuries. So, anyway, but I think that's an interesting. And it leads to a different, I think it does. There's something the endless frontier. There is a draw to that from an aesthetic perspective of the idea of continuing to discover stuff”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“And then there's a different picture, which is much more this like ongoing mystery, oh man, there's going to be more and more, maybe expect more radical revisions to our worldview. And I think it's an interesting, yeah, I think I'm kind of drawn to both. Like physics, we're pretty good at physics, right? Or like a lot of our physics is quite good at predicting a bunch of stuff and, or at least that's my impression. This is reading some physicists. So who knows?”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I sometimes think about this sort of two categories of views about this, like there are people who think, yeah, like the knowledge, like we've almost, we're almost there and then we've like. Basically, got the picture, right? And where the picture is sort of like, yeah, the knowledge is all just totally sitting there. And it's like you just have to get to like remote. There's like this kind of, just you have to be like scientifically mature at all. That's right. And then it's just going to all fall together, right? And then everything past that is going to be like this super expensive, like not super important thing.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Discovery of tech, then you might also expect more ongoing change and upheaval and churn insofar as technology is one thing that really drives kind of change in civilization. So that could be another, you know, people sometimes talk about like lock-in and there's like, ah, sort of, they envision this kind of point at which civilization is kind of like settled into some structure or equilibrium or something. And maybe you get less of that. I think that's maybe more about the pace rather than contingency or caps, but that's That's another factor. So, yeah, I mean, I think it is an interesting, I don't know if it changes the picture fundamentally of Earth civilization. We still have to make trade-offs about how much to invest in research versus acting on our existing knowledge. But I think it has some significance.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“On any picture, necessarily, that you've got everything. And on any picture, in some sense, you could end up with this case where you cap out, like there's some collider that you can't build or whatever. Something is too expensive or whatever. And kind of everyone caps out there. So I guess one way to put it is like, so there's a question of like, do you cap? And then there's a question of like how contingent is the place you go. If there's contingent, I mean, one thing, one prediction that makes is you'll see more diversity across our universe or something. If there are aliens, they might have like. Quite different tech. And so maybe if people meet, you don't expect them to be like, oh, you got your thing. I have our version. And so you're just like, whoa, like that thing, wow. So that's like one thing. If you expect more ongoing”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“You know, I sort of suspect that was always true. And, like, I remember talking to someone, I think I was like, we should, at least in the future, we should really get all the knowledge. And he's like, what do you want to like? You don't want to know the output of every Turing machine? Or in some senses, it's a question of, what actually would it be to have a completed knowledge? I think that's a rich question in its own right. And I think it's not necessarily that we should imagine, even in this sort of”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm not sure if that's true. I think if that's true, what kind of difference would it make? One difference is that. Well, so here's a question at some sense you have to at a more ongoing way make trade-offs between investing and further knowledge and further exploration versus exploiting, as you say, sort of acting on your existing knowledge because you can't get to a point where you're like, and we're done. Now, as I think about it, I mean, I think that's...”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, I think there's a few different aspects. My memory of this conversation, I don't claim to really understand Michael's picture here, but I think my memory was it sort of like, sure you get the fundamental laws. I think my impression was that he expects sort of physics, the kind of physics to get solved or something, maybe modulo, like the expensiveness of certain experiments or something. But the difficulty is even granted that you have the kind of basic laws down, that still actually doesn't let you predict where at the macro scale various useful technologies will be located. There's just still this big search problem. And so my memory, though, I'll let him speak for himself on what his take is here. But my memory was it was sort of like, sure, you get the fundamental stuff, but that doesn't mean you get the same tech.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Conscious entity necessarily, but it's also not like totally unaware or something. And like, so there's all this, like the consciousness discourse is rife with these funny cases where it's sort of like, oh, like those criteria imply that this totally weird entity would be conscious or something like that, like especially if you're interested in some notion of agency or preferences. Like a lot of things can be agents, corporations, all sorts of things. Corporations conscious. And it's like, oh man. But I actually think it's one place it could go in theory is in some sense you start to view the world as like animated by moral significance in kind of richer and subtler structures than we're used to you know and so like plants or um you know like weird optimization processes are kind of like outflows of like complex i don't know like who knows exactly what what you end up seeing as infused with the sort of thing that you ultimately care about um but i think it's it is possible that that doesn't map that that that like includes”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“In some sense, if you're like, well, where do things go? I'm like, I should be clear, I have a bunch of credence that in the end we end up carrying a bunch about consciousness just directly. And so if we don't, like, yeah, I mean, where will ethics go? Where will like a completed philosophy of mind go? Very hard to say. I mean, I can imagine something that's more. Like, I think maybe a thing that I think a move that people might make if you get a little bit less interested in the notion of consciousness is some sort of slightly more animistic. Like, so what's going on with the tree? And you're like, maybe not talking about it as a...”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I kind of expect to, I do actually kind of expect to continue to care about something like consciousness quite a lot on reflection and to not kind of end up deciding that my ethics is better, doesn't make any reference to that, or at least there's some things like quite nearby to consciousness. When I stub my toe and I have this, something happens when I stub my toe. It's unclear exactly how to name it, but I'm like, something about that, you know, I'm like pretty focused on. And so I do think.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“So there's a class of people who are called illusionists in philosophy of mind who will say consciousness does not exist. And this is sort of different ways to understand this view. But one version is to sort of say that the concept of consciousness has built into it too many preconditions that aren't met by the real world. So we should sort of chuck it out like Elaine Vitale. Like instead of the sort of proposal is kind of like at least phenomenal consciousness, right? Or like qualia or what it's like to be a thing. They'll just say this is like sufficiently broken, sufficiently chock full of falsehoods that we should just not use it. I think it feels to me like I am like there's really clearly a thing there's something going on with you know like I'm kind of really not”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Fully necessary criteria. I mean, clearly, I definitely have the intuition consciousness matters a ton. I think if something is not conscious and there's like a deep difference between conscious and unconscious, then I'm like definitely have the intuition that there's something that matters especially a lot about consciousness. I'm not trying to be like dismissive about the notion of consciousness. I just think we should be quite aware of how it seems to me how ongoingly confused we are about its nature.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Then this is going to have been like a bad thing to build our entire ethics around. And so now to be clear, I take consciousness really seriously. I'm like, I'm like, man, consciousness. I'm not one of these people like, oh, obviously consciousness doesn't exist or something. I'm like, but I also notice how confused I am and how dualistic my intuitions are. And I'm like, wow, this is really weird. And so I'm just like error bars around this. Anyway, so that's like one, there's a bunch of other things going on in my like wanting to be open to kind of not making consciousness. Like there's kind of”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Cellular rhetimida, right? That is sort of self replicating. It has some information, and you're like, okay, is that alive? It's kind of like, it's not that interesting. Is it kind of verbal question, right? Or I don't know, philosophers might get really into like, is that alive, but you're not missing anything about this system, right? It's not like there's no extra life that's springing up. It's just like, it's alive in some senses, not alive in other senses. And I think if you, but I really think that's not how we intuitively think about consciousness. We think whether something is conscious is a deep fact. It's this like additional, it's like this really deep difference between being conscious or not. It's like, is someone home? Is the lights are on? Right. And I have some concern that if that turns out not to be the case.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Even if you're like, no, no, there's no such thing as a laundry tale, but life, surely, life exists. And I'm like, yeah, life exists. I think consciousness exists too, likely, depending on how we define the terms. I think it might be a kind of verbal question. Even once you have a kind of reductionist conception of life, I think it's possible that it kind of becomes less attractive as a moral focal point, right? So right now we really think of consciousness where like, it's a deep fact. It's like, so consider a question like, okay, so take a.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I think that I don't know where this discourse leads. I just am suspicious of the amount of ongoing confusion that it seems to me is present in our conception of consciousness. Sorry, I sometimes think of analogies with like, you know, people talk about like life and like Alain Vital, right? And maybe. There's a world, Alan Vital was this hypothesized life force that is sort of the thing at stake in life. And I think we don't really use that concept anymore. We think that's a little bit broken. And so I don't think you want to have ended up in a position of saying everything that doesn't have a Lan Vital doesn't matter or something, right? Because then you end up later. And then somewhat similarly,”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I think there's also room for division of labor, right? Like, I think there can be, yeah, there are people who are trying to draw a bunch of pieces and then be like, here's the overall picture. And then people who are going really deep on specific pieces, people who are doing the more generative, throw things out there, see what sticks. So I think it also doesn't need to be that all of the epicemic labor is located in one brain. And it depends your role in the world and other things.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“An epistemic project to be like, wait, why exactly do I think that's false, right? And you really, you know, someone's like healthcare doesn't work. Medical care does not work, right? Someone says that and you're like, all right, how exactly do I know that medical care works, right? And you go through the process of trying to think it through. And so I think there's like room for that. But I think ultimately the real profundity is like true, right? Or like kind of things become less interesting if they're just not true. And I think that's, I think sometimes it feels to me like people. It's at least possible, I think, to lose touch with that and to be more flashy and it's kind of like, eh, this actually isn't, there's not actually something here, right?”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Thinking. And, you know, cool. But like if something's really important, let's just get it right. And I think, and sometimes it's like boring, but It doesn't matter. And I also think Stuff is less interesting if it's false, right? Like, I think if someone's like bra and you're like, nope, I mean, it can be useful. I think sometimes there's an interesting. Process where someone says, like, blah, provocative thing. And it's a kind of”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Throw stuff out there, right? Try to like what if it's like this or like try this on, or I have a hammer. I will hit everything. What if I just hit everything with this hammer, right? And so I think some people do that. And I think there is room for all kinds. Kind of think the thing where you just get it right. Is kind of undervalued, or I mean, it depends on the context you're working in. I think certain sorts of intellectual cultures and milieu's and incentive systems, I think incentivize saying something new or saying something original or saying something like flashy or provocative and then like kind of various cultural and social dynamics and like, oh, like, you know, and people are like doing all these kind of performative or statusy things. Like there's a bunch of stuff that goes on when people.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Virtue that I actually think is closely related in my head with some kind of seriousness and sincerity. I do think there's a different dimension, which is there's like kind of Trying to get it right. And then there's kind of like.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“To curate your kind of intellectual influences too rigidly in virtue of some story about what matters. I think it is good for people to like. Have space. I mean, I'm really a fan of, or I appreciate the way, like, I don't know, I try to give myself space to do stuff that is not about like, this is the most important thing. And that's like feeding other parts of myself. And I think, you know, parts of yourself are not isolated. They like feed into each other. And I think a better way to be a kind of richer and fuller human being in a bunch of ways. And also just like these sorts of data can be just really directly relevant. And I think some people I know who I think of as like quite intellectually sincere and in some sense quite focused on the big picture also have a very impressive command of this very wide range of kind of empirical data and they're like really, really interested in the empirical trends and they're not just like, oh, you know, it's a philosophy or, you know, sorry, it's not just like, oh, history, it's the march of reason or something. No, they're like really, they're really in the weeds. I think there's a kind of in the weeds.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, it might be worth distinguishing between something like kind of intellectual seriousness and something like How diverse and wide ranging and kind of idiosyncratic are the things you're interested in, right? And I think maybe there's some correlation where people who are kind of like, or maybe intellectual seriousness is also distinguishable from something like shooting the shit. Like maybe you can shoot the shit seriously. I mean, there's a bunch of different ways to do this, but I think having an exposure to all sorts of different sources of data and perspectives seems great. And I do think it's possible.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“The world is made of. And so if you don't have those, you don't have data at all. Yeah, it seems like there's some skill in learning history well.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, looking at details, looking at macro trends, and that's a dance. I do think it's nice. I think it's nice for people to be at least attending to the kind of macro narrative. I think there's some virtue in having a worldview, like really building a model of the whole thing, which I think sometimes gets lost in the details. But obviously, like if you're too, you know, the details are what.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I think there's kind of very little excuse for not Learning history, or I don't know, or sorry, I mean, I'm not saying I have learned enough history. I guess I feel like even when I try to channel some sort of vibe of skepticism towards great works, I think that doesn't generalize. To like thinking it's not worth understanding human history, I think human history is just so clearly. That's crucial to kind of understand what's structured and created, all of the stuff. And so there's an interesting question about what's the level of scale at which to do that, right? And how much should you be?”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“And then, but as a result, can miss some of the genuine value. But I think they're responding to a real failure mode on the other end, which is to kind of, yeah, be too enamored of this prestige and sacredness to kind of siphon it off as some like weird legitimating function for your own thought instead of like thinking for yourself losing touch with like what it what do you actually think or what do you actually learn from like i think sometimes you know these epic epigraphs careful right i mean it's like i think you know and i'm i'm not saying i'm immune from these vices i think there can be a like ah but bob said this and it's it's like oh very deep right and it's like these are humans like us right and i think i think the canon and like other great works and you know all sorts of things have a lot of value and you know we shouldn't i think sometimes like borders on the way people like read scripture or i think like there's a kind of like scriptural authority that people will sometimes like ascribe to these things and i think that's not um so yeah i think it's kind of you know you can fall off on both”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“So, on just the general question of how should people value great works or something? I think people can kind of. Fail in both directions, right? And I think some people, maybe like SPF or other people, they're sort of interested in puncturing a certain kind of sacredness and prestige that people can try to kind of like. Yeah, that people associate with some of these works. And I think there's a way in which”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I don't want to speak. I think some rationalists, you know, lots of rationalists love these different things. I do think.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I actually find it's kind of quite complimentary. Yeah, I will write these sort of more technical. Fully optimizing for like trying to do something impactful or trying to kind of kind of Yeah, there's kind of more of an impact orientation there. And then on the kind of essay writing, I give myself much more leeway to kind of just let other parts of myself and other parts of my concerns kind of come out and kind of, you know, self-expression and aesthetics and other sorts of things. Even while they're both, I think for me part of an underlying kind of similar concern or an attempt to have a kind of integrated orientation towards the situation.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that's very plausible, but I do think it's like an additional thing you could get for free or get quite commonly, depending on its nature, is something like pleasure. Again, and then we have to ask, how janky is pleasure? How specific and contingent is the thing we care about in pleasure versus how robust is this as a functional role in minds of all kinds? And I personally don't know on this stuff. And I don't think this is enough to get you alignment or something, but I think it's at least worth being aware of these other features. We're not really talking about the AI's values in this case. We're talking about the kind of structure of its mind and the different properties the minds have. And I think that That could show up quite robustly.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Something that's quite structural. It's much more defined by functional roles like self awareness, a concept of yourself, maybe higher order thinking, stuff that you really expect in many sophisticated minds. And in that case, okay, well, now actually consciousness isn't as fragile as you might have thought, right? Now actually like lots of beings, lots of minds are conscious. And you might expect at the least that you're going to get conscious superintelligence. They might not be optimizing for creating tons of consciousness. You might expect consciousness by default. And then we can ask similar questions about something like valence or pleasure or like the kind of character of the consciousness, right? So there's you can have a kind of cold indifferent consciousness that has no human or no like emotional warmth, no pleasure or pain. I think that can still be Dave Chalmers has some papers about like Vulcans and he talks about they still have moral patients.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and I think people have different views about this. So one possibility is that human consciousness, like the thing we care about in consciousness or sentience, is super contingent and fragile and most minds, most kind of smart minds are not conscious, right? It's like the thing we care about with consciousness is this hacky contingent. It's like a product of specific constraints evolutionarily genetic bottlenecks, et cetera. And that's why we have this consciousness. And you can get similar work done. So consciousness presumably does some sort of work for us, but you can get similar work done in a different mind in a very different way. And you should sort of, so that's like, that's the sort of consciousness that's fragile view, right? And I think there's a different view, which is like, no, consciousness is.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“Kind of mode with respect that looks to me much more like a there's also questions like does it try to kill you and stuff like that but I think that there are kind of features of the agents we're imagining other than the kind of thing that they're staring at that can matter to our sense of like sympathy similarity and”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“If you imagine the paper club is like a conscious being that loves paperclips, right? It takes pleasure in making paperclips. That's like a different thing, right? And obviously it could still, it's not necessarily the case that like, you know, it makes the future all paperclippy is probably not optimizing for consciousness or pleasure, right? It cares about paperclips. Maybe eventually if it's suitably certain it turns itself into paperclips. And who knows? It's still, I think, a different, it's actually a somewhat different moral.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source
“I'm not sure about that. I'm not sure I do care about a notion of intellectual descendant in that sense. If you imagine, I mean, literal paperclips is a human concept, right? So I don't think any old human concept will do for the thing we're excited about. I think the stuff that I would be more interested in the possibility of getting for free are things like consciousness, pleasure, sort of other features of human cognition. I think there are paper clippers and there are paper clippers, right? So imagine if the paper clipper is like an unconscious, kind of voracious machine. It's just like, it appears to you as a cloud of paperclips and there's nothing sort of, that's like one vision.”
2024-08-22 · Dwarkesh Podcast · Joe Carlsmith — Preventing an AI takeover · IDENTIFIED FROM THE TRANSCRIPT · source