YouSaid · the spoken record

Joseph Carlsmith

lines on the record: 86
first: 2022-08-03
most recent: 2022-08-03
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Put very little credence in those hypotheses Yeah, I don't see a lot of reason to think that. I see a good amount of reason not to think it. But it wasn't something I dug in on a ton.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Them independently and then add it up. They're not actually multiplicative processes on that conception. I do think there are kind of correlations between the estimates for the different parts, but it's sort of additive at a fundamental level.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so the way I was understanding and breaking down the forms of processing that you would need to replicate in the brain made them seem not multiplicative in this way. So an example would be if you think about simple examples. Suppose we have some neurons and they're signaling centrally via spikes through synapses or something like that. And then we have glial cells as well, which are signaling via like slower calcium waves. And it's a sort of separate network. You could think that if it were something like, you know, the rate of calcium signaling is dependent on the rate of spikes through synapses or something like that. Then that's an important interaction. But overall, if you sort of imagine like this kind of network processing, these are kind of, you can just, you can ask.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“More willing to say stuff like, well, we kind of basically know what's going on in a mechanistic level, which isn't the same as knowing kind of the algorithm, the sort of algorithmic organization overall and how to replicate it. I sort of lean towards the latter view, though I give weight to both and try to synthesize the kind of opinions of people I saw overall.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“There's definitely disagreement, and it was interesting and in some sense disheartening to talk with neuroscientists about just how difficult neuroscience is. I think it's easy, a consistent message and I have a section on this in the report, was kind of how far we are from really understanding what's going on in the brain, especially at a kind of algorithmic level. So in some sense, the report is somewhat opinionated in that, you know, there are experts that I found more compelling than others. There are experts who are much more in a sort of agnosticism mode of like we just don't know the brain is really, really complicated who sort of err on the side of very large compute estimates. A lot of emphasis on biophysical detail, a lot of emphasis on sort of mysterious things that could be happening that aren't happening. And then there are other neuroscientists who are more”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Which tries to extrapolate from the communication capacity of the brain to its computational capacity using comparisons with current computers. So it's sort of a triangulation of like you look at a bunch of different sources of evidence, all of which, in my opinion, are pretty weak. I think we're quite, well, the physical limit stuff is maybe more complicated, but it's sort of an upper bound. I think we are significantly uncertain about all of this. And my distribution is pretty spread out. But the hope is that by looking at a bunch of things at once, you can at least get a sort of educated guess.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“So I used a few different methodologies and tried to kind of synthesize them. So, one was looking at the kind of mechanisms of the brain and what we know about the kind of complexity of what they're doing and how hard it is to capture the kind of task relevant or our best guess about the task relevant dimensions of the signaling happening in the brain. And then I also tried to bring in comparisons with existing AI systems that are replicating kind of chunks of functionality that humans that the human brain has and in particular in the context of vision. So sort of how do our current vision systems compare with the parts of the brain that are kind of plausibly doing analogous processing, or they're often doing other things as well. And then I use the third method, which has to do with physical limits on the kind of energy consumption per unit computation that the brain is possibly doing. And then a fourth method I sort of just”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Lot of open questions as to how hard that is. But I do think this methodology is a lot more compelling and interesting if you are compelled by the kind of available techniques in deep learning and by kind of scaling hypothesis like views, at least as an upper bound. I think it's important. So there's different ways of kind of being interested in algorithmic breakthroughs. One is because you think deep learning isn't enough. Another is because you think they will sort of provide a lot of efficiency relative to deep learning such that an estimate like Ajaya is an overestimate because actually, you know, we won't have to do that. We'll make some sort of breakthrough and it'll happen a lot earlier. And I put weight on that view as well.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so I think the approach does assume that you can train a human level or sort of transformative AI system with a sort of non-astronomical amount of compute and data using current without major conceptual or algorithmic breakthroughs relative to what's currently available. Now, the actual methodology of JA uses allows you to assign probabilities to that assumption too. So you can, if you want, you know, say, I'm only 20% on that. And then you have, then there are sort of other, there are a few other options so you can also kind of rerun evolution, which is not, and so that's an anchor that she provides as sort of, and this is often what people will say as a sort of upper bound on how hard it is to create human level systems, is to do something analogous to simulating evolution, though there are”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so my focus in that report was not on the training extrapolation. That was work that Ajaya Catra at Open Philanthropy did using my reports estimate as an input. And her methodology involves assigning different probabilities to different kind of ways of using that input to drive an”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Brain or the sort of size of a kind of AI system and it's kind of overall parameter count and kind of compute capacity and that would be sort of analogous to humans and then you extrapolate from that to the training cost the cost to kind of create a system of that kind using current methods in machine learning and kind of current scaling scaling laws and that methodology though brings in a number of additional assumptions that I think aren't like just transparent that that's oh yeah of course that's how we would do it or that and so I think you have to sort of be a little bit more in the weeds to see exactly how it how it feeds in”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so the approach I took was to look at the evidence from neuroscience and the literature on the kind of computational capacity of the human brain and to talk to a bunch of neuroscientists and to try to see what we know right now about the number of floating point operations per second that would be sufficient to kind of reproduce the task relevant aspects of human cognition in a computer. And that's important. I mean, it's actually not clear to me exactly how important this parameter is to our overall picture. I think the way in which it's relevant to thinking that I've been doing and that OpenField has been doing is as an input into an overall methodology for estimating when we might see kind of human level AI systems that proceeds by first trying to estimate roughly the kind of computational capacity.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Some extent, I think my main criticism, or the first thing that would come to mind is that I think we will very likely, like I think competitive pressures are a source of kind of pushing the world in bad directions, but I also think there are ways in which kind of wise forms of coordination and kind of preemptive action can stave off the sort of bad effects of competitive pressures. And so that's a sort of that's the way I imagine avoiding stuff in the vicinity of what Robin is talking about. There are a lot of complexities there.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I would not call Robin's world a utopia. And so, you know, a thing I haven't been talking about is what should our overall probability distribution be with respect to different quality of futures? exactly how possible is it and how likely is it that we build something that is sort of profoundly good as opposed to mediocre or much worse and I would class robin scenario in the mediocre or much worse zone”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“So I think there Definitely analogs and similarities between some ways of relating to the notion of utopia and attitudes and orientations that are common in religious contexts and spiritual contexts. And I think it's, and I think personally, so I don't think it needs to be. But I actually think there are a number of really important differences between utopia and something like heaven. So centrally, utopia will be a sort of concrete, limited situation. There are going to be frictions, there are going to be resource constraints. It's going to be finite. There's a bunch of, it's still going to be in the real world. Whereas I think many most religious visions don't have those constraints and that's an important feature of their situation.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Much suffering is good to have, and a lot of the sort of discourse on utopia, I think gets distracted from basic facts about, at the very least, we can do just a ton better. And that's important to keep in mind.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so I mean, I certainly don't have a kind of hard threshold. Here's exactly where I'm going to call it utopia. I mean something that is profoundly better. I do think that if you have a finite, so very basic level, if there's only a finite number of states that the sort of effectable universe can be in and your ranking of these states in terms of how good they are is transitive and complete, then there will be a sort of top. And I don't think that's an important thing to focus on from the perspective of just getting it, just taking seriously that things could be radically better at all. I think like talking about exactly how good and what's the perfect thing is often kind of distracting in that respect. And it gets into these issues about like, oh, you know, how.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Think this is a really important feature of our situation. I think it's true that it can be dangerous, and if you're wrong about it or if you're acting in a sort of unwise way with respect to it, that can be really bad. But I also think it's just a really basic fact. And I think we just sort of need to learn to deal with it maturely and kind of pretending it's not true I think isn't the way to do that.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“That's a good point. So maybe the relevant notion of utopian here is something like to what extent is a concept of a radically better world kind of operative in your day-to-day engagement? To some extent, what I meant is that I think if I sat down and talked with most people, we could eventually with some kind of constraints on reasonableness come to agree that things could be a lot better in the world. Like we could just cure cancer. We could cure XYZ disease. We could just go through a few things like that. We could talk about the degree of abundance that could be available. And I think the question is whether that's like a kind of structuring or important dimension to how people are relating to the world. And I think you're right that it's often not. And that's part of maybe the thing I'm hoping to kind of push back against with that post is actually.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“That doesn't in some sense imply that it could be a lot better, or at least a plausible view of the world. And when I say utopia, I don't really mean anything much different from that. I think it's sort of, I'm not saying a perfect thing. I do have sort of a more specific view about exactly how much better things could be. But more broadly, it seems to me many, many people believe in the possibility of a much better world and are fighting for that in different ways. So I wouldn't pin the red flag specifically to the belief that sort of things can be better. I think it would have more to do with sort of what degree of rigidness are you relating to that belief with, how are you kind of how are you acting on it in the world? How much are you willing to kind of break things or kind of act in uncooperative ways in virtue of that sort of conviction? And there, I think.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so I think there are certainly reasons to be cautious in this broad vein. I don't see them as very specific to EA or I don't identify as utilitarian, but to utilitarianism, I see them as more sort of better understood as risks that come from believing that something is very important at all. And I think it's true that many acting from a space of conviction, especially where that conviction has sort of a flavor of, you know, it's interesting what exactly constitutes an ideology, but I think it's reasonable to look at EA and sort of be like, this looks like an ideology. And I think, you know, and I think that's, I think that's right. And I think that's sort of important to, you know, have this sort of relevant red flags about. I think it's pretty hard to have a view of the world.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Is the question something like is this a red flag kind of, you know, we look at other ideologies throughout history and they've been compatible with utopian thinking and maybe sort of effective altruism or utilitarianism or something is similarly compatible. So should we worry in the same way? Is that the question?”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Minds can be and do and see. And so I want to sort of chart that territory before we start making kind of big and irreversible decisions about what sort of civilization we want to build in a long term.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't assume that that's what utopia is about at all. I think we're at, I think, A, my own values seem to me quite complicated. I don't think I just value pleasure. I value a lot of different things. And more broadly, I have a lot of uncertainty about how I will think and feel about things if I were to go through a kind of process of significantly increasing my capacity to understand. I think sometimes when people imagine that, they imagine, oh, we're going to sit around and do a bunch of philosophy and then we'll have like solve normative ethics and then we'll implement our solution to normative ethics. And that's not what I'm imagining by kind of wisdom. I'm imagining something richer and also that involves importantly a kind of enhancement to our cognitive capacity. So sort of really, you know, I think we have very small or really limited in our ability to understand the universe right now. We have kind of, and I think there's just a huge amount of uncharted territory in terms of what.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so I definitely don't sit around thinking that utopia, we sort of know what utopia is right now and it's hedonium. I'm not especially into the notion of hedonium, though I think it's possible to brand is bad. I think people talk about pleasure with this kind of dismissive attitude sometimes. And hedonium implies this kind of sterile uniformity and you're sort of tiling people talking about they're like going to tile the universe with pseudonium and it's like, wow, this sounds rough. Whereas I think actually, you know, the relevant perspective when you're thinking about something like hedonium is the kind of internal perspective from which the sort of experience of the subject is something kind of joyful and boundless and kind of energizing, you know, whatever pleasure is actually like, pleasure is not a trivial thing. I think pleasure is a profound thing in a lot of ways. But I really...”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“So I think there's at least a tension, and the way I see the tension playing out or, you know, kind of being reconciled is specifically via the notion I referenced earlier of kind of you would, if you truly understood, come to see the utopia as genuinely good. But I think that process, I mean, ideally, I think the way we end up building utopia is we go through a long patient process of becoming wiser and better and more capable as a species. And it's in virtue of that process kind of culminating that we're in a position to build a civilization that is sort of profoundly good and radically different. But that's a long process. And so I do think, as I say, if I just transported you right there and you skipped the process, then you might not like it. And it is quite alien in some sense, but I still, but if you went through the process.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Know you have a kind of crappy job or like a beach vacation, and utopia is like everyone has beach vacation. And I don't know how you feel about beach vacations, but I think the difference is more like being asleep and being awake or sort of it's more, yeah, it's sort of, it's like living in a cave or living in under the open sky. I think it's like a really big difference and that that matters a lot.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so Utopia for me just means a kind of profoundly better future. And I think it's important because I think it's just actually possible. I just think it's actually something that we could do. We could make, if we sort of play our cards right in sort of non crazy ways, we could just build a world that is radically better than the world we live in today. In particular, I think we often in thinking about that sort of possibility underestimate just how big the difference in value could be between our current situation and kind of what's available. And I think often utopias are kind of anchored too hard on the status quo and sort of changing it in small ways, but imagine our kind of fundamental situation basically unaltered. And I think such that it's a little bit like the difference between”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Personally, I have a kind of nostalgia oh, you know, sure, it was risky, but it made me who I am or something like that. I also want to say, you know, I think it's true that when you look back on your teenage years, there is often a sort of, you know, let's say you did some like crazy, you and your friends, you start race, you know, around and you played chicken or something at the local quarry. And it's like, oh, all right, right. But, you know, you survived, right? And the real reason not to do that is the chunk of probability where you just died. And so I think there's a, you know, to some extent, the ex post perspective of looking back on certain sorts of risk is not the right one, especially for death risks. That's not the right perspective to use to kind of calibrate your understanding of how to feel about it overall.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“I doubt that they will look back with pleasure at the sort of risks and horrors of the 21st century. I mean, if you just think about how we, or at least I, tend to think about something like the Cuban Missile Crisis or World War II. I don't.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“So I think there's a couple different questions there. One is what would the experience be if you had it from the inside? And then there's, I think, a subtly different question, which was what would your take on this be if you kind of fully understood, where fully understanding is not just a matter of having the internal experience of being in a certain situation, but it's also a matter of understanding what that situation is causing, what sort of beliefs are structuring, the ideology, whether those beliefs are true, and all sorts of other factors. And it's the latter thing that I have in mind. So I'm not just imagining, oh, the feature will feel good if you're there because, you know, sort of by hypothesis, the people who are there, at least one hopes they're enjoying it, or one hopes they're thumbs up. If the people who are there aren't thumbs up, that's a strange utopia. But I'm thinking more that in addition to their perspective, there's a sort of more holistic perspective, which is the”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Is that the best futures are going to be such that if you really understood them and if you really experienced what they're like, which may be a big step and might require sort of extensive engagement and possibly sort of changes to your capacities to understand and experience, then you would think it's really good. And so I think that's the relevant standard. So for me, I worry less if the future is sort of unitially alienating. And the question for me is how do I feel once I've really, really understood what's going on?”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“So, yeah, a few comments there. So, one, I do think that if you took, you know, for most people throughout history, if you brought them to the present day, my guess is that fairly quickly, depending on exactly the circumstances, they would come to prefer living in the present day to the past, even if there are sort of a bit of future shock and a bit of some things are alienating or disturbing. But that said, I think the distance, the sort of gap between historical humans and the present is actually much, much smaller, both in terms of time and kind of other factors than the gap I envision between present-day humans and the future humans who are living ideally in a kind of radically better situation. And so I do expect sort of greater distance and possibly greater alienation when you first show up. My personal view is”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“I think there are a number of implications just from understanding timelines with respect to how you prioritize and just to some extent the sooner something is, then you need to be planning for it coming sooner and kind of cutting more corners or counting less on having more time. And yeah, I think overall the higher you think the probability of catastrophe is, the easier it is for this to become kind of the most important priority. I do think there is a range of probabilities where it maybe doesn't matter that much, but I think the difference between say one and 10%, I think, is quite substantive and the difference between 10 and 90 is quite substantive. And, you know, I know people in all of those ranges.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“So it's a mix of things. Right now, I'm thinking about AI timelines and what's called takeoff speeds, sort of how fast the transition is from pretty impressive AI systems to AI systems that are kind of radically transformative. And I'm trying to use that to provide more perspective on the probability that everything goes terribly wrong”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source
“Sure. So I work at open philanthropy on existential risk from artificial intelligence. And so I think about what's going to happen with AI, how can we make sure it goes well and are predicted, but how can we make sure that advanced AI systems are safe? And then I have a side project, which is this blog where I write about philosophy and the future and things like that. And that emerges partly from a sort of my background, which is I was before getting into AI and working at open philanthropy. I was in academic philosophy.”
2022-08-03 · Dwarkesh Podcast · Joseph Carlsmith - Utopia, AI, & Infinite Ethics · IDENTIFIED FROM THE TRANSCRIPT · source