YouSaid · the spoken record

Kate Crawford

lines on the record: 45
first: 2025-09-25
most recent: 2025-09-25
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Like to bring up, right? Because that is kind of one of the core concepts of safety. Is the smallest catastrophe, not the smallest catastrophe, like the smallest hazard can cascade into a large catastrophe. So if you're not able to address the things that are considered, you know, they consider this stuff inconsequential, then you're never going to be able to prepare for these, you know, much more large-scale events that they talk about. And that's very much like a standard safety perspective to have.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Regulatory system then becomes about risks that we have yet to see, you might as well not have regulation at all, right? So I think a lot of people talk to me about, well, what do you think of this framework and what do you think of that framework? I'm like, to me, practically speaking, as someone who has done risk assessment, it is equivalent of having no regulation because we're actually not addressing the risks of the harms that AI is posing. And we're in fact focusing on hypothetical risks. And there's this idea that I've heard before, well, what if those risks come true and you're unprepared? And the way that I see it is that if you're not prepared for today's risks and you're not building their frameworks for that, you're not going to be prepared for future risks because these frameworks and risk assessments built on top of each other. So if you're not able to mitigate for the lack of safety and security of AI models today, then you have no chance of mitigating against these hypothetical risks that people”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I think it is the existential risk, this concern that AI will somehow become self-aware or have these capabilities that lead to nuclear proliferation. And as someone who has worked on sort of risk assessments now for about a decade, you have to have real data to back up your claims. And so when you're then using risk assessment frameworks to try to substantiate hypothetical claims that there are no proof for, you're not doing science. To me, I'm willing to be convinced that perhaps AI models could have CBRN capabilities in the future. I am not opposed to that idea, but they don't have them now. And so for all of us to put our safety and regulation efforts, and that includes by the US government and the UK governments, to be about hypothetical risk that can't be measured, right, that can't be quantified or qualified. And our entire”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“World, right? And so this idea of existential risk, CBRN, alignment to me was like, no, but these are not the current harms that we're going to see if we're going to deploy AI systems in the CFT critical situations. And if we are going to deploy them in safety-critical situations like defense, we have to assess these systems as we always have for every other system. I thought introducing something like risk assessments would be helpful to the field because then people could understand the risks that come from using the systems, but what it ended up unfortunately evolving to and being used by many labs is that these risk assessments are now sort of being used like the end all be all of all assessments of AI being used in all systems everywhere. And that I regret very much, but that was never sort of the intention to begin with when we set out this work.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“For codecs, the idea was to introduce something like a risk assessment for AI, which is not what people were doing before. Prior, there was a lot of benchmarking, right? And these benchmarks didn't really consider the risks that the AI system poses with having specific capabilities. So the idea was to really try to investigate that and use some techniques inspired from safety critical fields. It was not meant to be a replacement for assessments for safety critical systems, right? And I think that's a really, really big distinction. This idea that they're pushing of general safety just does not align with how safety actually should be assured and sort of the real.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Simply don't work. I'm being sarcastic. That's not the case. And then the way that they define safety, again, is through the safety revisionism, right? They call it something else. It ends up being about something else and completely disconnected from actually being accurate for military operation. It ends up being, again, these high-level ideas of safety that we're seeing them push, whether it's about like CBRN. It's like, right, but can it do the thing that we're asking the AI systems to do? You actually never see an assessment of that in a lot of these frameworks. And so this is sort of why this idea of safety becomes very confusing because it has diverted so much from how we've traditionally used it to assess like nuclear plants or airplanes and so on.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“AI and the safety that they push forward is problematic because they want us to adopt this idea of universal or general safety that has to do with, as they call it, alignment. And it's this misguided idea that there exists like a universal safety solution that would make all general functionality of all LLMs safe, right? And this is also one of the ways that procurement is changing in the military where you're looking at companies like Scale AI. They are putting forward these types of general frameworks, but there is no standard safety approach to generic systems in any domain. In fact, this would contradict established safety practices that require sort of a well-defined use case to map risks against. And so often what we're seeing now happen is that companies like Scale AI, they say we're going to build a risk assessment framework for AI systems because the existing ones.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“The threshold is that depends on every single field because kind of the impact of the system will vary. An airplane crashing is very different from a nuclear plant crashing. So the nuclear plant doesn't have this idea that 300 people dying is the worst case scenario. In fact, it's much more than that and also has to do with environmental like nuclear disaster, right? And so this is why this idea of”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“They're meant to operate basically well, 99 to 99.999% of the time, depending on the kind of system that you're looking at, right? And the safety criticality. And then when that system fails, we then have to have mitigations in place, right? And there will be risks with that. And typically these mitigations are based on, like I said, these thresholds of how many people could be harmed. So in the case of like airplanes, I think that's a very simple example. A catastrophic incident is considered if everyone on a commercial airplane dies. So typically that number is like 300 people get harmed or die. That's the threshold for aviation as being the most catastrophic thing that could happen. So safety is actually very specific to the use cases. What we mean by 99.99% reliability often relates to these systems failing. But if you're looking at how to actually mitigate for those risks and what”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I mean, it's a lot more technical than that. But basically, right, we have these thresholds of these systems have to be accurate and be able to perform. So these are typically what we call reliability and availability measures of the system. So they can only fail, often even 99% is like one of, it's like the lowest threshold for a nuclear plant. It even goes up to 99.99%, right? And so obviously if we allow zero risk, we're never going to build anything, right? Like I think that's very important to remember is that with every technological system that there is some sort of risk. But you have to mitigate for when those systems fail. So this idea that our safety critical systems have this like 99.99% reliability means that”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Let's talk a little bit about earlier you mentioned safety of 99%, for example, at a nuclear power plant. What does that mean in context? We talk just now about the meaning of safety in that regard, but what would a safety of 99% entail? It's that there's a 99% chance that it won't harm people or the environment? Or, you know, what does that mean in practice?”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Of giving them permission to not only decide what counts as safe enough, which again breaks these democratic norms that we've had, but it also lowers and undermines existing safety threshold that would have otherwise regulated AI use in things like defense. So ironically, you know, this is kind of their way of how they bypass some of the safety measures that I've talked about earlier in that they're looking for this pot of money, right, from the military, but the safety thresholds for defense are extremely high. So what do you do? Well, you redefine what”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“This right, which a lot of governments have, they've sort of ceded that control of defining what safety is to them, puts them in a position to define what a risk threshold is or what actually safe enough means. And the entire idea of risk thresholds, because I imagine a lot of people might not know this, is to provide sort of a metric or a measure of the level of risk exposure that our society collectively agreed to take. And this often shapes how we determine the safety of technological systems, including nuclear plants. And typically this is done through a democratic process that we have established over decades in other high-stakes scenarios, like all of our thresholds for other safety critical systems have come from sort of democratically determined idea of what society thinks safety is. So in allowing AI companies to sort of co-op these traditional safety terms, we've”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“An assurance defense and safety critical systems, but instead redefine those safety techniques with washed down alternatives that actually accelerate the deployment of inaccurate AI in high-risk scenarios like defense or nuclear. So, for example, AI companies often reduce the term safety to now mean alignment or existential risks. Now, this is pretty distinct, right, from the definition of safety that I just gave you, because alignment focuses on human preference, right? And that makes us question, well, alignment with whom and which humans, right, and whose preferences and the existential risks that are also emphasized, like CBRN are hypothetical, like I mentioned, and are often used as sort of a pretense for an AI arms race to ignore other risks and safety thresholds like surveillance systems, right? So in allowing AI companies to”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“So if we take a step back and not think about what the AI companies have been telling us, what safety means for the past four years, or even more than that at this point, safety has historically meant, especially in the context of safety-critical systems, ensuring no harm to humans or the environment. So if you're thinking about aviation or nuclear power plants, for example, you want to ensure that when your systems fail and systems do fail, humans are not harmed, that there's no death and that there's no environmental catastrophe. It's quite a simple definition. Now, what is happening in terms of what safety now means is very different, and it has been redefined by AI companies as of late. So I believe AI labs engage in what I call safety revisionism, where they use the same safety terminology that are often used for regulating.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Back with Chief AI scientist Heidi Klaff discussing the ways in which AI companies are pushing into defense contracting. Before the break, we were talking about how real the risk is that AI systems might be used to develop nuclear or biological weapons. But now I want to zoom out and talk to Heidi about the broader field of AI safety and how she thinks it's changed since she worked with OpenAI years ago. Let's shift and talk about AI safety for a bit. So you helped establish and pioneer the field of AI safety engineering. What's the technical meaning of safety, like with your background? And how has the AI safety world changed that meaning or how has it become more colloquial now?”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Most people are thinking about as before this idea that they're somehow going to gain CBRN capabilities by themselves very hypothetical and not really tied to the reality that we're in right now with AI systems. But if we're taking AI systems today and we train them on sensitive military data, I have a concern of how those systems are going to be used.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Certainly, shouldn't it be AI systems because regardless of the data distribution, these systems are highly flawed and they're always going to have inaccuracy. As I mentioned before, often when we're looking at military systems that deploy AI, they can have as low of an accuracy rate as like 20%. And if you're being really optimistic, maybe 60 to 80%, right? These are the levels of accuracy that you're looking at with these types of systems. And so then to train a model on CBRN data, and then to then attempt to use it for those tasks is extremely dangerous when you're looking at those accuracy rates, right? For me, the concern that I have is that they will then think that these models are reliable because we train them on that set of data, and thus we can then use them and the military and in defense operations to dictate decisions about those types of systems and where they should be used and when. And I think that's a very different type of risk.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“We have not seen any proof of CBRN capabilities right now, but those capabilities could come to fruition if we start training very, very sensitive nuclear data, for example, nuclear technologies on these models. And I actually believe that the risk that comes with that is very different from what most people are thinking about. Most people are thinking about that the AI is somehow going to develop weapons by itself, right? Or it will give access to adversarial actors to do so. But even if it's just a military who has access to a model that has been trained on CBRN data, what that means is that they are likely going to use it for those purposes within the military. And that's extremely dangerous. Like if you're thinking about nuclear command and control, right? Who gets to essentially make the decisions about nuclear weapons deployment?”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Okay, so now I want to get into the CBRN side of things. Although, as we've talked about, you know, it may not be as big of a concern as AI companies are making it out to be. It is a big concern for the public, probably because of that marketing and just because it's a scary. So top AI companies say they're increasingly worried about the risk of that. Of course, they're not maybe worried enough to stop building. But I want to get into, again, how big of a risk this is. Let's just go into more detail there.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Systems are being used to some extent, right? We don't know all the details, but governments put out procurement documents of the type of data they want, how they want to use it, and how they want to store it, because then companies can offer services like what their AI can do with that, right? So I do think that it's not the case that they are just selling something commercially like they do to everyone else, and then they hope the military abides by it. It's a much more involved process to do military procurement that often requires testing and evaluation. And the companies typically have to be in the know about the technical details to see if they can offer support for that. Again, as I use a Microsoft case as one of the most recent examples of that being the case with their work with the IDF. So I think it's easy for them to point to terms of service, but as someone who has worked on procurement before, this is not a commercial contract.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“I completely agree with that statement. And I think something that often people miss out on is that militaries do not follow terms of service. They might do that if they're buying like a Microsoft Office suite for their bureaucratic purposes. But when it comes to military procurement, the companies do not have control and they know that over how these systems are being used. And they actually have no say in terms of the terms of service as well. These things get often determined by international law and also by the nation state itself, right? They have the power here. And so when people tell me, oh, but wouldn't that break the terms of service? And it's like, this is not how military procurement works, period, right? And we've also seen examples of, as I mentioned before, Microsoft working directly with militaries to implement some of these systems. So I would say that in a lot of cases, they are well aware of how they're”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“In terms of these companies often have in their mission statements, oh, don't worry, even though we're working with the military on this, this and this, we know for a fact that our tech isn't being used to directly harm people. But yeah, how can they really know that, can they?”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“I wanted to see if you agree with this take I'm about to tell you. So I once interviewed Meg Mitchell from Hugging Face and she said that for these types of military contracts, even if you have in your mission statement like anthropic and open AI do, that your tech can't be used to directly harm others. The problem is that you don't have control in the end over how your tech is actually being used with the military. If you do have any control, you definitely don't have control in the longer term once you already shared that with the military organization, especially without having security clearance and knowing really how it's being used down the line. She also was talking about what's considered direct harm. You know, what if you're summarizing social media posts that then lead to making a list of enemy combatants or potential people of interest that have a certain view on a topic on X, for example? I wanted to see if you agree with that.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Essentially, are sort of aware that they might eventually be used to be fine-tuned for military application and implement backdoors or sleeper agents which trigger specific behavior based on a specific command. And because of that, that's what makes them so unsafe. So sure, you might be able to fine-tune it on specific data that isn't then released publicly, which might remove some vectors of attack. But ultimately, at the end of the day, commercial models are already compromised. So when you're saying that they are more secure, I mean, they're more secure in the traditional security way in that you air gap the system so you kind of limit the control of people who have access to it and so thus people who can probe it and get information out of it but it doesn't remove the fact that commercial models are already compromised from the day that they're built because they're based on public data”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Injections, but this can happen in a sort of deeper level with what we call sort of web poisoning attacks, which can then be used to implement these sleeper agents, as we call them. And so this is in the commercial supply chain, right? The only way that these models are trained is to be trained on sort of mass amounts of data that are publicly available. They're already compromised, right? These models are also fine-tuned through methods like reinforcement, learning, human feedback, which unfortunately uses basically sweatshops of people in developing nations that are paid nothing to then make these models behave in a specific way. And you can imagine a military operation, right? Where a foreign adversary is able to sort of have a covert operation in which they run one of these data labeling and data fine-tuning shops.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“So I wouldn't say they're much more secure. They may be more secure in that they're more air-gapped. So for example, you can take a commercial model, right? And you can fine-tune it on a sort of sensitive military data. And then that model then becomes accessible to the military. But that still misses out on some of the biggest risks of that commercial model is that it was trained on data sets that were publicly available. And so a lot of research has shown that not only can you poison web data that these models are trained on, but you can implement what's called like a sleeper agent, which is given a specific prompt or a command, it will then behave in a sort of a harmful way that sort of the operator of that system did not intend based on something that was implemented in the training data or something that the model was trained on. And so we see this all the time with prompt.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“I also wanted to ask the models these companies use for government products, like their government design products like ClaudeGov, OpenAI's government product, XAI's government product by design have looser guardrails for government use and they're trained to better analyze classified information. And although these types of models allegedly underwent the same type of safety testing as these companies' other models, I'm using Anthropic here as an example. They have certain specifications for national security work. Like they have a greater understanding of intelligence and defense documents and they refuse less when they are asked to engage with classified information that's being fed into them. So in your eyes, how does the development work? How secure really are they? And what are the implications here?”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“We need to take a quick break. We'll be right back. We're back with Heidi Klaff of the AI Now Institute. Before the break, Heidi was breaking down the standard military procurement process and Wyatt feels like AI systems don't meet the rigorous standards we might expect when it comes to being used for high-risk operations. Now, I want to ask Heidi about the specific AI products being sold to the US military and whether they're really much more secure than the commercial models on the market today.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“USUD often are the ones that get to define the terms of how they want to use technology. And typically people abide by it because people want that pot of money, right? It's very lucrative to even be considered up for procurement because it means that you're sort of be on call for them, for potential other technologies. But even to get your foot on the door can take years. So it's a very different type of assessments than I think what people expect for commercial contracts.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“To be often the security thresholds is extremely, extremely high, right? They have to be air gapped, the supply chain has to be completely traceable. They have to know who coded the system, who developed the system, if there's any sort of backdoors that can be compromised, so on and so forth. And once sort of a system goes through that procurement, that safety and security procurement process, the D often just takes complete control of that technology, right? Like this is now something that they possess and are in complete control of using whichever way that they see fit, right? And so this is often why procurement for the military takes can even take many years, right? This is not a process that is meant to take a couple of weeks or even several months. Often this is quite a rigorous process. So this is very different from signing a commercial contract, right? Where you have traditional terms of service, the nation state, not just, you know,”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Military procurement is, well, it really depends, right? There's a huge amount of processes that exist for different types and the processes often strict depending on how critical the technology is going to be used like for example if it's going to be used for lethal operation versus bureaucratic operations, very different types of procurement. If we are looking at a sort of more general idea, typically the government puts out a specific ask for the type of systems that they're looking for and people submit to that procurement ask. Ultimately, these systems often have to go through what we call a testing and evaluation process for them to even be considered even before they sign the contract, right? This process is quite stringent in that it has their specific thresholds on how accurate these systems need to be and how secure they need.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Like 99%, right? That's like the minimum. And often the accuracy of AI systems is like 60% if I'm being optimistic about specific types, right? So there's an enormous gap here to make AI systems as they exist for foundation models be able to satisfy the strict testing and evaluation measures often required by military procurement.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“And we're talking about generative AI systems because AI systems have been used in the military for decades at this point. But these foundation models or large language models, whatever it is that you want to call them, they do not meet the sort of very basic threshold that it's typically expected for a military system, right? And so there is this kind of issue now that they want this pot of money, you know, as I mentioned before, they're trying to de-risk their portfolios through military contracts, but they have this issue where safety as defined by defense and safety critical system is too stringent for their systems to meet just by their nature, right? They're highly inaccurate systems. And when you're looking at, I can't really get into defense systems, but I'm going to talk a little bit about like safety critical system if you're looking at like a nuclear power plant, for example. You're looking at safety of”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“It's definitely both, right? Because, as you mentioned, these companies are pre-profit. And there's a really big pot of money in the military industrial complex. There simply is, and I think that's well known. But also, there's the aspect that these companies would not traditionally pass any of the testing and evaluation required for military procurement. Here's the thing that a lot of people don't know is that defense and military procurement is actually some of the most strict, you know, prior to this AI era, I mean, to sort of evaluate these systems. They have some of the most strict standards. And I think a lot of people assume that's not the case, but often our safety critical systems, if you're talking about like energy infrastructure, you know, and so on and so forth, is derived from those defense standards because of how robust they are. The thing with AI systems”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“So something that comes up a lot in my reporting and in my conversations with other people, even my pitching my editor, is that a lot of these companies are pre-profit. I mean, maybe all of them. And we're seeing a bunch of them unveil government products, enterprise products, and that seems to be where a lot of the money lies. So, you know, obviously OpenAI entropic and XAI have all unveiled government products designed for U.S. defense and intelligence agencies to use. They also all received those government contracts from the DOD. So with companies that burned through cash at super high rates, do we think that the government play is about seeing cold hard cash come in finally or staying above regulatory pressure or both? I'd love to hear your thoughts on that.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“That makes total sense. And that reminds me of how, you know, I think last week Senator Warren sent a letter about XAI's own contract with the DoD and expressing concern that the company hadn't done the same level of safety audits as other companies before receiving that type of contract that they weren't ready. She was worried about how the company would use data it had access to as part of its government partnerships. What did you make of that letter and kind of XAI's, I guess it seemed grandfathered in kind of approach to this DOD contract announcement?”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Risk their portfolio through government. And military contracts. So now we have Pivot from banning military uses always Their main mission being building. That benefit all humanity to now reliance on this narrative of a US-China AI arms race. Initiatives that not only boost Use of AI, but allows them to sort of avoid. And security scrutiny within military applications.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Meta, anthropic, and OpenAI, all announcing U.S. national security work that aligns them Tractors like Palantir, Andrew, and Lockheed. address or previous statements that LLMs or foundation models are unsettled. Defense use. So it was was being created where they behaved as if Always aligned with their mission, right? For example, That US National Security is Under this pretense of an AI arms race with China. For this AI ad Yet The reality of AI”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Well, too many, including myself. Didn't seem like it was Of when OpenAI removed this band in January 2024. Example that Israel at that time was ramping up. Targeting campaign in Gaza that we now know is being supported by Microsoft's cloud services that offer AI and in this case open AI models as extensions of their IT and cloud infrastructure. And so this rollback was really a significier on where AI companies were heading, what they were interested in deploying their technologies, despite OpenAI being well aware of the risk that their models posed in defense and safety critical settings, which is something that I actually worked on with them in one of our papers together when we were looking at evaluation of models like Codex. And interestingly enough, with the announcement of these collaborations that you mentioned and that”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“It reminds me a bit of something more recent, which is how open AI and Anthropic both used to have certain bans on military use of their products, and then they relaxed them. OpenAI walked its band back in January 2024 when it began to work with the DoD on AI tools. And the month before that, it partnered with Anderol. And for Anthropic, it partnered with Palantir in 2024 to offer Claude to intelligence and defense agencies with the US government. So I wanted to ask what you were thinking when you saw these announcements, like made month after month. What did you make of that kind of parade of changes that were happening over a couple of months?”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“First up, I wanted to talk about how AI companies have moved their goal posts a lot with regard to what they're okay with and what their mission statements allow regarding work with the U.S. military and other militaries. So do you remember the whole controversy over Google removing the phrase don't be evil from its code of conduct?”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Scenarios. I also wanted to know what this push to deploy military-grade AI means for bad actors who might want to use AI systems to develop chemical, biological, radiological, and nuclear weapons, a risk the AI companies themselves say they're increasingly worried about. Okay, here's Heidi Klaff on AI in the military. Here we go. Heidi Claff, Chief AI scientist at the AI Now Institute. Welcome to Dakota.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“I removed a ban on military and warfare use cases from its terms of service. Since then, the company has signed a deal with autonomous weapons maker Andorl, and this past June, signed a $200 million Department of Defense contract. And OpenAI isn't a loan. Anthropic, which has a reputation as one of the most safety-oriented AI labs, has partnered with Palantir to allow its models to be used for U.S. defense and intelligence purposes, and it also landed its own $200 million DoD contract. And big tech players like Amazon, Google, and Microsoft who have long worked with the government are now pushing AI products for defense and intelligence despite growing outcry from critics and employee activist groups. So I wanted to have Heidi on the show to walk me through this major shift in the AI industry, what's motivating it, and why she thinks some of the leading AI companies are being far too cavalier about deploying generative AI in high-risk.”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT
“Hey there, and welcome to Dakota. I'm Hayden Field, Senior AI reporter at the Verge, and your Thursday episode guest host. I have another couple of shows for you while Neeli is out on parental leave, and we're going to be spending more time diving into some of the unforeseen consequences of the generative AI boom. Today I'm talking with Heidi Klaff, who is chief AI scientist at the AI Now Institute and one of the industry's leading experts in the safety of AI within autonomous weapons systems. Heidi has actually worked with OpenAI in the past. From late 2020 to mid-2021, she was a senior systems safety engineer for the company during a critical time when it was developing safety and risk assessment frameworks for the company's codex coding tool. But now the same companies that have in the past seem to champion safety and ethics in their mission statements are now actively selling and developing new technology for military applications. In 2024,”
2025-09-25 · Decoder with Nilay Patel · How AI safety took a backseat to military money · IDENTIFIED FROM THE TRANSCRIPT