YouSaid · the spoken record

Meredith Whittaker and Kate Crawford

lines on the record: 71
first: 2019-04-08
most recent: 2019-04-08
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“No, this is not a government initiative like the internet was. There are very different incentive structures baked into the DNA of how those who are in the position to produce this technology are thinking about what it does and who it benefits. So, you know, this is a”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“I would agree with that. I mean, I think real talk for a second, you began that question beautifully by asking, what do we want this innovation to do? And right now this innovation is produced by a handful of private companies. They are the only companies that have the resources to build this kind of AI. There isn't a way to kind of bootstrap this from a startup in a garage. That's just not how this technology works. Whatever else their calibration is, they're looking for shareholder value, right? So I think there is a bigger question.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“I mean, and that's part of what's happened in Europe is exactly that conversation with GDPR. I mean, there are actually a lot of initiatives that are trying to do just that. It's to say what kind of data, in what context, held for how long. These are really good questions that we need to pursue a lot further. But what's interesting is that that's happening at a much slower rate than these technologies are being released into the world. And essentially, being live tested on populations all the time. So what we have is this kind of race now between how do you actually have those conversations with sufficient knowledge about how these tools really work when a lot of these things are protected by trade secrecy. A lot of these tools, you're not going to know how it's working or what data is being collected. So there's a real knowledge problem here as well in terms of that sort of speed problem of how do we catch up to what these technologies are doing. So I would say to you, yeah, we definitely need to do that to have those conversations.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“And everything I see of that. I mean, that's the part that I think socially we're still catching up to what that's going to mean, what that means for the private sector, what it means for the public sector, what it means for government. These are really big, hard, difficult questions.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Planetary. The only thing that's very different is you have never been tracked so beautifully ever in human history. Everyone here has a phone. You're all, it's not just biased data. There's so much data pouring out of this room right now, for example. It's insane what's happening. And so that's the difficulty is the amount of data that's being collected about you right now.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Do right now, if you look at what the international governance conversation looks like, it's much bleaker, right? And speaking from the US, it's real bleak right here right now. So how do we think about what is an international governance conversation look like? I noticed that Mark Zuckerberg mentioned this in his most recent op-ed telling us how we should regulate the internet. He was like, you know, we need global governance. I'm like, oh, that's great. But how are we going to get there right now? Because historically, I think we're in a very different moment. So I think you're right. We can look to these kind of key moments of technologies that really change the way we lived, but we also have to look at what were the governance structures and how do we get there. And that's one of the big questions hovering over AI right now is you can come up with local regulation, but you're really talking about technologies that are planetary in scale. These are planetary.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Oh, I love this question. Yeah, we do. In fact, one of the things that we have is we really focus on deep historical research because I think we can learn a lot exactly from moments in history where these big sort of general purpose technologies were sort of flooding into society and decisions had to be made about how to use them. I mean, one of the examples that I think people really go to a lot is nuclear, right? So we had this extraordinary potential for generating energy, but also terrifying horrors if it's actually used as a weapon. So what we saw was a really profound international conversation about how are we going to regulate this technology, this capacity. And we had the creation of things like the IAEA, the International Inspections Body that could say, hey, we should be able to inspect how you're creating this, what you're working with. Do you have like weapons facilities? Do you have energy facilities? This was a big international effort. And that's a very difficult thing.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“About them, they are, you know, our experience matters just as much as a technical design doc or a wired article about the superintelligence. And I think part of the job to steer us toward a better future with these technologies is to begin to recenter the conversation around what the lived experience of having these technologies shape and direct our resources and opportunities towards stories.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Algorithmic system that was brought in to distribute Medicaid benefits A caseworker shows up with his new system, sort of enters her info, it dropped her hours from something like 12 hours a day to eight. And I don't have the exact numbers, but it was enough that it seriously affected her quality of life. This was the difference between having a dignified life, living at home, getting the care she needed to survive, and not getting that. Now, thankfully, there was a lawyer who took that to court contested the algorithm, found that it was actually, there was a major implementation flaw, there were all sorts of other problems, but neither the caseworker on the scene nor Tammy had the ability to contest that decision or override it. So, you know, I would answer these are profound and material harms that are happening. These are things that actually affect us right now, and I think we all have a stake in talking about it.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Or that's going to kill us, these sort of speculative and hype-filled proclamations that focus on tech wizardry, right? So as a superintelligence or the next deep neural net, which is better than humans, which is a claim that Kate has examined and we're looking at. But it is actually affecting and shaping all of our lives in different ways, right? The harms are not evenly distributed, but this is in our lives, right? There are license plate profiling AI that is sort of tracking people as they go over different bridges in New York. You have systems that are determining which school your child gets enrolled in. You have automated essay scoring systems that are determining whether it's written well enough, like whose version of written English is that and what is it rewarding or not? What kind of creativity can get through that? You have systems that are being used. We have an example that is fairly chilling in Arkansas.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Have been more fun. I will answer this. I think part of the way you begin to get more people in the room is to focus not on this sort of technical wizardry on the shiny covert of some wired article or that it's”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“I mean, you can ask the question if we begin to scratch the surface of the political economy of the AI industry, you see a number of AI startups. They're all over the place. But ask any one of them, where do they host their infrastructure, right? Who runs their servers? It is Amazon, it is Microsoft, or it is Google, right? You can scratch below the surface a little more and say, like, actually, what kind of AI are you building? Oftentimes these companies are, in fact, just sort of repackaging models as a service that are sold by the big tech players. So again, where is this actually accruing to? Who is actually creating AI who has the capabilities to create AI? Is a question I think we need to answer as we answer the questions around what would responsible AI look like and what should these card whales look like?”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“I think our innovation issues have to do with other things besides this because it has to do with government research money, it has to do with all. But we are, I think, at a low of startup creation right now. And it has to do with large giant companies dominating. Like Google buys up every worthy AI company, if not Facebook and Amazon does. And so the whole culture doesn't come up. Someone's not going to displace them, essentially.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Innovate on ethics, innovate on accountability, innovate on clear guardrails. Otherwise, I mean, I think it is really interesting how innovation has become basically tethered to sort of rising share prices for a couple of Silicon Valley companies, right? Is that what we mean by innovation, or can we think about innovation and begin to redefine the term in ways that actually match our values more broadly?”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Although, I mean, he won on that one, but you know what I mean? But it was like a kind of an interest, I hear it from Silicon Valley all the time. But this is another false dichotomy too. I love this question because so often we hear it's like innovation or rules of the road. We either have some type of guardrails or we have thriving AI. It's like actually guys, no. You will have thriving AI when we have guardrails, when we have safety, when we have protections. People will be much more likely to want to trust these tools when we know that it's not going to discriminate against us or harm us or cause other forms of ongoing structural problems. So I think it's this tendency to see innovation as God and everything else is like restraining the power. And it's like, no, we actually will only have AI that is worthy of the name when it is really designed in harmony with the ways in which we want to live.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“I was interested in your comment about GDPR in contrast to where the US is in terms of privacy or tech regulation kind of generally and how it's important who gets to make the decisions are democratically elected government and representatives here or someone somewhere else with different kind of public policy kind of considerations So one of the challenges that I think U.S. regulators often have to grapple with is vis-a-vis a lot of these industries, we've often prided ourselves on the idea of permissionless innovation. So the speculative harm about what the heavy hand of government can come in and do to these new emerging fields.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“We've called this the apex predator problem, which if you're already an apex predator and you have all the money and all of the power in the world, what's the next thing to worry about? Oh, I know super intelligent machines. That's the next threat to me. But if you're not an apex predator, like if you're one of us, we've got real problems with the systems that are already deployed. So maybe let's focus on that, including Apex predators. Okay, all right, questions. There are actually some very easy to nag, just so you know. Questions from the audience?”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“As you know, when I interviewed Elon Musk a couple of years ago, who has talked about these issues of dangers of AI, he said he thinks we were talking about the terminator ideas that go in movies and everything. And he said, no, eventually they're going to treat us like house cats. We're just house cats. We'll be housecats to these systems and they don't want to kill us necessarily. They just don't care. He's wrong.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“I think that's not giving dolphins enough credit. I think actually dolphins are pretty smart. I think we're actually a way earlier stage than you might imagine. I mean, I think people are like, oh, we should be worried about the superintelligence. I'm like, guys, no, it is so far from that. We're talking about a basic 101 stuff. I mean, you know, to the degree to which, yeah, AI systems can tell the difference between a cat and a dog. But there are a lot of things that it cannot do, and particularly the way that humans are classified by AI systems would curl your toes. I mean, some of this stuff is like really terrifyingly basic and often wrong. So I actually think dolphins are kind of a step ahead at this point. And we're probably at the protozoa level at this point.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“I guess I'd put it a little differently. I'd say this field has worshipped at the altar of the technical for the better part of 60 years at the expense of understanding the social and the ethical. And we are seeing the fruits of that prioritization. And it's interesting because if you go back in the history of AI to the beginning and the sort of 1950s and 60s, it was a much more diverse field. You had anthropologists sitting at the table with computer scientists. It was this vision of how do we construct a world that we want to live in. Fair world. And we missed a boat for a couple of decades there by making that much narrower a conversation. And right now, as we have these real issues of homogeneity in Silicon Valley, we need to open those doors up, but we also need to get people in the room who are the ones who are most likely to be seeing the downsides of this system. We have to center affected communities and not just engineers on big south.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“And then last question, and then we'll get some questions from the audience. I've come to the conclusion recently that we know most of these people, and I don't find them to be particularly evil, like in terms of we don't have like a chemical manufacturer rubbing his hands together, going, ha ha, I've won, that kind of thing. It's more like, oh, no, I've come to the conclusion that perhaps they're incompetent. You know, the leaders are actually incompetent to the task, not stupid, but incompetent. They did not understand what they have created and now don't know what the hell to do.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“And it's interesting too. I mean, this is one of the things that we think is super important is how do you protect people inside companies who are going to be the whistleblowers, who are going to tell us things that we need to know, and who are actually going to do this sort of organizing work. And one of the things that's super important is to start saying, hey, this is going to be important for journalism. This is going to be important for research. It's going to be important for history that we understand how these systems work. So really being able to create structures where workers can unionize, where they can disclose, where they can actually hold to account the companies that they work for. I think this is going to be increasingly important and it's something that we've done quite a lot of research on.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Probably both. I don't researcher. We've founded a research institute, and a lot of what I do is look at the patterns of behavior across these companies. Are we seeing structural change that would actually result in significant improvements or clear answers to some of these problems? I think we have seen some of that.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“So, I think there are a lot of people in these companies. They don't want to be complicit. They're close enough to the tech to know where it fails and what it is good for and what it's not good for. And they are doing a lot of work to try to steer this ship in another direction. So that's actually giving me a lot of hope for Silicon Valley writ large is that there are these forces who are, you know, a lot of these people are comfortable, right? They could have easy lives, but you're seeing tens of thousands of people instead turn to face those with power over them and say, this is not okay. We actually need to think more clearly about these decisions. We need to think more clearly about the cultures we're creating. We need to think more clearly about the implications of our technology on geopolitics, on our social well-being. And I think that is something that gives me hope that there's actually the possibility of change.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, there's Well, I'm going to do a thing where There are a lot of people who don't get it across the board full stop. There are a lot of people who benefit from not getting it. That's a problem. There are a huge number of people who are getting it, right? You are seeing workers across tech take personal risks to protest the decisions of their employer, right? We're seeing that as one of the few checks we've actually had on these systems.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Sometimes it's in your house. I mean, did you see the story this week where basically there's a rent controlled building in Brooklyn where they're just installing facial recognition cameras? Like none of the residents are getting a say in this, and they're all starting to protest and say, we don't want to have facial recognition in our homes. This makes us feel like animals, like we're being tagged. And of course, it's in more low-income communities that are being gentrified. I mean, it's this story around we have to look at these sort of deep social and economic context to understand why and how these tools are being used.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Well, Kate said it beautifully, but I want to highlight another distinction between China and the US along those lines, China, you have a party and it more or less centralized state, although it's very factionalized as well, that openly acknowledges these are the uses we're putting AI to, this is what it's going to do. The social credit score is in law. They've written down what it is. It is pretty transparent about the application and the purpose. There isn't much subterfuge. In the US, there is currently no law governing the application of facial recognition. There is no way for us to know if we walk in a store that we are”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Well, look, you know, it's interesting. If I look at something like how we can reduce power costs, right, if we look specifically at the environment and climate change, there's been some really interesting work that's been done using AI systems to say, hey, we can actually modulate the use of the electricity grid to make sure we're much more efficient. We can look at how, I mean, think about how much energy is wasted in cities and in giant server farms, which we've done lots of research in as well, looking at data centers. I mean, we can do stuff there. That excites me. We've actually got big challenges, particularly on the environment side, where we can do real work. But the minute this stuff touches complex social systems, you are looking at way messier terrain. And that's when you need to think much more in a much more nuanced way about how you might be affecting people's lives.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Exactly. So bias sometimes is one problem, but it's by no means the only one. Sometimes the real question is just, should we be using AI in this context at all? Even if it works, would that be okay? That's the question we have to start asking. It's not just, let's fix it so that it's working great, then everything is fine. The question is, is it actually an appropriate technology in this context?”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Sometimes, yeah. It's like who decided Black Mirror was like a design spec? I'm like, no, guys, that's not what they're trying to say. Yeah. So, I mean, look, here's the thing about the social credit score. It's really creepy. And what really disturbs so many of us is that it could really change people's opportunities in life. We've already seen many, many people blocked from domestic travel. We've got real concerns about what's happening to the Uyghur population in China right now. Like, this is really scary from a human rights perspective. But here's what isn't told as much. The US has many similar systems that are either in place or about to be in place in the next couple of years. I'm sure you read the news that, for example, in New York, insurers can now have been given full permission to look at your social media to decide how to modulate your insurance rates. So that sounds very similar to the sorts of things that we're concerned about in China. So I think sometimes as this tendency to think, oh, China's the bad guy and that would never happen here. It's like, actually, we have to do a lot of work.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“I mean, social credit score is kind of a really interesting system. It isn't fully implemented until 2020. So a lot of it is a sort of speculative debate at the moment. But what we've seen already is that these scores are being used basically to track everything you do online. So if you spend a lot of time doing online gaming, if you pay your bills on time, then your score will go up. If you don't pay your bills on time, if you say something negative about the government on a forum, then your score goes down. And if your score is low, it impacts your ability to do everything from buying a train ticket to getting your kids into the school that you want to go to to getting the job that you want. So it's profoundly connected to all of these other sorts of things that you'd want to do in everyday life. So that's all. And it moves to the move.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Yes. Facial recognition, social scores, heavy into AI. Obviously, Kaifu Li wrote a book that talks about this and how fast they're moving past us in this area because of the interest in data, that their ability to collect data is unfettered. And their citizens allow it, whether they're going out of stores, what they're buying in stores, not just looking at what you're doing in face, but what you pick up and put down in stores. The surveillance is everywhere. It's a surveillance economy as far as I can tell. But it benefits from that because they get all kinds of insights because you do. And that's not biased insights. That's all data. And it does reflect the world of how people are moving, whether it's transportation, whether it's anything. They can have data on everything. Is that something we have got to think about? Because here's a country that's just going to be swimming in data.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Just a big plus one to what Kate is saying and the work here. I would emphasize that. And I would say it's also, I think, important to delegate responsibility to experts who are coming from outside the AI domain. Because at this point, a lot of these questions actually aren't AI questions. They aren't about, are you using a deep neural net to do this? It's about under what policy is this implemented? In what context is it implemented? Was it trained on data that reflects that context? Is it going to be used in ways that are transparent, that are contestable, that are safe? How is safety proven? These are all actually expertise that people from say in the healthcare domain, you would want doctors. You would want nurses unions. You would want people who understand the arcane workings of the US insurance system. You would need them all at the table on equal footing with AI experts to actually, you know,”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“To monitor data. This is the big debate. There's already been a debate about this for many years now, which is, do you try to give more strength to existing agencies or do you create a new super agency for AI, right? And this is something we looked at in detail in our sort of research last year. And we made some recommendations specifically about at this point because we need some regulations, I'd say, quite urgently, we need to empower existing agencies to do what they're doing, but to also include looking at AI, right? So, I mean, if you're the FAA and you're focused on, hey, how do we think about safety in planes? You're the right agency with the right expertise to be thinking about how AI starts to impact your particular domain. Same thing goes for the FTC. Same thing goes for many agencies where we want to say, hey, give them the power to look at these issues. Maybe one day we'll get like a super agency, but we can't wait that long.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“But you did make it in your college dorm room, so that was great. Yeah, but I'll say that there are some really interesting senators right now who are asking different questions and they're looking at algorithmic accountability. That's really key to see. They're having different conversations about privacy that realize that it's not just about individual privacy, it's about our collective privacy. It's the fact that if you make a decision in a social media network, that can affect how data from all of your contacts is being extracted as well. I think there's an increasing level of literacy and that's something that's super important. So we need to really support that with more.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Absolutely. Who? I mean, realistically, we see this as being something that needs to be an evidence led process. So what we want to see is more because that's actually popular these days. Yeah, you know, it's like, can you actually show us how this technology works? Do you understand what that is? I mean, we saw some things, I mean, you remember sort of Mark Zuckerberg in front of Congress. I mean, that wasn't one of those shining moments of seeing how regulators really understand AI. We can do better.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“In 2018. And it's interesting, it's not a perfect piece of legislation, but it has had impact internationally. So what is interesting now is that the US is facing a decision. It's like, are you going to be regulated by other countries saying, hey, we're not going to accept this, or are you going to give protections to US citizens? I think this is the key moment to start making that regulation, to make it real. The question is, who gets a seat at the table to decide what that's going to look like? And this is going to be one of the most important things that happens in the next five years is how AI is going to be regulated and all of those adjacent technologies.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“And it was certainly watered down from a sort of original framing, and we're starting to see that happen again around issues like facial recognition. You're seeing multiple states move towards actually saying, no, we need to regulate facial recognition for very good reasons, because this technology can be deeply troubling in the way that it's being used. But again, there's a lot of fights going on about how strong those rules should be. I mean, it's interesting, we made a recommendation, again, from the AINOW Institute based on this research, saying that, look, we think notice and consent isn't enough. We think things like facial recognition actually need to be something that we all debate and take seriously. And communities should be allowed to say, hey, no, we don't want this in our backyard rather than just being told, hey, you've walked into public space, so basically you've already consented. I mean, that presents a set of concerns. But these are the debates that are happening right now at a state level. But then internationally, you see a lot more movement. So, of course, we had GDPR, the General Data Protection Regulations, come into effect in Europe.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, we still don't have any kind of federal privacy law, which is kind of extraordinary in this day and age. Well, it's interesting. I mean, I think some of the most exciting steps have been happening at the state level. We saw California pass the strongest privacy bill of the country. It's actually kind of amazing. This goes in for.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“And are they qualified, right? Who chooses who's on the board? There's a kind of recursive question here about who's guarding whom. And I think ultimately it's a great step that we're seeing these issues be taken seriously. I will say four years ago when we started doing this, it was a lonely room. There weren't that many people who were concerned. There were a lot of people who'd argue that these were not problems. Now that is not the case. These issues are serious and they're being taken seriously. But what we don't see is real accountability. What we don't see are mechanisms of oversight that actually bring the people who are most at risk of harm into the room to help shape these decisions.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Historically marginalized populations, or are you going to sort of get your act together and make some significant structural changes to ensure that what you create is safe and not harmful? And I think in the wake of these controversies, there is been kind of a ethics theater almost. And we actually look at this in our 2018 report where we looked into these a little bit. All of these questions around what do these boards actually do, right? Are product decisions run by them? Can they cancel a product decision? Do they have veto power otherwise? Is there any documentation on whether their advice was taken or whether it was not?”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“The creator of police tech, AI enhanced body cameras and police surveillance drones, has an ethics board. Salesforce constituted something along the lines of an ethics board, right in the wake of a kind of crisis where a lot of their workers and a lot of other people were asking them not to sell tech to ICE, right? Facebook is sort of creating this ethics panel in the wake of massive global controversy as a very diplomatic word for what's going on at Facebook. But in a sense, what you're seeing is that these panels are, you know, we use the term ethics washing, right? Where there are serious and significant questions that are at the doorstep of this industry right now, right? Are you going to harm humanity and specifically, you know,”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“There's also efforts on the behalf of industry to test themselves, right? Now, Google has started its own advisory panel, right? Facebook has its content moderation panel that they're hoping will save them. It won't. Newsflash Newsflash Newsflash, that'll be a column in the New York Times in 23 minutes. Talk about that. You're not on the Google Advisory Panel and you're at Google and you're an expert in this. Is that correct?”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“So, this is one of those things where you're like, no one even gets to look at that system because they're like, oh no, it's proprietary. Sorry guys, it's actually a perfectly good system. It's neutral. It's objective. We have to be much more critical of these systems. So, I mean, that's really what Meredith and I do and what we stand for is saying we will do the research to actually test these systems, which is why it's so important that we can order and see.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“And that tells us One, it's actually much harder to automate these tools than you might imagine because. Amazon's got some pretty great engine. Not like they don't know what they're doing. And it tells”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Have probably tried sending CVs and resumes in to get You may not know is that in many cases companies are using AI systems to. Resumes to decide where. Of an interview. And that's fine until you start hearing about System An AI Automatic resume sc”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“So, I think we need this is when you're talking about the architecture of. I had a really great podcast with Nicole Wong, who used to be chief legal counsel at Twitter and Google, and she was involved with building this, helping build this architecture. And one of the things she talked about was the architecture of, say, a Google search or something on Facebook. And one of the things that was interesting is you can build on, initially, you can build on context accuracy and speed and you get pretty good results when you do that, but when you start to engage, when the pillars you build are engagement virality and speed, we end up with Alex Jones. Like that's where we go. That's right where we go. I mean, it does because that's what, as you can see, there's lots of, there's a very good article today in Bloomberg about that. They build it for that to create it, and therefore that's what happens. And then they're surprised that it happens. I do want to get to the benefits because there are benefits.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“I would say again to answer that question, and this is not to be cagey, is how are you measuring benefit? And that's one of the key areas I think we need to look at more closely, right? So in increasing crop yield, that might be a huge benefit. But is that coming at the expense of soil health? Is that coming at the expense of broader ecological concerns? Is that displacing communities that used to live on that land? I'm sort of making up these examples as questions you'd want to ask before you sort of claim blanket benefits from these technologies. Similarly, like if you're looking at harms, who's measuring harms to whom? A lot of the issues come from setting one objective function. So this is sort of one goal for the AI system. And one we've seen a lot lately that is fairly problematic is kind of engagement in social media, right? That is the only goal that is sort of sought after in a lot of these boardrooms by people who are creating these systems, right? How do we get”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Let's talk about those benefits and harms. What does it do in the new society with AI making decisions? And in some cases, it does notice of inefficiency. On things like crops or weather, it's hard to have bias in those kind of things. Have benefits, correct? You all have an AI now institute, so you must like it AI. Talk about the benefits. Where does it work really well and where doesn't it?”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“I mean, I think there are a lot of ways we could diagnose that, but I think the cost of diversity is pretty clear, right? The people who bear the costs of discrimination, of exclusion, of racism within these companies are the same people who bear the cost of bias, of errors, and of sort of, I would say, oppressive uses of AI outside of these companies. So there is, you know, there is making a direct causal link is something that we're going to need more research to begin to put together. But there is, it is very clear that the people who are benefiting from these systems match a specific demographic profile and the people who are being harmed by these systems are those who have historically been marginalized.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT