YouSaid · the spoken record

Meredith Whittaker and Kate Crawford

lines on the record: 71
first: 2019-04-08
most recent: 2019-04-08
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“People of color, there's all kinds of. And again, one of the issues is that we aren't seeing enough data on this and enough sort of emphasis on the urgency of this problem. But one anecdotal evidence is Timique Gebru, who is a preeminent machine vision researcher. She's a woman of color. And when she first went to NURIPS, which is the biggest machine learning conference, she said she was one of six black people out of 8,000. So she was a co-founder of Black and AI. She's been doing a huge amount of work sort of spearheading this with a couple of colleagues to make a lot more space for black people to participate in machine learning. And there have been sort of initiatives that have been grown out of that. But that is emblematic of a huge problem because...”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Well, we don't have much data, but the data we do have is unequivocal, and our daily experiences, women in tech is confirms this data. And then some, there was a wired study that came out last year, and it said that around 12% of the papers that were submitted to the big machine learning AI conferences were submitted by women, right? So you're looking at a field that is even less diverse than the very undiverse computer science field. Right now we have about 15 women getting computer science degrees. This is down from 10 years ago. It's down from 30 years ago when you had rough parody, right? So you've actually seen the field as it has grown in power and prominence.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“About the who's in the room part, Meredith, because that's one of the issues that has been brought up again and again around what happens with AI. This is the probably you'd argue AI is the biggest growth area for tech going forward, one of them besides self-driving would be one. There's a whole bunch of things, automation, robotics, but AI is the really big next direction of the future. Yeah, so talk about that.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“That's actually a really complex set of issues, partly it's that feedback effect that people are kind of searching for a generic image and they might choose a male doctor, for example. But sometimes it's because of where those images are coming from. So if you're scraping it from, again, from very particular types of photo sets like Getty, for example, really push for more diverse images of people in these kind of classic photo sets because it'll become really cliched in terms of what you could get. So long story short, search is really complicated and people are trying to fix it, but it's much harder than you might imagine. And it keeps being more and more layers of the onion that really have to be looked at. So instead, I think we have to ask different questions around, okay, how do we think about data construction practices? How do we think about how we represent the world and the politics of AI? Because these systems are political. They're not neutral. They're not objective. They are actually made by people in rooms. And that's why it matters who's in the room, who's making the system, and what types of problems they're trying to solve.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Go ahead and move on. Yeah, but what is it? We have like 9% female CEOs. You didn't even have that in search data, right? Right. And the first female CEO that came up in these searches that we were running at the time was Barbie CEO. And you're like, okay, that's a problem. And it's funny because there's kind of like a whack-a-mole problem right now, right? So industry is like, oh, okay, we see a problem that we're actually like, if you look up physicist right now, you know, you'll still see some differences. So again, around professions, around these kind of cliche stereotypes around gender and race, you keep seeing them get reflected. That's because of what you...”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Right, yeah, and then China. We'll get to that. And then China. It's a whole other story. We'll get to China. You got some issues over there. Exactly. So, yeah, and I loved your example of search, because search data is a really good thing to look at. Some of you might have seen one of the classic search tricks that you could do. This was a couple of years ago. Yeah, we ran the CEO example in the lab and it was like, wow, it's all white dudes. And in fact, the first female CEO came up. Did you have a look at that? No, but it actually.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“The right term, you have a data cleansing. Well, see, that's currently one of the things industry is really contending with right now, which is how to create what are called fairness fixes. How do we clean up the data? How do we make neutral and fair AI? Well, the more we've been doing this research, the more concerns we have about this sort of idea of a simplistic tech fix. Because in the end, you're talking about cultures of data production. And if that data is historical, then you are importing the historical biases of the past into the tools of the future. So essentially training data from the past is deciding how to decisions will be made by AI systems. That's going to be a real problem. First of all, if you can't see into the training data, if it's a proprietary company that doesn't have any kind of transparency protocols, you can't see the data. Second black.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“For example, well, it is not infallible. It only reflects what's in the data, which is why this question of is the data coming through biased policing practices that have a record of arrest that is actually a record of corruption is really important because once that is filtered through one of these systems, people take it as the product of a smart computer, that it's infallible, that it is sort of mathematical wizardry and probably not to be contested. Whereas if they looked at the practices that were creating that data, you would realize there was something really wrong with those and you actually need to change it.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, yeah. It's kind of like people, but people have a bit more nuance and complexity. So taking the most basic and kind of a canonical example, right? You show a machine learning system 100 million pictures of cats, but you've only shown this machine learning system, cats that were colored white, right? That system would then recognize cats, but probably misrecognize darker colored cats, right? You can show the machine learning system any kind of large corpus of data. It models the world through that data. That's all it will ever know. That's all it can ever see.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“The data that's getting, explain training data again to be the key. Training data is you teach the systems and then they learn, right? But what you teach them with at the beginning is how they learn at the end. It's like people, I guess.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“So, if we have dirty data, we're actually forming our predictive policing systems, you're ingraining the sorts of bias and discrimination that we've seen over decades into these systems that in many ways just are above repute, because people say, oh, well, you know, it's neutral, so it must be completely fine. And so you see these kind of vicious circles emerging because essentially the training data itself.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Predict for that. So, for those of you who have sort of seen how there are these kinds of heat maps that are used to sort of basically isolate areas and cities where basically police couldn't predict that crime might occur, or in some cases it's a person-based list to say this person looks like they're the sort of person who might commit a crime looking at their social network. We can ask really hard questions about”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I mean, this is actually one of the big areas for our research at AI now is really lifting up the hood on AI systems and looking at the sometimes quite weird and sticky and gooey training data that goes into the pipes. And some of the ways you do that is really by looking at where does that training data get sourced from? So I'll give you an example. One of the studies we recently published looked specifically at predictive policing data. And we'll explain this.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Right, so let's talk about that issue data because you just said something in a room in Silicon Valley by a group, a certain group of people, which is typically the same group of people that are putting them in. And I speak of pretty much, you know, the data is there. It's mostly white men, younger, correct? Is that correct? Yeah. To this day. So here you have this issue where the data is going in and whether the data is correct. Let's talk about the issue of data and data as gold in Silicon Valley now. Talk about the systems and how they're created and how you can get faulty, how it moves that way.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“So Kate and I met, and I was so relieved. We met on a bus on the way to a conference, and suddenly there was someone who was speaking this language and sort of, you know, helping me think through ideas that I had felt fairly alone and thinking about. And we started talking about this. And we shared a similar set of concerns, right? If these technologies are being threaded through some of our most sensitive social institutions, what are the guardrails, right? What are the guardrails when we begin to automate criminal justice based on the assumptions of people in a conference room in Silicon Valley? What are the guardrails when we begin to automate education, when we begin to do sort of automatic essay scoring and eye tracking for students to determine attentiveness or intelligence, right? How do you make sure these aren't replicating patterns of discrimination?”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“And I had the dumb luck of being in the place where I was watching the ascent of AI. I was watching people take data that I knew was faulty or fallible or incomplete and begin to pump it into AI systems and make claims about the world that I didn't believe were actually credible or verified.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“And say, well, I mean, my path was through industry, right? I had been at Google for over a decade. I ran a research group there. And I think Kate and I came to very similar conclusions that were fairly heterodox during my days in industry through very different paths. So Kate has been an academic. She's one of the founders of the field. She's been setting this stuff for over a decade. I worked on large-scale measurement systems. So I was really at the, you know, how do you deploy servers across the globe and create the kind of data that would be meaningful, right? How do you make data that has a certain type of meaning? And then how do you ensure that meaning? So I was right, as Kate and I joke, this sort of epistemic guts of these questions, like, you know, what is the ground truth? And it was constantly slipping.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, happy to. I mean, essentially, we started really about four years ago by realizing by looking around internationally and realizing there wasn't a single AI institute that was focused on the social, political, and ethical implications of these tools. And so Meredith and I realized that we had to make our lives a lot harder and actually do it ourselves. And so now we had the AI Now Institute at MYU. And it's really the world's first institute to really center these concerns. And we created it essentially as an interdisciplinary institute that we can't be resolving these issues just from computer science and from engineering departments. We actually need a much bigger lens. We need to be drawing on social science, on humanistic disciplines, on ethical, philosophy, as well as anthropology, sociology, criminal justice. If you actually want to build tools that affect social institutions, you need to have experts in the room, but you also need affected communities, people who are likely to see...”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“So, we ended up looking at 13 jurisdictions across the US that were specifically under legal orders because of biased or illegal or unconstitutional policing. The data that was being created by things like planting evidence or racially biased policing was being piped into predictive policing systems. We found multiple cases, Chicago being one of the most obvious.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“There are license plate profiling AI that is sort of tracking people as they go over different bridges in New York. You have systems that are determining which school your child gets enrolled in. You have automated essay scoring systems that are determining whether it's written well enough, like whose version of written English is that”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT
“Hi, I'm Kara Swisher, editor at large of Recode. You may know me as someone who's both intelligent and artificial, but in my spare time I talk tech and you're listening to Recode Decode from the Vox Media Podcast Network. Today in the red chairs are Kate Crawford and Meredith Whitaker, the co-founders of the AI Now Institute is a research institute at New York University that studies the social implications of artificial intelligence. Even if you don't realize it, Whittaker said, AI is already having effect on our day-to-day lives.”
2019-04-08 · Decoder with Nilay Patel · Recode Decode: Meredith Whittaker and Kate Crawford · IDENTIFIED FROM THE TRANSCRIPT