YouSaid · the spoken record

Jen

lines on the record: 67
first: 2021-08-10
most recent: 2021-08-10
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“and get pushback in that might not have been possible had civil society been looped in all along the way and you know kind of incorporated and neutralized almost in that sense so you know I'm not sanguine about the odds of them just not deploying this CSAM thing at all and you know like don't get me wrong I would love to be wrong with the slippery slope arguments that the next thing will be demanding this for GIFCT and then it'll be Les Majesty and deep fakes and copyright infringement. I would love to be proved wrong about that even as silly as it would make me look, but I'm not sure that that's going to be the case.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“I do agree with Jen that I think certainly for the stuff about Siri that is uncontroversial they'll keep rolling that out I'm not certain but it seems like the iMessage stuff either it wasn't messaged clearly at the beginning or maybe they really did change over the course of the last few days in terms of what they said they were going to do if that is true and I'm not sure whether it is then that indicates that maybe there is some room to at least make some tweaks however the fact that they rolled out this whole plan as a fait accompli that's going to be put into iOS 15 at the very end you know without any consultation suggest to me that they are definitely going to go forward with these plans with that said you know it might be that there may be some silver lining in the fact that civil society was not consulted at any point in this process that now maybe there's an opportunity to use this concerted blowback as a way to try and”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Think the image hashing match shifts. I don't know about the nanny cam, again, for lack of a better word. We need to call it something. I'm not sure if that one sticks, but I predict that they will double down on the. CISM image scanning for all the different reasons we've talked about today. I think Rihanna really hit the nail on the head. I mean, I think there's some very kind of political strategizing going on behind the scenes here. And that if they are trying to take a bigger stand on encryption overall, that this was the piece that they had to kind of give up to law enforcement in order to do so.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Forth with the thing that they're doing it as is. But as Jen, I can tell you Jen and I have both been consulted before by tech companies who have something that impacts upon privacy things and they'll preview that for us in a meeting and take our feedback. And that's standard practice for tech companies, at least at some point, if you don't really care what people's feedback is, then you roll it out later and later in the process where you get feedback from people. But if they had really wanted to minimize the free expression and privacy concerns, then they should have consulted with outsiders. Even if there are voices they thought that would be too screechy, as the executive director of NickMick called everybody who expressed any kind of reservation about this, even if they didn't want to talk to what Apple might think is somehow the lunatic fringe or whatever, they could have talked to more moderate voices. They could have talked to academics. They could have talked to me, although I'm probably too screechy for them, and at least taken those concerns back.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“I think part of it is. The fact that Apple ruled this out with maybe a one days heads up to some people in civil society orgs and maybe some media one day before their announcement isn't helpful. Like nobody was brought into this process while they were designing this to tell them here are the concerns that we have for queer 12-year-olds here are the concerns for privacy here are the civil liberties and the human rights concerns, all of that. It looks like this was just rolled out as a fait accompli on no notice with I have to say really confusing messaging given that there are these three different components and it was easy to conflate the first two of them and get mixed up about what was happening and that has further caused a lot of handwringing and wailing and gnashing of teeth but if they had involved elements of civil society other than presumably nicmic itself and probably law enforcement agencies maybe some of the worst could have been averted or maybe they would have ignored everything that we would have said and just gone”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“This brings me to just the way they've communicated about this, which we were talking about briefly before it started recording. You both mentioned big media debates happening in civil society organizations with policymakers, with academics, with researchers about how to handle these things, about the state of encryption, about the various trade-offs. It does not appear that Apple engaged those debates in a substantive way before rolling this out. Do you think if they had or if they had been more transparent members of that community? The reaction wouldn't have been quite so heated”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“And it's really kind of revealing the feet of clay that I think is kind of underpinning the whole thing. We thought our devices were ours and Apple had taken pains during Apple versus FBI to say your device is yours doesn't belong to us. Now it looks like, well, maybe the device really is still Apple's after all, or at least the software on it.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Well, Apple is saying one of the protections against non CSAM uses of this is that they have a human in the loop who reviews matches if there is a hit for a sufficiently large collection of CSAM. They will take a look and be like, yep, that matches the databases. If what they're looking at is the Thai King and a crop top, then they can say, what the heck? No, this isn't CSAM. And supposedly that's going to be another further layer of protection. I think that I have already started seeing some concerns, though, about like, well, what if there's a secret court order that tells Nick McDistick something in there? And then Nick McEmployees have to just, you know, go along with it somehow. That seems like something that could be happening now, given that photo DNA is based off of hashes that Nick Mick provides even now for scanning Dropbox and whatever. And so this is really highlighting how it's just trust all the way down. You have to trust the device. You have to trust the people who are providing it in the software to you. You have to trust Nickmick.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“So you're saying the danger there is there's a child safety organization in some corrupt country and the dictator of that country says there's eight photos of me sneezing and I just want them to not exist anymore add them to the database Apple will never know that it's being used in that way The photos will be detected and potentially reported to the authorities.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“A list of hashes that Nickmik provides or that other child safety groups in other countries provide if they don't have visibility into what is in those databases, then again, you know, it's just as much of a black box to them as is to anybody else. And this has been a problem with GIFCT that we don't know what's in it. And we don't know, does this contain important human rights documentation or news or commentary or whatever rather than just something that everybody can agree nobody should ever get to look at ever, not even consenting adults?”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“And I think there are some debates going on in different places right now about whether there are legitimate applications for using CSAM to train AI and ML models. Is that a permissible use? Is that re-victimizing the people who are depicted? Or would it have an upside in helping better detect other images? Because the more difficult side of this is detecting new imagery rather than detecting known imagery that's in a hash database. So even there, that's a really hot button issue there. But it gets back to Jen's point, which is if you start from the fuzzy cases and work backwards, Apple could sort of say we're not going to do this for anything other than CSAM because there's never going to be agreement on anything else other than this particular database. Apple also has said they are not compiling the hash databases, you know, the image databases themselves. They're taking what is handed to them.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Right, for example, it would be real easy for this is what law professors Daniel Citroen and Bobby Chesney call the liar's dividend, it's real easy for a government that does not like evidence of something that actually happened, something that is true and authentic but inconvenient for them to say that's fake news, that is a deep fake, this is going in our database of hashes, of deep fakes that we're going to make you implement in our country. So there's all of these different issues that get brought up on the free expression side. Once you're talking about anything other than child sex abuse material or even there it takes a special safe harbor under the federal law that applies to make it okay for providers to have this on their services as Jen was saying otherwise that is just a felony and you have to report it. If you don't report it you don't get the safe harbor and that provider is also a felon. National Center for Missing Exploited Children is the only entity in America that has allowed to have this stuff.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“I think that there are regimes in different countries for the protection of fundamental rights that look a little different from our constitution, but they exist. And so when there have been laws or surveillance regimes that would infringe upon those, there are other mechanisms where people have brought challenges and where some things have been struck down as being incompatible with people's fundamental rights as recognized in other countries. And, you know, it's just very difficult to engage in that line drawing, whether it's human rights abuse documentation that also counts as terrorism. If you look at it a particular way, or whether a lot of the research that I also do besides encryption, I have like a side hustle talking about deepfakes. And there is absolutely, I think, a lot of interest in trying to figure out, okay, how do we keep mis and disinformation from undermining democracy, from hurting vaccine rollout efforts, and also from having deepfakes influence in election?”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“I would argue that your example points to one of the easiest examples of that whole genre, and that it's much harder from kind of those extreme examples to work backwards to what is terrorism versus what are groups engaging in rightful protest on terrorism-related issues, for example. The line drawing becomes much, much harder. And I think also to kind of add to some context to what Rihanna is saying, I mean, we are very much talking about the US and the fact that this content is legal in the US. In Europe, those boundaries, I think, are much broader because they're not operating under the First Amendment. And I'm not a lawyer, so I'm definitely speaking a little bit outside my lane, but there isn't the same free speech absolutum in the EU, obviously, because they don't have the First Amendment as we have here in the US. And so the EU has been much more willing to try to draw lines.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“A database of image that exists that are just illegal, and you can't have them and you can't look at them, and there's no value towards even pointing at them and saying, look at this for things like scholarship or research or all the other kinds of exceptions we have. A database of images of terrorism video the Christchurch shooting, there are fuzzier boundaries there, right? There are legitimate reasons for some people to have that video or to have other terrorism related content to report on it to talk about it, to analyze it. And because of that, that is a fuzzier set. It's inherently more dangerous to implement these kinds of filters.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Things. And that's why there's already been a lot of criticism of the GIFCT database that I mentioned and why it would be supremely difficult to build out a database of images that are hate speech, whatever that means.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“I think that there are absolutely a lot of folks that you could talk to who would quietly admit that they might think that if this really did get limited only ever to CSAM for real, that might be a compromise that they could live with, even though we're talking about moving surveillance down into your device and really there's no limitation on them from only doing this for iCloud photos. It could be on your camera roll next. I think that if we really believed that this would not move beyond CSAN, there are a lot of folks who might be happy with that trade-off. Going back to your question about what a backstop might be, though, to keep it from going up beyond CSAM, this goes back to what I mentioned earlier about how CSAM is really unique among types of abuse. And once you're talking about literally any other type of content, you're necessarily going to have an impact on free expression values, on news, commentary, documentation of human rights abuses, all of these.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“I think that they will fold. India is one place that a lot of people have pointed to. India has a billion people. They actually are not that big of a market for iPhones at least commensurate with the size of the market that currently exists in China. But the EU is. The European Union is a massive market for Apple and the EU just barely got talked off the ledge from having an upload filter mandate for copyright infringing material pretty recently. And there are rumblings that they're going to introduce a similar plan for CSAM at the end of this year. And so for a large enough market, basically, it's hard to see how Apple thinking of their shareholders, not just of their user's privacy or of the good of the world, continues taking that stand and says, no, we're not going to do this for whatever it is they're confronted with. Maybe if it's Les Magistlaws in Thailand that say you are banned from letting people share pictures.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“And so, my prediction is that the next thing that Apple will be pressured to do will be to deploy the same thing for GifCT as they are currently doing for the Nickmik database of hashes of CSAM. And from there on, I mean, you can put anything you like into a hash image database. And my thinking is that Apple has just said if we are asked to do this for anything but CSAM, we simply will not. That's fine, but why should I believe you? Previously, their slogan was what happens on your iPhone stays on your iPhone, and now that's not true, right? And my thinking has been they might abide by that where they think that the reputational trade-off is not worth the upside, but if the distinction, the choice is between either you implement this hash database of images that this particular government doesn't like, or you lose access to our market, and you will never get to sell a mac or an iPhone in this country again. For a large enough market like China,”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“So, this is my primary concern. And the direction that I think this is going is we don't have ready-to-go hash databases for hashes of images of other types of abuse of content besides CSAM, with the exception of terrorist and violent extremist content. There is a database called GIFCT, GIFCT, that is an industry collaboration collaboratively contribute imagery to a database of terror and violent extremist content. largely arising out of the Christchurch shooting a few years back, which really woke up a new wave around the world of concern about providers hosting terrorist and violent extremist material on their services.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“But certainly the Chinese government or the Indian government or other more oppressive governments would certainly take advantage of. Is there a backstop here for Apple to not expand the capability beyond CSAM?”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Seen a lot of pushback against that idea. Just to be honest, personally, if the outcome is the same, right, they're scanning done of stuff you put on the cloud, I think that is the consumer expectation once you upload something to somebody else's server, they can look at it. They can, I don't know, copyright strike it, they can scan it for CSAM. Stuff is going to happen once you give your data away to a cloud provider. That does feel like a consumer expectation in 2021, whether that is good or bad. I just think it's the expectation. It seems like this is a very complicated mechanism to accomplish the same goal of just scanning in the cloud. But because it is this very complicated mechanism that is given inch so they won't take a mile, the controversy seems to be they're not just going to take the inch, right? Governments around the world went out, ask you to expand this capability in various ways that maybe the United States government want to.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“In the cloud, the way that other entities do. But certainly, I think that looming over all of this is that there has been several years' worth of encryption battles, not just here in the US, but around the world, primarily focused in the last couple of years on child sex abuse material, but prior to that it was terrorism, and there's always concerns about other types of material as well. And so one thing that I think is kind of a specter that's looming over this move by Apple is that they may see this as something where they can provide some kind of compromise through doing this and hopefully preserve the legality of device encryption and of end-to-end encryption writ large and maybe try and rebuff efforts that we have seen, including in the US even just last year, to effectively ban strong encryption from being legal for providers like Apple to offer to their users whatsoever. And so this might be, you know, if we give it.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“I had heard that idea from someone else I'd talk to about this and mentioned it to my colleague at SIO Alex Damos. And Alex is convinced that this is a prelude to announcing end-to-end encryption for iCloud later on. Certainly it seems to be the case that even at present, however it is that they are encrypting iCloud data at rest for photos, that that somehow makes it in their words, and this is what they have said, too difficult to decrypt everything that's in the cloud, scan it for CSAM, and do that at scale. And so it's actually more efficient. And in Apple's opinion, more privacy protective to do this client side architecture instead. I don't know enough about the different ways that Dropbox encrypts their cloud, that Apple encrypts its cloud, that Microsoft encrypts its cloud versus how iCloud does it to know whether Apple is in fact doing something different that makes it uniquely hard for them to scale.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“They could just say we're scanning all the data in the cloud directly and handing it over to the FBI or Nickmaker, whoever, but instead they want to encrypt that data. So they've now built this other ancillary system that does a little bit of local hashing comparison against a table in the cloud. It generates these complicated security vouchers. And then it reports to Nick Mick if you pass a threshold. All of that seems like”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Another tension that I want to explore with both of you, which is the sort of generalized surveillance tension around encryption and Apple specifically. Apple famously will not unlock iPhones for law enforcement, or at least they won't do it here. They say they don't do it in other countries like China. They have wanted to encrypt the whole of iCloud and famously the FBI talked them out of it. And in China, they've just handed over the iCloud data centers to the Chinese government, the Chinese government contains those keys. So they're all over the place in terms of how they are dealing with encryption. I know, and I believe what they want to do is encrypt everything and just wash their hands of it and walk away and say, it's our customers data, it's private, it's up to them. They cannot, for various reasons, do you think that tension has played into this system as it is currently architected?”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“And privacy shouldn't be a market choice. I think it's a market failure for the most part across industry. A lot of the assumptions we had going into the internet in the early 2000s was that privacy could be a competitive value. I mean, we do see a few companies competing on it. Duck DuckGo comes to mind, for example, on search. But bottom line, privacy shouldn't be left up to, I think, a release many aspects of privacy shouldn't be left up to the market.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Wrong way, I'm going to move to Canada by saying, well, I'm just going to abandon Apple devices and move to Android instead. But Android devices are basically kind of just a local version of your Google Cloud. I don't know if that's better. And at least you can fork Android and I wouldn't want to run a forked version of Android that I siloed from some sketchy place. But we're talking about a possibility that people just don't necessarily understand the different ways that the different architectures of their phone works. And you make a really good point. And a point that I've made before is that people's rights, people's privacy, people's free expression, that shouldn't depend upon a consumer choice that they made at some point in the past. That shouldn't be path dependent for the rest of time for whether or not their data that they have on their phone is really theirs or whether it actually is out in the cloud. But you're right that as the border becomes blurrier,”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Spend a lot of time unfortunately listening to wireless carrier marketing materials and tech company marketing materials Not great. But right, the idea that you have a local phone that has a networking stack that then goes and talks to a server and comes back, that is almost like a 1990s conception of modern devices, right? Like actually in 2021, everything in your house is always talking to the internet. And the line between the client and the server is extremely blurry to the point where we market the networks, right? We market 5G networks, not just for speed, but for capability. whether or not that's true but that fuzziness between client and server and network means that the consumer expectation of privacy on local storage versus cloud storage i'm wondering if like if this is actually a line that we crossed or just because apple announced it we are we are now perceiving that there should be a line it's a great point because there are a number of people who are kind of doing the equivalent of if the election goes”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“I would add some of the dissonance here is the fact that we just had Apple come out with the ask apps not to track feature, which was already in existence before, but they put they actually made prominent that dialog box to ask you when you were using an app if you want the app to track you. And so it seems a bit, again, dissonant that we just rolled out that feature and then suddenly we have this thing that seems almost more invasive on the phone. But I would say someone who's been studying privacy in the mobile space for almost a decade, you know, there is already extent to which these phones aren't ours. So the fact that anytime you have third-party apps downloading your data, like that's already been so much a feature of this ecosystem that I guess in some ways, it is a paradigm shift, but maybe it's a paradigm shift in the sense that we had areas of the phone that we maybe thought were more off limits and now they are less so.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Only in how But also from a legal standpoint, there is a big Difference between data that you hand over to a third party and assume the risk that they're going to turn around and report to the cops versus what you have in the privacy of your own home or in your briefcase or whatever. And so I do view that as a big change.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“That it might be That it might be hack Moving it down onto the device. If right now it's only for photos that are in the cloud. I think it is very different and is intruding into. What we consider, I think, a more private space Thought we could take for granted. Would stay that way. And so I do view that as”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Up until now, your local computer has not scanned your local storage in any way. But once you hit the cloud, all kinds of scanning happens. That's problematic, but it happens. But we have not yet entered the point where law enforcement is pushing a company to do local scanning on your phone or your computer. Is that the big bright line here that's causing all the trouble? I view this as a paradigm shift. To take where the scanning is happening from in the cloud, where you're making the choice to say, I'm going to upload these photos into iCloud. It's being held in third parties hands. You know, there's that saying. The cloud, it's just somebody else's And you're kind of assuming.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Necessarily well understood from the outside, but just to say that there's a bit of a symbiotic relationship here where if the company's crack down too much and force it all off their services, it all ends up on the dark web, completely out of the reach of law enforcement without really kind of heavy investigative powers, that actually in some ways disadvantages law enforcement. So to some extent one could argue that they need the companies to not crack down so much that it completely disappears off their services because it makes their job much harder. So there is a very weird tension here that I think needs to be acknowledged.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“And so I guess there's almost some question about whether Apple is kind of ensuring that there will be less CSAM on iCloud Photos three months from now than there is today because they're being more transparent about this and about what they are doing. Yeah, there is a really complicated relationship here between the companies and law enforcement that I think Bears mentioning, which is that the companies broadly are the source of all this material, you know, hands down. I don't even know if you see offline CSAM these days. It's all online and it's all being traded on the backs of these large organizations and holding it as illegal. So every copy they hold is a felony, essentially a criminal felony. And so at the same time that they are the source of this material and law enforcement wants to crack down, law enforcement needs them to report it. So there's this tension at play that I think is not”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“They've certainly learned that they won't get any plaudits for that. You've identified that this might be a point where they've been saying other organizations scan using photodNA in the cloud, and they do so over email. And I don't know how well understood that is by the general public that for most of the services that you use, if you are uploading photos, they are getting scanned to look for CSAM for the most part if you're using webmail, if you're using a cloud storage provider. Dropbox absolutely does. But you're right that they are not necessarily that forthcoming about it in their documentation. And that's something that might kind of redound to the benefit of those who are trying to track down and catch these offenders is that there may be some misunderstanding or just lack of clarity about what is happening that trips up people who trade in this stuff and share and store this stuff because they don't realize that.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“We assume that they are. At least here Apple is saying we're doing it. Here's the method by which we're doing it, that method That addition of capability to the iPhone is problematic in various ways. But they're copying to it and they're explaining how it works. Do they get points for that?”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“And you view it from Apple's side most charitably. At least Apple announced something right, they are being transparent to a degree that we went and asked Google, hey, do you do the scanning Google Photos? And there's no way to know. I just don't know the answer to that question. I think if you went to Dropbox and asked them, they would just kind of not.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Of oversight, and that's part of the frustration here, I think, is that it's very difficult to say you just have to trust us and trust everything all the way down at every point, from Nick Mick on down, and simultaneously say, you know, just know that what we're doing is not something that has other collateral harms. Because for anything outside of CSAM, you have more ambiguity and legitimate use cases and context where it matters. When it comes to CSAM, context does not matter. And that is something that I've been saying in recent days is there's no fair use for CSAM the way that there is for using copyrighted works. And so there's just this lack of information that makes it really difficult for folks like Jen or me or other people in civil society or other researchers to be able to comment. And it's really, you know, Jen, I'm so glad that you have this background so that you...”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“They are not very forthcoming because everything that they tell people to explain what is they're doing is basically a roadmap to the people who want to abuse that process and want to evade it. And so it is uniquely difficult to get information about this from the outside as a researcher, as a user, as presumably a policymaker, as a concerned parent because of this veil of secrecy that hangs over everything to do with this whole process from what is in the database to what are different providers doing. Some of that sometimes comes out a little bit in prosecutions of people who get caught by providers for uploading and sharing CSAM on their services. There will be depositions and testimony and so forth. But it still kind of black box. And so that makes it hard to critique to suggest improvements to have any kind of”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“And I don't want to just point the finger at Apple. I mean, I think that this is an industry wide problem with a real lack of devotion to resources behind it. The trouble with this particular context is how extremely unique CSAM is compared to any other kind of abusive content that a provider might encounter. It is uniquely opaque in terms of how much outside auditability or oversight or information anybody can have. I mentioned earlier that there's a risk that people might be able to try and reverse engineer what's in the database of hash values to try and figure out how could they subvert and sneak CSAM around the database, right? The other thing is that it's hard for us to know exactly what it is that providers are doing, as Jen was saying, there's a bunch of different techniques that they could take and different approaches that they can employ. But when it comes to what they are doing on the backend about”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Well, I think it's important to understand again, like I said, this is a community of users and different communities use different products in different ways. And when you're in product design, you're designing a product with particular users in mind. You kind of have your optimal user groups that you want to privilege the product for, you know, who you want to attract, how you want to design the features for. And the kind of work I did to try to understand this community, it became very clear that this group of users, they know what they're doing is illegal. They don't want to get caught and they use things very materially different than other users. And so if you're willing to put in the time to understand kind of how they operate and put in the resources to detect them and to really see kind of how they differ from other users, they're not loading up photos to share with friends and family. Like they often, they're operating under subterfuge. They know what they're doing is highly illegal. And so there's often a great deal of”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“And it's only gotten tremendously worse in that time. I mean, a lot more people have gone on the internet since then as well. So it is kind of a question of scale. But I would say industry across the board has really been bad at really trying to defeat this as an issue.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“They just simply do not want their product associated with this type of material, and they're going to be in a very public way, decide they're going to take a stand against it. Another though is very defensive because you're right. I think that there are people for whom if you're going to get caught using an Apple product, it's probably because you weren't necessarily extremely well versed in all the ways to try to defeat this type of thing. I think it's really important to remember when you talk about these issues and you think about this group of people is that they are a community. And there are a lot of different ways that you can detect this content. And I would say I would feel a lot better about this decision if I felt like what we were hearing is that kind of all other methods had been exhausted and this is kind of where we are at. And I am in no way in the belief that all other methods have been exhausted by Apple or by kind of the larger tech community at all who I think has really failed on this issue given I worked on it back in 2002 to 2004.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Right. So there's a couple things here. One is that you could take the position that Apple is being extremely defensive here and saying essentially, hey, pedophile community, we don't want you here. So we're going to, in a very public way, work to defeat your use of our products for that purpose, right? That may be quite effective. I want to actually add a little context here for why I'm in this conversation, which is before I worked in academia, I used to work in industry, and I worked for about two years building a tool to review CASM material and detect it. When I worked on this project, it was very clear from the beginning that the goal was to get it off the servers of the company I was working for. There was no higher goal here. We were not going to somehow solve the child pornography problem. And so that's where I have that particular insight in that, you know, one of the reasons Apple could be taking the stand, one, it could be a moral issue. It could be that they've decided that.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“This comes back to that efficacy question that we're talking about with Siri. Right now, in order to detect CSI material, you have to be somebody who has it. B, be putting it into your camera role and then C uploading that to iCloud Photos. I feel like this is, you know, criminals are dumb maybe. Like they're going to get caught, but it seems very easy for anybody with even a moderate amount of interest to avoid the system, thus reducing the need for this controversy at all.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Heard a lot of people talking about like does this still does this mean it's not end to end encryption anymore does this mean it's a backdoor I don't care I don't care what we're calling it that's a way of distracting from the main things that we're actually trying to talk about here which I think are what are the policy and privacy and free expression and data security impacts that will result from Apple's decision here and how will that go out beyond the particular CSAM context and will what they're doing work to actually protect children better than what they've been doing to date so you know quibbling over labels is just not very interesting to me frankly”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Using the word backdoor because it's a very loaded term, it means different things to different people. And I don't know that I agree with that because this is all still happening on the client is very careful to not mess with their end-to-end encryption for iMessage. It is a way Agree gives an insight into what people are doing on their phone that was not there before. I don't know whether that means that you could characterize it as a backdoor.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“The concerns that has been raised with having this kind of client-side technology being deployed is that if once you're pushing it to people's devices, it is possible, potentially, this is a concern of researchers in this space for people to try and reverse engineer that, basically, figure out what is in the database. There's a lot of research that's done that this fears on one side of, well, what if something that is not CSM gets slipped into this database? The fear on the other side is what if people who have really strong motivations to continue trading CSAM try to defeat the database by figuring out what's in it, figuring out how they can perturb an image so that it slips past the hash matching feature, and that's something that I think is a worry that once this is put onto people's devices rather than tackling server side as currently happens with other technologies such as photodNA that scan photos in the cloud that you are opening up an avenue.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT
“Of theory behind why they've decided to take this particular tactic because it seems, I mean, when I first heard about it, the idea that I was going to have the entire Nick McHash database sitting on my phone, I mean, obviously, you know, hashes are extremely small text files. We're talking about just strings of characters that to the human eye just looks like garbage, right? And they don't take up a lot of memory, but at the same time, the idea that we're pushing that to everybody's individual devices was kind of shocking to me, and I'm still kind of in shock about it because it's just such an use case than what we've seen before.”
2021-08-10 · Decoder with Nilay Patel · Here’s why Apple’s new child safety features are so controversial · IDENTIFIED FROM THE TRANSCRIPT