YouSaid · the spoken record
Cristos Goodrow
- lines on the record
- 81
- first
- 2020-01-25
- most recent
- 2020-01-25
- sittings or episodes
- 1
- sources
- podcast
Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections
“I mean, I think Google is thinking about virtual reality. I don't think about virtual reality too much. I know that we would want to make sure that YouTube is there when virtual reality becomes something, or if virtual reality becomes something that a lot of people are interested in. But I haven't seen it really take off yet”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I think there's a big opportunity there. And then, as I said before, we want to make sure that YouTube finds a way to ensure that it's acting responsibly with respect to society and enriching people's lives. So we want to take all of the great things that it does and make sure that we are eliminating the negative consequences that might happen. Lastly, if we could get to a point where all the videos people watch are the best ones they've ever watched, that would be outstanding too.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I think introducing people to more of what's available on YouTube is not only very important to YouTube and to creators, but I think it will help enrich people's lives because there's a lot that I'm still finding out is available on YouTube that I didn't even know. I've been working YouTube eight years and it wasn't until last year that I learned that Could watch USC football games from the 1970s I didn't even know that was possible until last year. And I've been working here quite some time. So, you know, what was broken about that that it took me seven years to learn that this stuff was already on YouTube even when I got here.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that YouTube is evolving to take the place of TV. You know, I grew up as a kid in the 70s and I watched a tremendous amount of television. And I feel sorry for my poor mom because people told her at the time that it was going to rot my brain and that she should kill her television. But anyway, I mean, I think that YouTube is, at least for my family, A better version of television, right? It's one that is on demand. It's more tailored to the things that my kids want to watch. And also they can find things that they would never have found on television. And so I think that at least from just observing my own family, that's where we're headed is that people watch YouTube kind of in the same way that I watch television when I was younger.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Credit for. I mean, in terms of search and discovery, we have been working on trying to summarize videos in text or with some kind of labels. For eight years, at least. And we're kind of so. So if you were said.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“And I don't know if predicting the next frame is really going to get there because I'm not an expert on compression algorithms, but I understand that that's kind of what compression video compression algorithms do is they basically try to predict the next frame and then fix up the places where they got it wrong. And that leads to higher compression than if you actually put all the bits for the next frame there. So I don't know if I believe that just being able to predict the next frame is going to be enough because there's so many frames and even a tiny bit of error on a per frame basis can lead to wildly different videos.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I mean, we have systems that already watch all the videos on YouTube, right? But they're just looking for very specific things, right? They're supervised learning systems that are trying to identify something or classify something.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, video is just so much information. And so precision becomes a real problem. It's like a, you know, you're trying to classify something and you've got a million classes and you, the distinctions among them, at least from a From a machine learning perspective are often pretty small, right? You need to see this person's number in order to know which player it is. And there's a lot of players. Or you need to see the logo on their chest in order to know which team they play for. And so, and that's just figuring out who's who, right? And then you go further and saying, okay, well, was that a goal? Was it not a goal? Like, is that an interesting moment, as you said, or is that not an interesting moment? These things can be pretty hard.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“But folks are working on the more automatic version. It's interesting people might not imagine this, but a lot of our systems start by using almost entirely the audience behavior. And then as they get better, the refinement comes from using the content.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't remember how it's presented. And the other thing I would say is that right now, it's based on creator annotations.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“So Google now on Google search will help you find the segment of the video that you're interested in. So if you search for something like how to change the filter in my dishwasher, then if there's a long video about your dishwasher and this is the part where the person shows you how to change the filter, then it will highlight that area. Provide a link directly to it.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“We've actually had projects that attempt to achieve this, but it really does depend on understanding the video well and our understanding of the video right now is quite crude. And so I think it would be especially hard to do it with a conversation like this. One might be able to do it with Let's say a soccer match more easily, right? You could probably find out where the goals were scored. And then, of course, you need to figure out who it was that scored the goal. And that might require a human to do some annotation. But I think that Trying to identify coherent topics in a transcript like the one of our conversation. Is not something that we're going to be very good at right away”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, it's not something that I think we've done yet, but I can tell you that I think clipping is great. And I think it's actually great for you as a creator. And here's the reason. If you think about, I mean, let's say the NBA is uploading videos of its games Well, people might search for Warriors versus rockets, or they might search for Steph Curry. And so a highlight from the game in which Steph Curry makes an amazing shot is an opportunity for someone to find a portion of that video. And so I think that you never know how people are going to search for something that you've created. And so you want to, I would say, you want to make clips and add titles and things like that so that they can find it as easily as possible.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“You asked me about how to make a video go viral or make a viral video. I don't think that if you or I decided to make a video about 96 million balls that it would also go viral. It's possible that Derek made the canonical video about those. Black balls in the lake.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I think it's the system basically doing what anybody who's recommending something would do, which is you show it to some people and if they like it, you say, okay, well, can I find some more people who are a little bit like them? Okay, I'm going to try it with them. Oh, they like it too. Let me expand the circle some more, find some more people. Oh, it turns out they like it too. And you just keep going until you get some feedback that says, no, now you've gone too far. These people don't like it anymore. And so I think that's basically what happened. Now,”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I think people expect that if a video is going viral and it's something they would be interested in, then I think they would expect YouTube to recommend it to them.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“I don't know how you make a viral video. I know that we have in the past tried to figure out if we could detect when a video was going to go viral. And those were, you take the first and second derivatives of the view count and maybe use that to do some prediction. But I can't say we ever got very good at that Oftentimes we look at where the traffic was coming from. You know, if a lot of the viewership is coming from something like Twitter, then maybe it has a higher chance of becoming viral than if it were coming from search or something. But that was just trying to detect a video that might be viral. How to make one? I have no idea. You get your kids to interrupt you while you're on the news or something. Idea.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Saying that they're more satisfied. They're giving more videos five stars after they watch them, then those would be indications of that the experiment is successful, that it's improving the situation for viewers. But we can also look at other things. Like we might do user studies where we invite some people in and ask them, like, what do you think about this? What do you think about that? How do you feel about this? And other various kinds of user research. But ultimately before we launch something, we're going to want to run an experiment so we get a sense for what the impact is going to be, not just to the viewers, but also to the different channels and all of that.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“You mentioned that AB experiments. And so just about every single change we make to YouTube, we do it only after we've run an AB experiment. In those experiments, which run from one week to months, We measure hundreds, literally hundreds of different variables and measure changes with confidence intervals in all of them. Because we really are trying to get a sense for ultimately, does this improve the experience for viewers? That's the question we're trying to answer. And an experiment is one way because we can see certain things go up and down. So for instance, if we notice in the experiment, people are dismissing videos less frequently or they're”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“And over time, you collect some data and you refine your situation so that you're making less heuristics and you're building a system that can actually learn what to do in different situations based on some observations of those situations in the past. And you keep chipping away at these heuristics over time. And so I think that just like with diversity, I think the first diversity measure we took was, okay, not more than three videos in a row from the same channel, right? It's a pretty simple heuristic to encourage diversity, but it worked right. Who needs to see four, five, six videos in a row from the same channel? And over time, we try to chip away at that and make it more fine-grained and basically have it. Remove the heuristics in favor of something that can react to individuals and individual situations”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“You start trying to solve a problem, and the first way to solve a problem is often with a simple heuristic, right? And you want to say, what are the videos we're going to recommend? Well, how about the most popular ones, right? That's where you start.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Exactly, right? Like, if there were no people who came to YouTube tomorrow, then the algorithm wouldn't work anymore. So that's a critical part of the algorithm. And so when people talk about, well, the algorithm does this, the algorithm does that, it's sometimes hard to understand, well, you know, it could be the viewers are doing that and the algorithm is mostly just keeping track of what the viewers do and then reacting to those things. In sort of more fine grained situations. And I think that this is the way that the recommendation system and the search system and probably many machine learning systems evolve is”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, we don't usually think about it as the algorithm because it's a bunch of systems that work on different services. The other thing that I think people don't understand Is that what you might refer to as the YouTube algorithm from outside of YouTube is actually a bunch of code and machine learning systems and heuristics, but that's married with the behavior of all the people who come to YouTube every day.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“And, you know, I think that it probably in some of these cases contributes to better videos once they come back, right? Because a lot of people, I mean, I know myself, if I'm burned out on something, then I'm probably not doing my best work, even though I can keep working until I pass out. And so I think that the taking a break may even improve the creative ideas that someone has.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, taking a break is absolutely okay. And the reason I say that is because we can observe many examples of being of creators coming back very strong and even stronger after they have taken some sort of break. And so I just want to dispel the myth that this somehow necessarily means that your channel is going to go down or lose views. That is not the case. We know for sure that this is not a necessary outcome. And so we. Want to encourage people to make sure that they take care of themselves. That is job one, right? You have to look after yourself and your mental health”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, the first thing is we want to make sure that the YouTube systems are not contributing to this sense, right? And so We've done a fair amount of research to demonstrate that you can absolutely take a break. If you are a creator and you've been uploading a lot, we have just as many examples of people who took a break and came back more popular than they were before as we have examples of going the other way.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“When I'm not watching YouTube, say, gosh, I really would like to see something from someone or about something, right? And so that's why I think this connection between fans and creators is so important for both because it's a way of sort of fostering a relationship that can play out into the future.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I mean, we're trying to help creators find the biggest audience that they can find. And the reason why that's You brought up creator versus video. The reason why creator channel is so important is because if we have a hope of people coming back to YouTube, well, They have to have in their minds some sense of what they're going to find when they come back to YouTube. If YouTube were just the next viral video and I have no concept of what the next viral video could be, one time it's a cat playing a piano and the next day it's some children interrupting a reporter and the next day it's some other thing happening, then it's hard for me to.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Right, which is great. Makes me feel great. But then, of course, when I ask them, well, what is it that you love about YouTube, not one time ever has anybody said that the search works outstanding or that the recommendations are great? What they always say when I ask them, what do you love about YouTube is they immediately start talking about some channel or some creator or some topic or some community that they found on YouTube and that they just love. And so that has made me realize that YouTube is really about the video and connecting the people with the videos. And then everything else kind of gets out of the way.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Literacy is low, and they think that YouTube can help in those places because you don't need to be able to read and write in order to learn something important for your life. Maybe how to do some job or how to fix something. And so that's another way in which I think YouTube is possibly changing society. So I've worked at YouTube for eight almost nine years now. And it's fun because I meet people and you tell them where you work. You say you work on YouTube and they immediately say, I love YouTube.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, the fact that unlike other mediums, there's not someone sitting at YouTube who decides before you can upload your video whether it's worth having you upload it or worth anybody seeing it really right and so you know there are some creators who say like I wouldn't have this opportunity to reach an audience, Tyler Oakley. Often said that, you know, he wouldn't have had this opportunity to reach this audience if it weren't for YouTube. And so I think that's one way in which YouTube has changed society. I know that there are people that I work with from outside the United States, especially from places where”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, not the hours themselves, but The fact that you return on another day. So that's probably the most simple indicator. People don't come back to things that they don't find value in, right? There's a lot of other things that they could do. But like I said, ideally we would like everybody to feel that YouTube enriches their lives and that every video they watched is the best one they've ever watched since they've started watching YouTube. And so that's why we survey them and ask them like, is this one to five stars? And so our version of success is every time someone takes that survey, they say it's five stars. And if we ask them, is this the best video you've ever seen on YouTube? They say yes every single time. So it's hard to imagine that we would actually achieve that. Maybe asymptotically we would get there. But”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“I think, first of all, if you come back and watch more YouTube, then that's one indication that you found some value from it.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“And other times they just are done and they're ready to move on to something else. And so the question is, well, what is the something else? And one of the first things one can imagine is, well, maybe something else is the latest video from some channel to which you've subscribed. And that's going to be very different for you than it is for me, right? And even if it's not something that you subscribe to, it's something that you watch a lot. And again, that'll be very different on a person by person basis. And so Even the watch next, as well as the homepage, of course, is quite personalized.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“It's very, very personalized. So Your experience will be quite a bit different from anybody else's who's watching that same video, at least when they're logged in. And the reason is that we found that users often want two different kinds of things when they're watching a video. Sometimes they want to keep watching more on that topic or more in that genre.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“That's a good tip. We should keep that fact in mind when we design these features. I'm not sure I really thought about it that way, but that's a very interesting perspective.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“There is a line that you don't cross. And if you go too far, maybe your thumbnail is especially racy or it's all caps with too many exclamation points. We observe that users are kind of They're sometimes offended by that. And so for the users who are offended by that, we will then depress or suppress those videos.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“On the other hand, I have other more pop science type books, and they all have very interesting covers, right? And they have provocative titles and things like that. I mean, I wouldn't say that they're clickbaity because they are indeed good books. And I don't think that they cross any line, but that's just a decision you have to make, right? Like the people who write classical recursion theory by Piero Freddie, he was fine with the yellow title and nothing more. Whereas I think other people who wrote a more popular type understand that they need to have a compelling cover and a compelling title. And, you know, I don't think there's anything really wrong with that. We do take steps to make sure that”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Think that you can look at it as an attempt to game the algorithm, but even if you were to take the algorithm out of it and just say, okay, well, all these videos happen to be lined up, which the algorithm didn't make any decision about which one to put at the top or the bottom, but they're all lined up there, which one are the people going to choose. And I'll tell you the same thing that I told Derek. I have a bookshelf and they have two kinds of books on them, science books. I have my math books from when I was a student. And they all look identical except for the titles on the covers. They're all yellow. They're all from Springer. And they're every single one of them. The cover is totally the same Right”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“I really don't think I have a sense for the relative sizes of those groups, but I would say that. Know people come to YouTube with some certain amount of intent. And as long as they, to the extent to which they try to satisfy that intent, that certainly helps our systems, right? Because our systems rely on kind of a faithful amount of behavior. Right? And there are people who try to trick us, right? There are people and machines that try to associate videos together that really don't belong together, but they're trying to get that association made because it's profitable for them. And so we have to always be resilient to that sort of attempt at gaming the systems.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“And remember, the collaborative filtering part of the system starts by the same user watching videos together, right? So the way that they're probably going to do that is by searching for them.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Right? I mean, and so you're right, the people who don't skip it and select it are going to be delighted. But other people might say it. Yeah, this is not what I was looking for.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Right, and I would go further and say that if you want people, human beings, to select your video insert, then it helps to have, let's say, World of Warcraft in the title. Why would a person's, you know, if they're looking at a bunch, they type World of Warcraft and they have a bunch of videos, all of whom say World of Warcraft, except the one that you uploaded, well, even the person is going to think, well, maybe this isn't somehow search made a mistake. This isn't really about World of Warcraft. So it's important not just for the machine learning systems, but also for the people who might be looking for this sort of thing. They get a clue that it's what they're looking for by seeing that same thing prominently in the title of the video.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“World of Warcraft wasn't in the title. It was match 478, you know, A team versus B team. And World of Warcraft wasn't in the title. Just like, come on, give me literal.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“For instance, we were a few years ago having a live stream of some competition for World of Warcraft on YouTube. And it was a very important competition. But if you type World of Warcraft in search, you wouldn't find it.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Analyzing the content of the video, right? And what I can tell you is that. Our ability to do it well is still somewhat crude. We can Can tell if it's a music video. We can tell if it's a sports video. We can probably tell you that people are playing soccer. We probably can't tell whether it's Manchester United or my daughter's soccer team. So these things are kind of difficult. And using them, we can use them in some ways. So for instance, we use that kind of information to understand and inform these clusters that I talked about and also maybe to add some words like soccer, for instance, to the video if it doesn't occur in the title or the description, which is remarkable that often it doesn't. One of the things that I ask creators to do is please help us out with the title and the description”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“That's right. And so, you know, we can't just. Close our eyes and say, Sorry, you're using it wrong. We're not going to pay attention to what you've done. We need to embrace all the ways in which all the different people in the world use the subscribe button or the like and the dislike button.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“And we tried to let them understand actually, it doesn't cost anything, it's free, it just helps us know that you are very interested in this creator. But then we've asked other people who subscribe to many things and don't really watch any of the videos from those channels. And we say, well, why did you subscribe to this if you weren't really interested in any more videos from that channel? And they might tell us, well, I just, you know, I thought the person did a great job and I just want to kind of give them a high five.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“It's very important, right? We want it's predictive of satisfaction. But it's not perfectly predictive. Subscribe. If you subscribe to the channel of the person who made the video, then that also is a piece of information and it signals satisfaction. The years, we've learned that people have a wide range of attitudes about what it means to subscribe. We would ask some users who didn't subscribe very much, but they watched a lot from a few channels. We'd say, well, why didn't you subscribe? And they would say, well, I can't afford to pay for anything.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, you mentioned commenting, also sharing the video. If you think it's worthy to be shared with someone else you know.”
2020-01-25 · Lex Fridman Podcast · Cristos Goodrow: YouTube Algorithm · IDENTIFIED FROM THE TRANSCRIPT · source