YouSaid · the spoken record

Ryan J. Salva

lines on the record
72
first
2022-09-04
most recent
2022-09-04
sittings or episodes
1
sources
podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

  1. “Come som So, easy one. How can folks find me? I am Ryan J. Salva everywhere. Twitter, GitHub, pick your choice, LinkedIn, Ryan J Salva. And then how can folks be salvaged? Like, please, like there is a 60-day free trial of co-pilot that is there for everyone to pick up and use and go try it out. And when you do post either on Twitter or hacker news or on discussions, GitHub discussions, your experience, give us the good feedback, give us the bad feedback. I am so hungry to see how people are using it in novel ways and where they're running up against kind of the rough edges too. Like I said, there's lots of room for us to grow and improve from here. But I'm pretty confident that developers will be pretty. Freaking amazed at what it's already capable of”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  2. “There are many, but I think for today, I would probably beat myself up if I didn't say Uga Damor. Uga is the primary researcher who really kind of like is the true innovator for co-pilot. He deserves credit for the initial work and is a brilliant technologist and futurist. I really, really respect him a lot.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  3. “My go-to kind of throwaway answer there about did they teach me something about 18th century art and its connection to religious trends at the time? Someone taught me that. It was astounding. It was actually a university candidate. So someone who was still in university. And she was from Vanderbilt University.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  4. “And they're graded on three different criteria. So one is completeness. Do they actually finish the lesson inside of one minute? Two is complexity. So it's one thing if you teach me how to, I don't know, pat my head and rub my stomach at the same time. It's another thing if you teach me something about 18th century ardent connection to religious trends at the time. And then last is really clarity. Oh, yeah, clarity is the last one. Clarity is like, do I actually understand, did I learn something by the end of the lesson? Did they convey the idea fully and wholly?”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  5. “Let's see here. So I'll give you a fun one more than it is a challenging one. This is kind of my icebreaker interview question, particularly for more early to mid career product managers. So I ask them to teach me something new in one minute. And so I'll usually like I'll pull up my phone and I'll like start the timer. I'll give them a second to think about it and start the timer.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  6. “I don't know how if this counts as recent, but it's one that I watched recently, which was Arrival. Yeah, that counts a rival. So, movie about ostensibly about aliens, but is really about language and memory. And I found that really, really compelling.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  7. “Oh, God, there's so many. I consume hundreds of hours of podcasts every month. It is crazy. Yeah, so I can choose many. I'll give you just one. So the memory palace with Nate Tobao is an excellent storytelling podcast. He does about 20 minute vignettes, usually selected from kind of American history. He also was the artist in residence at one of the museums in Washington, D.C. And if you're ever at, I think it's the American History Museum or something like that, if you're ever there, you can go to like different rooms in the museum and he'll tell you stories about the kind of the objects or the rooms that you see there. It's a magical experience recommended to anyone.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  8. “It's a collection of short stories. And essentially what it is, is it is like, if you're watching a movie and like the villain gets their opportunity to have their big speech, which kind of explains why they are, who they are, right? And makes them maybe a little bit vulnerable in that moment. It's that speech like 10 times over for different hideous people, terrible, terrible people. So it's an interesting read. I recommend it.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  9. “Oh, good question. So one of them is a book on user experience called Make It So. It's a reference back to Star Trek. And the idea here is essentially that user experiences that are presented to us in sci-fi often make their way into our everyday products and tools 20, 30 years down the line. It is a great eye-opening, illuminating, and just really fun book. So that's one. And then completely different take. I'll go outside of tech and I'll just do like entertainment value. There's a David Foster Wallace book called Breathe Interviews with Hideous Men that I love.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  10. “About 25, maybe 30% of the team's capacity should generally be on just operations. Like how do we keep our in-market products meeting customer expectations? And then the remainder of it was about 60% or so is really on incremental progress for our end market products. How do we make iterative improvements and continue to actually realize payoff for the larger bets that we made one, two, three, four years back? And from a rough kind of like distribution, that's generally how I run kind of my larger teams. That works when you have larger teams, though. At startups, you know, where we were pretty much only a big bet. Obviously, your percentages get very different. And it becomes a matter of you're all in for that one proverbial lottery ticket, right?”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  11. “Yeah, so as a both a product manager and a portfolio manager of multiple products, I'm responsible for multiple product lines at GitHub. The allocation of time, of focus, energy, and resources becomes a really challenging question, the answer to which isn't always the same depending upon the time kind of world circumstances, original circumstances, technology circumstances. As a general rule, as a general principle, I certainly try to make sure that we're always reserving some capacity for bold, audacious experimental research projects. You can kind of think of those like those really uncertain bets as being 5 to 10% of the team's capacity.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  12. “It often feels very connected. I mean, like, certainly the education side of things taught me that the importance of dialogue, the importance of skepticism is valuable in so much more than esoteric armchair ponderings. It's actually applicable to the real world.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  13. “We want you to keep all of those same systems in place to make sure that humans who are leveraging tools like Copilot continue to produce that good quality code. But there's a lot of, at the same time, like... Where is AI stack? Is AI eventually going to be? This is back to your question about where will we be five, ten years from now? Will it be writing 90% of the code? We don't want copilot to be that to replace anything. We want it to augment. The idea here is really that AI is an enabler for developers to focus on the creative work, to stay in the flow, to be able to move faster, right and kind of working through those anxieties, working through that healthy skepticism takes conversation. It takes dialogue. And that takes us on the product side having kind of that guided conversation with the community.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  14. “Right, like, is there going to be a new attack vector that we just haven't really thought of yet around AI that might produce negative consequences? We think that we've done a really good and responsible job of that by making sure that first, we're very clear that co-pilot is not a replacement for a developer. It will never be. We do not want co-pilot auto generating code where a thinking, reasoning, breathing human being is not on the other side of that keyboard, making recent decisions. We do not want copilot to replace any other part of the stack, whether it is static analysis tools or unit tests or whatever kind of measures you're putting in today to make sure that your humans produce good quality code.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  15. “So it's a couple of different reasons. I mean, one, like I said, we are trained on public code and like not all of the community is really sure like when is it okay to train a model on public code? When is it not okay to train a model on public code? Is copilot producing secure suggestions? Is co-pilot producing buggy suggestions? There's a lot of doubt. There's a lot of very healthy skepticism. And I actually, I mean that genuinely. I want people to be skeptical of co-pilot. We owe it to ourselves as a community to be skeptical of any AI because just like there's great potential for benefit, there's also great potential for harm. How like people keeping us accountable like how are you preventing things like model poisoning?”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  16. “I would also say here that operationally another challenge has been how do we create a model that the community really feels like ownership over, right? And like the dialogue that's had to happen as we brought an AI tool to market, especially one that is trained on public code, right? Has required a lot of dialogue between us and our community. And every good product manager should be spending as much of their time as possible with their customers, with their potential customers. Copilot in particular has been a more complicated kind of rollout because we as a kind of as an industry, as a society are still figuring. Out how to make sense of it. And so the amount of kind of give and take between developers and us as a product team has really required us to scale up more of the product team than it has the engineering team.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  17. “Yeah, so there's a few dimensions of that. One is of a problem that's very much of our time in the world, namely that supply chains have been disrupted dramatically over the course of the last few years. And it turns out that co-pilot for both training and operating the models requires some very rare and unique GPUs that there's not a lot of global supply of. And so part of it is just like, can we get enough hardware in order to run these things? We've actually earmarked quite a bit of capacity and we are greedy, greedy, greedy for more capacity globally. As soon as we can produce those chips and get them in data centers, we do it. So that's been one kind of unique challenge.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  18. “Function that they're trying to figure out. Yeah. I want Stack Overflow to stay in business, but I would mind a little bit less context switching myself.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  19. “Is that not only will we be inviting more developers or more people to become developers, essentially providing a layer of abstraction a little bit or at least a little bit of a kind of a hand in development, but that also the really experienced developers are focusing on much larger problems and focusing on outcomes and creativity rather than really kind of low level kind of difficult rote memorization of things like syntax or ordering of parameters and the like.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  20. “And you just have to, as the contributing developer, just review it to make sure it's accurate, send it on its way, and you don't have to put an extra effort for that. There are lots and lots of different opportunities for AI to essentially be able to take some of the drudgery out of our work so that we can focus on creative acts. What I hear from developers and what I experience myself is that Copilot kind of forces me to think a little bit more about what are the design patterns I'm trying to create? What is the end user experience or the outcomes that I'm trying to drive with my code? And that I can kind of rely on co-pilot to scaffold out a lot of that so that I can focus on more creative work. And that is really what I hope for our industry five, 10 years from now.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  21. “It does. It enables even mediocre developers like myself to be able to do some pretty amazing things. Where's it going? Like, so first, I think, I hope it's obvious to most developers that AI is going to infuse pretty much our entire development stack and the not so distant future. Copilot is really just kind of the very tip of the sphere for a lot of innovations and better managing maybe our build cues or helping to here's a great one. You know, I don't know about you, but often the comments that I get with commit messages and PRs aren't super great. It kind of, you know, it puts a lot of effort onto the code reverer to go figure out what the developer was actually trying to do. What if AI could summarize all of your changes with your pull request?”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  22. “Yeah, yeah. And so just to put a fine point on that stat, it is 40% is specifically for Python developers. Candidly, it varies depending upon the language. Because as you might imagine, some languages have better representation in kind of the public domain than others. And usually both the volume and the diversity of training data correlates with the quality of suggestions, which is then represented by either the number of lines written.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  23. “Or simple block lists is maybe another proof point both of how AI as a solution for common development problems is getting way better. It's solving more parts of our stack or filling in for more parts of our stack. And how at least in our case, we were pretty fortunate to be able to deliver on or depend on a parent company's contributions to solve a real acute problem that GitHub probably could not have solved on our own.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  24. “And so always kind of like dancing the dance of like editorial content, we're actually at a place now where we're able to partner with the Azure Department of Responsible AI. And they've created some really extraordinary models that help detect, I'll call it sentiment, for lack of a better word. Basically, when there is something that is patently offensive. Because there are some words that in some contexts may be offensive and in some contexts may be totally reasonable, right? Especially when you get into software for medical kind of scenarios, right? And so being able to start to shift a little bit, to shift a little bit to focus or to rely on AI models that can also do a better job than we could with crew.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  25. “And so, you know, often we would get feedback from developers of like, hey, this particular word was blocked and like that it was blocked either was offensive to me or prevented me from being able to kind of get good value out of the product”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  26. “When we started out, I'll continue on the example of the crazy pair programmer whispering crazy things into your ear. When we first started out, we didn't really have any filter on copilot whatsoever at the very, very, very early days. And then eventually we're like, okay, it needs to be slightly more controlled experience. We need to edit out some of the most egregious stuff. And so we introduced a simple block list of words. And these block lists are always fraught with peril. Like, you know, which words are okay, which words not okay. All of a sudden we become editors and that's just of like language and that's kind of a scary place to be. I'm not comfortable with it at least, but like at a certain level, it has to be done because otherwise you're going to create a bad developer experience.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  27. “It is conversations with a very, very wide cast of characters. This product in particular, I probably spent more time with legal than any other product that I've ever kind of been responsible for, all wonderful, creative people. But it's not just legal. It is also privacy and security champions. It is frankly developers, like the people who are using it, like listening to them, like, hey, what works here, what doesn't work for you here, why is this offensive? Why is it not offensive?”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  28. “It's going to be really distracting. And so really kind of coming down to some principles about what is the use case we're trying to solve, what is appropriate. I put this in scare quotes, behavior of the AI kind of bot sitting side by side with you helped us kind of create some principles or some guidelines for the developer experience that we wanted to create.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  29. “The framing of co-pilot as an AI pair programmer, I think is a useful one. Pair programmer, I suspect most of your listeners will know, but like if pair programmer is usually two developers sitting side by side, working on a problem together. One's at the keyboard and the other one's kind of like helping them talk through it, talk through the ideas and make corrections, that kind of thing. Well, if copilot is your AI pair programmer and they're whispering crazy stuff into your ear and they're bringing politics into it or gender identity into it or I don't know whatever other, they're spouting off, slang and slander and all that kind of stuff. You're probably not going to be able to focus on your work, right?”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  30. “Fun enough Well, and I mean, this is the best problem in the world to have. Talk about kind of customer astlic for co-pilot in particular, the amount of chatter, the amount of customer feedback that was coming in, especially for us with AI. I mean, the world is still figuring out AI candidly. We're getting a lot, lot better at it, especially in the last couple of years, you know, with things like Dolly and Copilot. It brings with it not only engineering challenges, but also, frankly, ethical challenges, right? Like, you know, and legal challenges, like making sense of how we actually, what are expectations are of AI. And if AI produces something that is offensive, who's at fault? Our stance on it, well, we ended up coming to is actually.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  31. “Experiment in a research project to an operational product. And often because those researchers are, they're the first wave that come over. Engineers and researchers who are really thinking about what is the idea that we've created? What is the new thing that we've brought to market and can bring that vision to it?”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  32. “And feel like they control the roadmap. And so making sure that you're not outsourcing innovation exclusively to an R&D team, but that is happening within the product team as they take ownership over the idea and over kind of the use case and the customer. Last, I would say here is really that engineering fundamentals in a lot of ways are the contracts that differentiate an R&D team from an operational product team. And bringing that fundamentals process into it is going to feel candidly a little bit unnatural to the researchers. And that takes, therefore, a little bit of cultural change management for everyone to just kind of like adapt their way of working and understand that we're graduating from an”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  33. “The criteria for moving researchers back into their R&D team, whatever that happens to be for your organization. That can't be based on a calendar. It needs to be based on a replacement, in-seat, who's actually doing the job and has picked up all of the skills necessary. And only then can the research and move back. So make sure that you've got continuity of expertise and skill sets and domain familiarity before you move over. I feel like we've managed that pretty well today. As well, it's critical that the team who is taking over from the R&D shop feels like they have control over their own future. You can't really delegate roadmap to an R&D team. The team who's responsible for maintaining the product, for building the product, who has the closest feedback loop with the end customer, they're the ones who really need to own.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  34. “Go back to GitHub next to do what they do best, which is be innovative and creative and think about the next moonshot. That process, that took, well, we're actually still kind of at the tail end of it now. Here we are, like I said, roughly a year and a half after kind of the initial kind of creation of the product, having gone through technical preview, achieved general availability. We've now hired in a team around them and the researchers starting actually as early as last month have started to gradually move back over to GitHub Next and a EPD squad, multiple EPD squads actually are now taking the product forward and starting to respond to customer feedback to thinking about, okay, how do we now as a product team carry this roadmap forward from An idea that originated in GitHub Next.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  35. “And for a finite period of time, move them over to create a new EPD squad. We want them to be researchers, but we need to do knowledge transfer and we need to actually kind of provide the seed for a team that could eventually operationalize and productize. And that kind of began the technical preview where we started to invite tens of thousands, then hundreds of thousands to the technical preview. And in that technical preview, when we started to see like crazy mind-blown emoji tweets and threads on hacker news about people getting really, really excited about it, that's how we knew it was time to start scaling and it was time to really start thinking about how do we do hiring so that we can build in some insulation around these researchers so that they can eventually”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  36. “really started. And this is actually where my role in the product life cycle kind of really started to increase. I had kind of been in tight connection and kind of been monitoring the work and kind of like consulting a little bit with the next team prior to that. But it was that moment when we identified that, okay, this is actually something real. Customers are saying developers are saying this is magical. This does something extraordinary that I could not do on my own, that we started to think about, okay, how do we transition this over? And so from there, we really just like, okay, we think we've got a hit here. We think we've got something that we can actually bring to developers. And so we made an intentional decision to take some of the researchers who were in the next.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  37. “And also, when you do get to a place where that team has an idea that is clearly connected to a representative set of customers who have a genuine problem and there is signal with at least medium confidence that this solution, whatever it is, solves it in a novel way. That's the time to start thinking about, okay, like let's actually put a little bit of, I'm going to call this market testing. It's nothing so formal as market testing. It's really like, let's start to actually bring prototypes of this in front of more and more customers to kind of test it out and see, hey, is this actually solving a problem for you? Is this something you would use? And this is where the transition between next and EPD at GitHub.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  38. “Yeah. I mean, I think the first step is to invest in it. Like the first step is really hire really smart people, attract smart people, and give them the opportunity to be creative. Don't expect anything out of them that is going to turn into a money maker or something that is going to be beholden to fundamentals around security, privacy, uptime, accessibility, all that groovy kind of stuff up front. They need space to create an experiment.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  39. “Yeah, it is, I would say it is not necessarily a Microsoft thing, but it is definitely at GitHub, how we have really contextualized it. Not to say that there aren't teams at Microsoft who might also kind of use kind of that methodology, but where we've been really maybe explicit or intentional about it is at GitHub, where we've actually ring-fenced a team to think about that Horizon 2 and Horizon 3 work and kept them separate from EPD, EPD here being engineering, product and design, the folks who are”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  40. “Not necessarily a concrete definition. For me, I usually ballpark it as first horizon as the next year, second horizon the next three years, third horizon next five years. But we generally think of it more as like a measure of ambiguity and confidence level more than calendar dates.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  41. “Yeah. So the original team that was working on co-pilot at GitHub was a team that we called GitHub Next. And essentially their job is to work on second and third horizon projects, what some folks might call moonshots, right? Things that we never really expect work in the next one or two years, but Mike, three, five years down the line actually turn into something meaningful.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  42. “That's fairly accurate. Yeah. I mean, we had a model that really was amazingly good, like a step level change in actual intelligence, right? And then mirroring that up against a really good use case that actually changes developers' fundamental experience of the creation process, the creative process.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  43. “And we also experimented quite a bit. So it's not just about the model, but it's also about what you feed the model. How do you prompt the model to return back a useful response? And so this kind of began a journey of experimentation for what we call prompt crafting.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  44. “So, depending upon where you're in the world, your latency can go up or down a little bit from there. But it seems like the sweet spot is somewhere around 200 milliseconds.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  45. “Was not just like the model creating the model is one thing, but then figuring out how to use the model in terms of what parameters do you want to kind of like adjust for? What do you want to optimize for in terms of a great example of this is performance, right? When you're in a code editor, you don't necessarily want to type type type and then have to wait one second, two seconds, three seconds to get a suggestion back when you're entire goal is to stay in the flow. And so we would run experiments to see like how many milliseconds are the right amount such that a developer doesn't feel like they're being interrupted by co-pilot and a suggestion. What's the answer that it seems like right now it's around 200 milliseconds?”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  46. “So the period in between kind of open AI almost taking down GitHub and then us really arriving at the kind of the user experience. Part of that was frankly a lot of really smart researchers at OpenAI experimenting and doing what like only world-class AI researchers can do. It was a lot of them experimenting, occasionally asking for updates to the data set, tossing back to us a model that we might play with and tinker around with. And these models have like literally thousands of parameters that you can pass to them. So when you're really thinking about kind of GPT-3 and codecs and then the transition from that to something like copilot.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  47. “Pretty early on, we landed on this user experience that is co-pilot as most developers experience it today. That was now, I want to say that was at least 16 months ago, 16 months ago. Since then, we brought it to developers. Just”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  48. “But it's kind of out of the cursor, right? And you had to really, even if you weren't switching over to a different window, you still had to switch over to a different panel, which itself was a little bit distracting. And we eventually came to this idea of inline autocomplete. We were able to, with the kind of partnership of some of our friends over on the Microsoft side of things, partner with our friends in Visual Studio Code, be like, hey, There's not really an extensibility yet in your editor for this multi-line autocomplete, but we've got an idea for how this might work, played around with kind of the actual presentation of it, what should the keystrokes be, what should the presentation layer be, you know, the gray italicized text seem to be a good way of indicating that it was ephemeral, as it were.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  49. “And we started experimenting with different user experiences, right? Do we want it so that you, I don't know, right click and get a little side panel that comes up with a bunch of different options for things that you might want here? That was nice because it would give you like.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source

  50. “Like silver film that would be preserved for thousands of years in this Arctic Code Bowl. Well, we took that same data snapshot and we brought it to our friends over at OpenAI to see like, okay, what can we do with these large language models built on public code? Well, it turns out we can do some pretty cool things. Just like a translation tool that goes from English to Spanish, Spanish to German, you can also go from English to Python or Python to C sharp. You're like, okay, this is cool. We can start to get not only translation, but a little bit of predictive text here as well. And we're all, I think, fairly already familiar with predictive text already in our code editors, as IntelliSense, but in, I don't know, you go to your favorite word processor, and chances are that you've got some kind of predictive text happening there as well.”

    2022-09-04 · Lenny's Podcast · The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot) · IDENTIFIED FROM THE TRANSCRIPT · source