YouSaid · the spoken record

Thomas Dohmke

lines on the record: 91
first: 2024-08-19
most recent: 2024-08-19
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“They feel ready to contribute back, and I think Copeland will lower that barrier of entry. And, you know, one last thing is that I'm from Germany. I grew up with German as my first language. I learned Russian and then English. And we will probably always have an accent when speaking English. But most of most kids on this planet do not speak English at age six. There's a large population that does speak English, but a lot of them do not speak English, while open source and technology is predominantly in English. For them, the entry of the barrier of entry is going way down. And it will allow them to explore their creativity before learning a second language, before learning the fluency in that second language, before having the confidence of now I can type a feature request against the Linux kernel and say I want this. And here's actually the code I've already implemented. What do you think? And I think that's going to be completely changing the dynamic on this planet.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“And you want to make a change, your biggest challenge is going to be to figure out what are the places where I have to make those changes and how do I not piss off the maintainers by creating a pull request that is incomplete or that doesn't follow their coding standards or that doesn't follow the way they want to collaborate with each other because at the end of the day the open source comments are defining of how they want to collaborate and that's totally cool like every company defines their culture and every open source project defines their culture and the contributors that are coming in especially those that are early in career often have FOMO not FOMO anxieties in their heads of while I file my first pull requests and the reaction will be not oh this is so great Thomas that you sent that to us but this is a go back and learn how to code and then if once you have learned that and again this often not actually happening but I think most people have that anxiety in their heads and they're waiting forever and”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“I mean, I have chief product officer friends which are literally saying I found the fun back encoding with the help of co-pilot because it's not this, you know, Whether your CEO's chief product officer, or whatnot, most of your day is spent in email and meetings and customer calls and podcasts. And then when you have an hour on Sunday spending that in a productive way is incredibly hard because you have to get back to your environment, right? Like whether that's your building model train houses or whether that's code is equally similar because you have to prepare your workspace again with something like Copilot, it actually is much easier because you can just open your project where you left it and you ask copilot, how do I do this? And you don't have to start navigating all that complex world of open source libraries and models and whatnot. And so we are building the AI native developer workflow and we actually think this is going to be incredibly empowering for both developers working on their private project but also for open source maintainers. If you look at an open source project today,”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Know the dynamic today between engineering and product management, you often have like either overspecified or under specified issues. And then so the product manager then has to go back with the engineering team and say, well, that isn't actually what I wanted or the engineers go back with the product manager and say, well, this isn't specific enough. So having AI in that planning piece is already a win for both sides. In fact, we have seen product managers evening us and saying, well, now I can implement the thing myself. At least, you know, can try what that does to the code base and see how long it will take.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Or you made them because you were trying out how things work and whatnot, while Copilt, when it looks at the changes, it just plainly writes down what it sees. And as such, you get a very detailed write-up and you can obviously customize that to be shorter or longer. But it also describes to you stuff that you may inadvertently have changed. And so you're saving a lot of time by just avoiding the iteration later in the cycle. So we're bringing co-pilot into all parts of the developer workflow. We are looking into building what we call copilite workspace, the AI native development workflow, which is really cool because it allows you to take an idea and bring that into code with the help of a co-pilot. So it's not adding copility to your editor. It's inventing the whole developer workflow from scratch. And you write down our idea and it looks at that idea and the existing code base and it writes your plan. And you can look at that plan and says, well, that isn't actually what I wanted. And if you think about”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“If you look at where we've gone for the last year or so, it's like we have extended co-pilot into different parts of the developer lifecycle. We originally announced that as copiled X, you know, co-pilot coming to other parts of the workflow, not just autocompletion, not just chat, but actually bringing it into everything that the developers do because we believe there's a lot of value there. You know, very simple feature that we launched last year is summarizing the pull request. So when you have done all your changes to code and you submit that for review, you no longer have to write the description yourself. You can use copilot to write that description for you. Now you're saying, well, that's trivial, right? Like you can do that yourself. You're not saving that much time. But the truth is, you know, A, you're coming out of, let's say, a three-hour coding session and you have to write all the things up that you did during that time. A, you will have incredible confirmation bias of what you believe you did with us, what you actually did, right? Like because you're only remembering the changes that you thought are important and not the ones that you maybe exit.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Through a tuning process. And as such, now they have a customized version of that model that is combination of the public model, the one that OpenAI or Meta has released, but also their own data where the parameters were changed. Now, obviously, that model needs to be within the private tenant of that customer until unless the customer decides to make that model public through their own API common scenario that you can imagine is companies having their own programming languages, like SAP has ABA.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“You know, we have this line that every employee learns. Microsoft runs on trust. And we believe that if we lose that trust, earning it back is incredibly hard. And we have gone through moments in my career at Microsoft and certainly in Microsoft's 50 years where a lot of the trust was lost and it took a while to get back. And so as such, you know, I think the model providers themselves have enough data and will be finding ways to get access to the data without us sharing it with the multiple providers. Or it's certainly not without the approval of the customer. Now there's one caveat to this, which is somewhat orthogonal, but is easily intermingled with that question, which is there's an increasing demand of customers wanting to fine-tune a model based on their data, right? And what that means is that taking their source code in the GitHub scenario or other data in other scenarios and change the parameters of the model change the”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, it's important to get out of the tech bubble. And what is obvious to the people working on the product is often not obvious to the customers. And as the customer base is growing, more people ask those questions. I think it is incredibly important. In fact, you know, it's equally important as it was with the cloud or as it was with systems like Exchange and Gmail to say, hey, if you're deploying your application on our cloud, we're obviously not looking at your source code and using that source code to make other products better or sharing that source code with other people deploying on the cloud. And the same is true for models. People see these models as a compute layer. As such, they want to use that, send something computed and get it back and not implicitly giving anyone access to that data to make the model or the compute layer, if you will, better. And so I think that continues to be a cornerstone of Microsoft strategy.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Where they also see the benefits of AI, where they understand both our legal position, but also our, I'd say, human position of why we believe training the models on that code is the right thing for society. That's a complicated question. I'm not saying I have all the answers, but I can tell you that at GitHub, we have always been committed to work with the open source community, work with regulators, you know, fighting for the rights of open source maintainers with the European Commission. And ultimately, giving GitHub away for free for every open source project. And we're not asking the question, is it really open source or is it open weights or like it's public? It's public, but it's not an open source license, right? We're giving you free repo free issues, free actions, free models now with GitHub models, free code spaces. And we're engaging with the community with things like GitHub sponsors where an integration with Patreon and other things where we enable maintainers to build a creator.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Developer today, whether that's an open source or whether that's in a company is onboarding to a new project. Whether you are joining a team or they're just picking up somebody else's work to make a bug fix for that, navigating that code base is incredibly hard because you don't know what the person thought when they wrote all this, while the AI can somewhat reliably figure that out and help you navigate that code base and you reason with it together and you ask questions and it gives you a wrong answer that's okay too because the human paragrammer does that as well. So I think the value is proven. But that said, you know, and I think that's the second piece is that we do need to work as an industry with these people raising the concerns and working through those and figuring out what is the right model that the open source foundations, the open source maintainers, those that have been spending most of their private life on maintaining that small library that supports half the internet? How do we put them into a place?”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Of all, I think the outputs are definitely worth it already. We've seen significant productivity gains for developers. We have seen 55% as one such statistic from a case study that we did with 100 developers, 50 with and 50 without copilot and copy with copilot with 55% faster. We see some of the statistics from competitors and customers confirming that both in the short term and in long term, developers are seeing significant productivity gains even in the later part of the developer lifecycle in like successful builds and more deployments to the cloud from the team using Copil with the team without using Copiland. I think though more important is we see very clear feedback and surveys, our own surveys and customer surveys that developers are saying they're more happy, more satisfied, more fulfilled they no longer have to do all the repetitive tasks. I think that's where the dishwasher analogy works really well and it's easier for them to onboard to a new project if you think about one of the biggest challenges for”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“State actor perspective or bad actor perspective, that's a risk vector that nobody wants to take. There needs to be the human in the loop to make sure what is deployed is actually a secure code and not introducing vulnerabilities or viruses and whatnot. I think it's a question really of how big is the task where you can trust the LLM enough that it results in a”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“It requires We believe that at GitHub that the human will be in the center, that's why we call the thing copilot. And we believe there has to be a pilot. Now, that doesn't mean that the co-pilot doesn't fly the plane at times. They do, right? In real life, and there's going to be scenarios where a large language model is scoped enough in a task that it needs to do to fix, for example, security vulnerability. We have that already in public preview, what we call autofix, which takes a vulnerability and actually writes the fix for it. But then there's still that moment where now the pilot has to come back and say, yeah, that's actually the fix that I want to merge into my repository. And I don't think we're anywhere close of the pilot being replaced by AI too. And from a security perspective, in itself, that is also a risk that companies probably are not willing to manage anytime soon that AI and AI works together and merges code and pushes it into the cloud with no human involved. Because purely from national...”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Right, but I would just hear that comparison a lot, but I have to be honest with you, I never give a task to one of my colleagues at the Verge and assume that they will just make up bullshit at scale. Right? That's not how that goes, right? And with LMs. The thing that they do is hallucinate. And sometimes they hallucinate in the correct direction, and sometimes they don't. And it's unclear to me whether they are actually reasoning or just appearing to reason. Like there's a lot of things we want these systems to do. And I'm just curious if you think the technology can actually get to the end point because it requires them to be different than they are today in some meaningful way.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“What we're doing in workspace, which takes you from an idea to implementation without ever leaving GitHub. And the AI helps you at every step of the way. But you know what's important? There's like points in that flow where the human needs to come in and look at the plan and say, yeah, that's actually what I wanted. And, you know, often I like to think about that in the same way as I think about the relationships that we have with our co-workers and how often do you at the Verge give a task to somebody and then ask yourself how do specific do I have to get and how long do I want to go until I need to check in with them and see if they are on the path that I had in my head in my head”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Can do a limited set of tasks, and I think as you define those tasks in a very clear box of what that is, what you want the LM to achieve, like auto completion in Copilot as much as a scenario, they can be very successful, right? Like the reason that we started with the autocompletion was not that we didn't have the idea of chat and we didn't have the idea of explaining code or building an agent that does it all. It was that the model didn't do any of those scenarios at a sufficient rate. And developers are very, have very high expectations. And so if you deliver a product that serves 60% of scenarios, you're not going to be successful because your reputation is going to dive down really fast, whether it's on social media or on our own community forums and whatnot. And I think, you know, those scenarios have expanded over the last four years from auto completion to chat.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“It's easy to argue that Lama is a competitor to file three and GPT 4. We have Mistral in there with actually the latest Mistral large model as well. And so I think we are open to being the platform provider that is both competing and partnering with sometimes the same company.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“On and oft, or what is the, there has been winters and summers, I guess, in that relationship. But, you know, these days, my iPhone is full of Microsoft apps. And I'm having this podcast on a Mac and I use it Mac day in, day out. In fact, when I joined Microsoft in December 2014, Microsoft bought me a new Mac, my startup at Macs. And it was at the time already under Satya, very natural to say, well, if you want to work on a Mac and that makes you more productive, we're totally down on that. We're not forcing you to use a Windows PC. And I think that's now coming back to your question. I think that anything is possible as long as it aligns with both our strategy. Where do we want to go with GitHub? What products do we want to build? And the models launch is actually a perfect example, right? Like we do have Meta's model in there, which could be...”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“You know, it is against Microsoft's DNA to partner. I mean, I think the classic example is Apple, where they have been on and off.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“If I can make the business case at the end of the day, if I can make a successful business case where I can show that we can generate revenue, we have healthy gross margins and ultimately profit margins in the long run, I think anything is possible. I would say never say never, whether it's Google or AWS or any of the chip providers. I don't think there is a mantra that I couldn't do that. I think it's a much bigger question, can I do it in such a way that we're still achieving our business goals as GitHub and as Microsoft, right? And as such, you know, while I'm the CEO of GitHub, obviously I'm an executive at Microsoft and we need to have that one Microsoft thinking in the grand scheme of things to grow the overall business. We're all tied to the mothership, whether it's Ryan at LinkedIn and the game studios, Mustafa in AI or Thomas in GitHub. They're part of Microsoft. And we're working with Satya and the SLT very closely to make Microsoft successful. But I don't think”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“But the reason I'm asking is a limited integration company inside of Microsoft. GitHub did start as an independent. It has a different relationship to the developer ecosystem than even Azure does, right? Azure is a big, important part of the developer ecosystem, but Azure exists in a much more competitive environment than GitHub, which people think of as almost a utility, right? It's there. You can use it. Everyone uses it for everything, particularly open source community. It is a focal point of a lot of things. It doesn't seem to have the commercial aspect that something Azure might, but it's still a business. And sometimes its priorities and the needs of its users might run against Microsoft's desires. I'm just trying to suss out where that is and how you manage that tension.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Well, I'd say we haven't had that discussion because so far we haven't seen the business case for that, right? At the end of the day, we're running GitHub as a business that contributes to Microsoft's earnings reports and the overall success of the business. As I mentioned earlier, we're turning 50 next year and playing the infinite game. The big goal for Microsoft is...”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“I'll just give you an example. There's no way you're going to go use one of Google's models, right? To run Copy. Like, that seems totally out of bounds, unless it isn't, in which case that would be huge breaking news.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Very early on we identified that we need an Azure AI platform and so that team underscore Guthrie started building that in parallel to co-pilot and before we went and made co-pilot generally available in June 2022 we were already on that stack we were already benefiting from responsible AI. My team is doing red teaming and collaborating closely with Sarah Bird's team that runs the responsible AI team in the platform but we are mostly relying on their technology and we collaborate very closely and I think that's the new way of working with Microsoft that we have benefited greatly even though we are independent and limited integrated limitly integrated I don't know”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Early days of GitHub co pilot, there has been a monthly co pilot meeting with 100 plus people across Azure, across the Bing team, across Kevin Scott's CTO organization that have been in a loop of what we were doing in terms of building the co-pilot, deploying the co-pilot, commercializing the product, but also what they are doing and how we can leverage the stack. And I think the most fascinating thing is that I think all the co-pilots is the first time, at least in my time at Microsoft, where everybody from the early days started on a common stack, you know, the Azure AI platform or Azure AI services as it is sold to third parties. And so it's not like we build our own stack and Bing built their own stack and then somebody came and said, well, we should really, you know, standardize that on a new stack. And then everybody else in the future starts with that new stack, but all the old timers are like, well,”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“The first time I actually met him was at the TED conference in Vancouver because he had a talk and I had a talk and we literally ended to each other backstage. That was, I think, about a month after him joining was announced. And obviously the first couple of weeks in a large company like Microsoft are always stressful and many people want to meet. So I left him alone and we just ran into each other and shook hands and exchanged a couple of intro sentences. And then in the meantime, we met both in what is the senior leadership meeting under Satja at the SLT meeting every Friday, talking about mostly AI topics. And I've also met with him and his team to talk about actually similar questions that he asked earlier about, you know, how do we get more agile on models? How do we move faster on being flexible on the next model generation? What can we learn from the Microsoft co-pilot? Now, as you know, the GitHub co-pilot was the first one that we ever built. And as such, you know, there has been a continuous learning loop across all of Microsoft since the very”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Of that installed base helping us to grow market share and a tight feedback loop. But at the same time, every time we want to experiment, we have to balance between that experimentation and breaking things and keeping the current customer set happy, both actually on the technical side, but also on how do we invest the engineers and the product managers, the designers that we have into one or the other, not both. Or both, actually, or both.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Lines of code you highlighted, but it also looks at the context. And while if you copy and paste stuff into a generic shared agent, you have to collect that context yourself or give it to the tool in the pond. So I think it shows one of the weaknesses, which is like the world is moving fast. And so you have to be really, really agile. And we don't know what's the next big thing in AI in the same way that you would have had a hard time predicting in 1994 that Amazon would become the big tech company, the member of the Magnificent 7 that it is today. It took them a decade or so to actually turn the first profit. And then so, you know, it's hard, you know, in the tech sector to predict what's coming next. And especially in this AI race, I think our biggest weakness is that we have already a large product and market with a large installed base where then moving fast is a challenge in itself. We have the benefit.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“The biggest weakness for a pilot like Copilot that was early in this generative AI journey, we launched Copilot, we announced the first version of Copilot, the preview in June 2021. That was like a year and a half before ChatGPT came. And then we did GA in June 2022, still like, you know, almost half a year before ChatGPT. And then ChatGPT came and changed everything, right? And until that point, we thought Chat is not a scenario that works good enough for coding. Clearly we were wrong on that. And clearly, then quickly we moved to AdChat to the copil portfolio and make that great for developer scenarios within the IDE, within the editor, because it allows people to have all the context that's available, right? The power of copilot always has been that it knows what's in your file. And so when it suggests code, it actually has the variable names and it knows what open source frameworks are using. It actually looks at adjacent tabs. And so when you ask questions to explain code, it not only looks at the”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“His running co pilot for you is that a cost center or is that actual profitable? Because that's really the conversation across all of AI right now. Are we actually using this to make products and make money?”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“The earnings call script that in the last year, 40% of the revenue growth came from co-pilot and the run rate is now 2 billion and run rate obviously is forward-looking, right? So those are a bit of different metrics. We're really happy about the co-pilot growth and where this is going. And Satya keeps sharing the number of organizations that have adopted CopyThe. And I think what has been remarkable is that it's not only the cloud native companies, the startups, the Silicon Valley core that have adopted Copil is really the largest companies around the world. The traditional industries.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Us on this planet. Hopefully, they also all have a GitHub account, but more so, the goal is to enable about 10 or so by the time we achieve that goal, probably 10% of the population to start coding as they can start learning to draw an image or start playing the guitar or any literacy is 100% hopefully or aspiration as humans. And I think coding should go in the same direction. Everybody should be able to code and explore their creativity. And so coming back to your Microsoft question.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“She reports into Scott Guthrie, who runs the Cloud and AI division. And of course, that way we are very much aligned with what the cloud is doing and also what the Azure AI platform team is doing where we partner with on this GitHub models launch that we talked about earlier. But as a CEO of GitHub, I have a leadership team across the whole range of function of an engineering leader and a product leader, a CEO and a chief people officer and chief finance officer like who am I missing, a chief of staff. And we are working together as a company and not as a functional Microsoft organization. And so as such, I'm operating much closer to a CEO than a typical Microsoft engineer leader. And I think that's a lot of fun that gives me a lot of energy and it gives me a lot of motivation as we can fully focus on GitHub and making GitHub bigger, our goal, our winning aspiration is to get to 1 billion developers.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“I'm coming up on 10 years at Microsoft in December, which I wouldn't have believed when I started at Microsoft, given that I came through a small acquisition myself at a small startup called Hockey app that I got acquired in late 2014. I joined GitHub six years ago and then became the CEO three years ago. And today, GitHub is very much structured within Microsoft as it was when we acquired it in 2018. I was actually on the deal team working with Nan Friedman and others of getting the deal done and then joining GitHub that way. We are a limited integration company, as Microsoft calls it. So we have adopted some of the Microsoft processes. Our employees get stock grants for Microsoft and they invest that stock very similar to Microsoft employees. My manager is the president of the developer division, Julia Liusen, who also has all the Microsoft developer tools, like Visual Studio Code and Visual Studio.net, some of the Azure services that are near to developer workflows like Redis and API management and whatnot.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“I just want to point out you started with 5% sounds pretty good and you ended with 50% less GPUs. So it feels like the numbers are maybe a little bit higher than 5%. You mentioned Microsoft earlier. I feel like I'm way late to the Dakota questions on this episode. We've immediately got into the weeds. But GitHub is part of Microsoft. The acquisition was made several years ago. You're a new CEO of GitHub within Microsoft. You were at Microsoft before. How is that structured now? How does GitHub work inside of Microsoft?”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“And in the model world, you know, that still plays a role, right? Because your model has to have 99.99 whatever uptime, but also the model quality, the response quality, if that dips, you also have to monitor that. And you almost have to run through the exact same process with your site engineering team, site reliability engineering team to say, okay, something is going wrong, what is it, and when the stack maybe did an operating system update, you know, patch Tuesday or something like that, maybe a network router changes. Oftentimes when we now deploy GitHub in a new data center, the big question is, can the network bandwidth actually support our load given the scale of GitHub as a social network? And so all these things play a role now, not only in model uptime, but also in model output, right? And that's, I think, where all these questions come into play before we make the decision of saying, well, okay, we're ready to move through the latest GPT model or the competing model.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Evaluation is matched in the online evaluation where people work with real code and not with synthetic scenarios. The way I like to think about these in web services ever since the cloud became a thing 99.9 or more in terms of uptime percentage is the gold standard, right? Like everything less than that and you're going to be on hacker news or on the verge all the time saying your system that a startup XYZ or big company XYZ is down again and is preventing everybody from getting to work, right? And we have seen that both with GitHub and with other collaboration tools like Slack or Teams and others. Like if Slack is down on a Monday morning, everybody is like, well, I guess I'm off work today.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“We're saving half the GPUs, and that seems like a pretty good deal that I would certainly go into a deeper evaluation process and try to figure out is it worthy to now go into the two handful of regions where we have deployed the model? Because we are running in different Azure regions with clusters of GPUs to just have low latency. So European compiled user is connecting to GPU cluster and France and Switzerland and UK and Sweden, I think. If they're in Asia, they're having all GPU cluster in Japan, but then India is probably closer to the European cluster. So they're going that way around the world. And then we have different ones in the US and we're expanding almost every month to a new region to get more scale. And so switching to model has switching costs across all these clusters. And then we're coming back to the ABC testing question of how do you do that? So you actually have enough confidence that the off.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“I think the nuance there is that we are checking in offline evil for C and C++ and C sharp and JavaScript and TypeScript and Python and Ruby and Go and Rust. So far I haven't seen a model update even within the GPT line where all the languages across the board are better off the start. Some are better and some are worse. We're looking at different types of metrics. Obviously successful build is one of them. Like is actually the code building in the test suite, but also like how many lines of code did you get compared to the previous model or the competing model? And if that number of lines goes down, the question then becomes, well, is that better? And is it using a smarter way of writing that same code or an open source library? Or did it get worse? And it kind of like, well, it's the builds. It doesn't actually create the right output anymore. And so I'd say, you know, if somebody comes to me, you know, one of my engineers or data centers is like, this model has everything better across the board.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“If one of your engineers came to you with an argument to switch to another model, what would the winning argument be? Would it be 5% more efficient, 10% less cost? Where would the metric be where you'd say, okay, it's time to switch?”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“The costly part is evaluation test suite and the meta prompt or the system prompt. And you can kind of imagine and co-pilot as it sits in the editor, there's a lot of these system prompts for different scenarios, like there's different system palms for summarizing a pull request where there's one that autocompletes text or one that helps you with debugging an error, which Copala does in that sort of these suites of prompts that very specific today for different models and actually think”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“You mentioned your thing about models as commodities like chips, like AMD chips versus armed chips or whatever. Have you architected your various systems so that if you wanted to make a big model switch to Mistral or something, you could. Would that be very costly to do? Would it be easy to do?”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Saying no is much more important than saying yes. And especially in this age of AI, it meant that while we invested in all these AI topics like Copiled Workspace and models, we also made the conscious decision on leaving things behind that we won't invest into anytime soon.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“No, you know, the reality is most decisions you don't get to do an AB test, right? Your life doesn't have AB tests. You know, the price point that we set for co-pilot, we are kind of like stuck with that until we make a decision to change it. But you don't really want to sell at $19 to some set of customers and a different price point to other customers that minus discounting, obviously, but that doesn't really work. And so the same is true when we made the decision to launch co-pilot and then put considerable resources within the company onto Copilot, it also meant we remove funding from other projects that we codes also have done, right? Like the reality is in any company or resource constraint, even the largest companies, in fact, you know, I think the largest company's biggest weakness is that they are so big they think they can do everything. And the truth is they're still resource constraint. And so they still have to say no way more often than they can say yes. And I think that's the thing that I remind myself, you know, almost every day that”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“But many other decisions are just a question of intuition. Like the tagline for our conference, that's certainly a discussion, but then we decide on that taste and intuition.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“You know what I'd like to tell my leadership team is give me options and give me like a set of choices I can make and tell them tell me what are the pros and cons but also then you know and that maybe is a bit of my German DNA I often ask the questions what about the options that are not here like what are you not telling me what are we missing what what am I not seeing on on these options I think it's it's actually more important to think about what's not presented here and why are we not even discussing this were those just picking between a and b and you know last year i'd say you know let's be real many ceos and and many leaders leverage experience or intuition to make decisions many small decisions can be just made without a document without you know data like i'd love to be data driven and love to look at data especially when it comes to things like determining pricing or determining you know model updates as we talked earlier and whether 5 is enough or not”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Have a veto, or I hope at least I do have one. Definitely I have a vote. But honestly, I also often don't want to have a voice. I'd like to view things because I'm interested to just browse through GitHub issues and GitHub discussions where the company is discussing things. But when engineers are talking about the ups and downs of using React as an example, I'd love to read all that stuff because it helps me to understand what's happening and also turn it out to a certain degree. But I don't need to raise my voice or even have a vote on that. I have a strong engineering leadership team and a strong set of distinguished engineers and principal engineers that can make those decisions and will be accountable for them within the DI framework.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“Two frameworks that we closely follow at GitHub. One is what we call the DRI directly responsible individual. And then often we ask the first question is who's the DRI? And if we don't have one, we should have one. And so we have one person in the company that runs the project. And if we need to have a decision to be made, ideally DI can make the decision by consulting all the stakeholders or they can bring the decision to the leadership team and me to discuss Volo, the ideas and where we want to go. The other framework I like is view voice vote veto, basically deciding who in the group has actually what rights in the discussion, can they have a view? Can they have a voice? Do they have a vote or do they have a veto? And in different decisions to be made, you have difference of these roles. And obviously within the large framework of Microsoft, I often have a voice in the framework of GitHub.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT
“That you wanted to type. And so we're using a fast small model, a small-ish model in auto completion. In chat, we have a mix of four Turbo and actually 4.0 has rolled out in the meantime. Then for newer scenarios like copilot workspace, we have been on 40 for a while and we have compared 4.0 to other models to see where do we get the best returns in terms of code rendered and changes made to the code base to solve the problem that Copy Workspace tries to solve. So we're comparing within the same model generation newer releases that we're getting from OpenAI and we're also comparing these models against other open weights, open source and private models that are accessible to us through Azure.”
2024-08-19 · Decoder with Nilay Patel · GitHub CEO Thomas Dohmke says the AI industry needs competition to thrive · IDENTIFIED FROM THE TRANSCRIPT