YouSaid · the spoken record

Lukas Biewald

lines on the record: 48
first: 2023-08-03
most recent: 2023-08-03
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“No, this has been fun. I mean, I just think the message that I'm trying to tell the world is that we're really trying to make tools for this new LLM workflow that people are calling LLM ops. And so my advertisement for weights and biases is like, hey, if you knew us and liked us for our ML ops stuff, try our LLM op stuff called prompts. I think it's not amazing yet, but I think it's kind of ahead of the market and it's about to get a lot better because we are like investing every resource that we have into making as good as possible. And we're really listening to feedback and iterating. So if people want to email me directly and tell me some issue they had with prompts, I really want to hear it.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“So, I don't know. I have this feeling that nobody does enough of that, but I don't really know. I think people are lying to each other about how much actual kind of customer meetings they're doing. And then it's like, you know, when you get to a customer, it's so precious. It's just like, man, like show up prepared and like ask the tough questions. Like I think like I feel like one thing about me is like I always like default to like wanting people to like me and it's a terrible trait in a in a CEO. You know, it's like I felt like I've always like coping mechanisms for myself to like not just like kind of flip into that mode. But I think it's good for customer discovery because I'm always like so afraid that they secretly hate my product, you know, that I get like really insecure and I'm just like, okay, like, you know, tell me more, you know, like, like, are you sure this is really like working for you? It actually does actually help in that one important entrepreneurial process. So lean into your insecurities with your early customers.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, you know, the advice I always give is like the generic advice that everyone says it's like even truer than you think. It's even truer than like I know even though I like deeply believe it. So it's like caring about like if you're making something people want like everybody knows it, but like no one cares about enough, right? Like people just they get distracted. They do other weird stuff. Even I do it, I understand. But like you should care more than you think no matter how much you think. I've never met anyone that cared too much about that. And then spending time with customers, it's like, it's so critical. Everyone says they do it, but I don't really believe it. Like, I feel like I'm obsessed with this. I mean, like, getting like when you're an early company, getting like three customer calls in a week, that's like tough, man. I mean, you got to like scrape and claw and like beg to get those meetings and you know like two of them are going to like cancel. So I don't know. People tell me, oh, I met with like 30 customers this week or something. It's like”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“I think I've just been a more confident person in myself. Like anytime I start thinking, okay, long term versus short-term, it's just like you always want to think long term. Like everybody wants you to think short term. Like everyone's going to push you to think short term. They wouldn't say it like that, but it's like, you know, it's like people can see like ARR growth. They can see user growth. It's harder to see product quality, right? And so I think I'm a competitive guy who likes metrics and likes accountability. But I actually think that can get counterproductive for me where you start like sacrificing short-term things to grow these external facing metrics. And I just really try to fight that myself. I think everybody like chases, every entrepreneur chases like short-term, like ARR numbers like in quarter. But then it hurts your growth rate the next quarter. It's like it would actually be better always to like push out deals. But nobody thinks like that, right? You can't think like that.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I mean, I think one thing was extreme clarity about who we were serving. So I'm surprised I don't hear this more because the ways that biases started with a customer profile. And I think it's actually a nice way to start a company because especially as a founder, you have to spend so much time with your customers. You have to seek them out. Like picking a customer that you love, I think is a really good thing for your mental health. And so that was like a big thing. And then I think like.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Know, I'll be honest, actually. I'll be like totally honest. I find it incredibly stressful because I still feel bad that we lost the scale. It's just lingered with me. And I admire scale. Actually, I know hard that business is. So I have just like deep admiration for their execution. But as a competitive guy, I kind of can't get over it. So I've always inundated with questions from VCs. Like whenever any annotation company's raising, I know about it because everyone calls me. But I honestly try I know I should be closer to it, but I try to stay away from it just because it caused me so much anxiety to look at what's going on that I just can't deal with it.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“But I don't know, like ML researchers aren't so precious in my experience generally. They kind of want to get a job done. And I think they're kind of happy to that we have a stable business that generates money in a normal way and isn't going anywhere. Or at least that's what I tell myself”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“It's had a pro. There's been a major pro, which is that all our competitors are open source. And what that means is that they don't get to see how users actually use their software. And so I think our software is a lot more ergonomic because we have like metrics on what people actually click on if people aren't clicking on a button, we remove it, if people pick an option all the time, then we know to make that the standard option. As we've grown and you kind of can't just rely on anecdotal user feedback, that I think has made our product a lot better. Like people find it nicer to use. At the same time, I understand why people want to go to open source stuff, but honestly, I feel like it's a little bit of a DevOps mindset also. I mean, DevOps people, like they're obsessed with, you know, open source. And usually like the MLOps people we talk to in companies really want like an open source piece, which is why our client is open source. Things actually runs in your servers is open source.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Funny, like, I think the tools thing I've always felt like I've always felt like kind of proud of making tools for developers, like that's always felt really good because I think developers sort of know what quality is. I kind of like making a tool for someone that could make the tool themselves because it kind of raises the bar. And this definitely, my grandfather was like a pattern maker, which is like a sort of, you know, like the person makes a pattern for other machinists. And he had the same attitude of like, look, I'm making this stuff for other engineers. And like, there's like an honor in that. So I definitely feel that pressure and love it. The open source closest thing was really just like we didn't know how to make an open source business. So we kind of started off closed source because we just, we actually wanted to have like a working business.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Barely, but you know, if I had to pick one end of that spectrum. And my co founder, Chris, probably excuses more thread software developer. And Sean's probably somewhere in between.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“And so you get all these companies that come out of like every MLOps team then realizes they could raise like a shitload of funding. And so like you got like every major company, their MLOps team went off and like raised money to like make a new product in the market, which I think from an investor, that's logical, right? It's probably they have a good thing, but they're just like not good at connecting with actual developers, right? Because they're actually like DevOps is like a little bit of a different discipline where you're sort of obsessed with the reliability. Kubernetes seems like simple to you. And that's just not like the experience of like an ordinary developer. Like, you know, like my co-founders or me. And so I think the joy of weights and biases is we're kind of making software for ourselves. And I think it turned out that maybe in the median of my three co-founders was actually the target audience for us here. I think I scheme more towards an ML researcher.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“And these chips worked for deep learning. It just broke the entire stack. It was like the first time in my career where I'm running into linker errors. I'm like, what the fuck is a linker? Like I vaguely remember this from, you know, like a CS class I took, you know, like, and so it's like, I think that ML researchers really had to kind of become software developers. And then at the same time, the AI class is the most popular class. All these software developers and smart ones kind of become ML researchers. I think that line has weirdly blurred. But then I think there's a funny thing that also has been happening where every DevOps person on the planet rebranded themselves as an MLOps person all of a sudden.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“It's like developers and researchers that kind of blend together. But I think that what happened in the sort of MLOP space is that you got a lot of, well, the early companies had to sell to executives, which I totally understand. Like that's what Crowdflower had to do. And the problem there is you kind of get stuck in these like multimillion dollar deals and like you just can't get out of that. Like you can't switch to like a PLG motion. And so the early companies, I think, are kind of stuck, right? With like these products that CIOs love and the, you know, engineers hate. And that's just like, I just didn't want to do that with weights and biases, no matter how big the market is or how juicy that is. And the good news is it's not a good market, like a developer-oriented sales better. When you look at like developers versus ML researchers, that line has really blurred in the time that we've been doing it. And I think that there's sort of like subtle differences. But, you know, when NVIDIA came along,”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Like software, right? So it's like it's not like a machine, you just like press copy and then you have more of it. And so, yeah, I mean, we see that. We see like a lot of, you know, I mean, FinTech probably obvious to you guys, but like there are kind of, I think, always out in the forefront, you know, of this stuff for lots. I mean, like, there's like consumer oriented stuff that you'd recognize, like, you know, making chatbots not annoying, right? And then there's like, you know, kind of more, you know, financial forecasting and things like that. But yeah, I mean, it's funny. We don't do any vertical-based marketing because there's not one vertical that's like dominant enough to warrant it. And our customers bounce around between verticals so much that I think the common thread here is people doing like ML and data science versus any particular application, which I just do is super cool. That means it's sort of like table stakes for everyone.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Like a lot of the reasons you spray a whole field with pesticides is just because it's like so expensive to do something smarter. And so, you know, I think that crop yields and the cleanness of the farming practice is about to dramatically improve. We worked with John Deere for years back from a figure eight days to weights and biases.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Oh, yeah, it's so cool. I mean, the coolest thing about running weights and biases is the customer set is everyone. I really think every Fortune 500 company is doing something with ML that they actually really care about. And it's always surprising, right? Like we work with, you know, most of the big game companies. Like, I'm not a big gamer. So I'm vaguely aware of riot games and Unity and stuff. But, you know, but they do all this cool stuff with ML to like, you know, make the games more fun, to make, you know, models in the games. And this is like big investments. They've really, really care about because, you know, again, like we're sort of the last step in your journey as to want good tooling for your ML team. You kind of need something to work. So you hire an ML team. You get into production. Then you like run into problems. Then you come to waste and biases. So like we see stuff, you know, after it works. And like, you know, like ag tech, like, you know, big agricultural companies. I had like never heard of some of them when they showed up. And then there's like these huge, you know, businesses that are actually using ML to find ways to do like clean.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Hundreds of people, right? Like, you know, companies will hire a few people for an experiment, but they're all gearing up to operationalize this stuff. And that just gets me really excited. I mean, they could all be wrong, I suppose. And I don't really have any insider knowledge except for the seats that get bought on, you know, ways and biases. But when I see that, I get pumped because I just like, you know, the drugs that they're working on, you know, the diseases that they're curing, it's like the ones that our relatives have, right? Like, you know, Alzheimer's and Parkinson's and these are kind of horrible things. And I think there's just a huge promise in being able to do physics like inside a computer versus in the world.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I mean, the one that stands out for me because this is the one that's really different than my figure eight days is pharma. So I actually think this is kind of flying under the radar a little bit, but every pharma company is making major investments in ML and not just on the sort of like, I mean, they do have these operations to sort of like sell more drugs to doctors that uses sort of like light ML. But I think the thing that's really exciting is the actual testing of drugs before they have to test them in the physical world. And that's obviously working super well. And I think I see this before too. It's like autonomous vehicles and stuff. It's like there's a big lag there, right? Before you get something through all the clinical trials. So no drug developed by ML has gone through clinical trials. But if you look at the behavior of all of the big pharma companies, I can tell that it's working because they're hiring.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Oh, totally. And there's tons of product issues too, right? Like, you know, like Notion and Zapier both have these really compelling demos and they're both products that I use, but then I actually don't use the LLM piece of them myself. And I wonder, I have no insider knowledge of the level of adoption, but I think they're, I think they haven't gotten it perfectly right yet, despite a lot of thinking and really smart people working on it”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Fact they talk about it like constantly like constantly like everyone's talking about it but in enterprises like boy I don't know if I've like used a product of like any enterprise that actually like was backed by a NLM and there's a bunch of things that make it hard it's like you know it's kind of unfair because this stuff has only been out for like six months or so but it is like I think the adoption may be maybe take a little longer the short term than people think”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Like desperately trying to find them because these are our customers. Our stuff is just like our ethos is like we want to help people do things in production. So it's like if you're not in production, we're not relevant to you. So I like, I mean, back in January, February this year, we were looking for design partners that had stuff in production. And boy, was it hard to find, right? Like, you know, now there are more. But even when you, you know, you find people that are sort of like claiming to have those things in production, it's sort of like, well, it's like, you know, it's coming. We have like all these like sort of like prototypes, you know, running. And so I think it'll change. I think it's changing quickly, but I think it's a funny moment where, I mean, I think if you actually looked at the TAM today of tooling for like, oh, I was like, I don't know. I bet you it's small. And I think also, I think VCs maybe sometimes have this funny window where you see all the companies that are using LLMs. But the enterprise adoption has been slower. I mean, despite”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Kind of takes them a while, so I'd rather sort of support what looks like the rational workflow. I mean, I think the insane thing must be crazier to be an investor in this world is like very, very few people have LLMs in production. Like there's probably more companies that have raised money as like LLM tools than companies that have LLMs in production, which is insane. It's just like an insanely saturated tools market with very few people getting things out. But it's because...”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“It's funny. I feel like lately what I've been telling people is I'm just trying to see the world clearly as it is today. I can't predict the future and I can barely keep track of what people are doing today when I consider it like my full-time job. So I'm scared to prognosticate what might be coming. But I think you're right that that's what's happening now. I think there are a bunch of things that could change, right? Like I think like, you know, GPT is way far out ahead and it's hard to fine-tune it, not even possible with GPT 4. And I think that that is like a little, that's not like a technical limitation. I think it's sort of like a business model in a limitation. So that might change. I think that there's a lot of hidden costs to running your own model. I think people are really enamored with the idea of running their own model. And I've kind of seen this before where I think at the end people do rational things, but they.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“And we pushed out prompts. This is really my co founders Sean had really put a lot of effort into making our stuff really flexible because he's like, you know what, Lucas, there's going to be changes coming. We don't know exactly what they are. But like, you know, kind of from the beginning, we really tried to build very flexible infrastructure. So this was kind of a moment where we could really sort of like flex that and get out a product for monitoring and stuff. And, you know, now it's like, you know, kind of, it's our number one priority is getting out more tools for this new workflow.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Existential threat here. And I think they were like, hey, you know, we don't see it in the data. Are you sure? Maybe you're being paranoid. And I guess I do feel sure. And I don't want to say I'm the only one or like pay myself as the hero. My co-founder is also seeing this and people talking about it. But it's sort of like, you know, this threat is like now, right? And we have to actually get the whole company to do this thing because it doesn't show up in any of our metrics yet. I just really believe that, you know, our customers are rational and they're going to do a thing that makes sense for them. And so I see a lot of my colleagues being like, oh, there's going to be like lots of different models. It's like nice if it were true. But like what I see everyone doing right now on July 27th is using GPT. Like I see like 95% of the people out there using GPT for these ML tasks. And so it's like, look, we got to support that. And so we really rallied the whole company behind it.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Well, it's really hard, right? I mean, so what happened was we have a great business that makes an ML set of ML tools for training models. And we actually helped most of the LLMs out there were built using weights and biases. And then we started to see, like, wait a second, some of these ML tasks you could just ask the LLM, right? So instead of doing like a sentiment analysis model, you could just be like, hey, is this document positive or negative sentiment? Like for structuring documents, you could just be like, hey, find all the names in this document. And it actually works super well. And a little piece of me is a little bit sad about that because we have this like great simple, relaxing business that grows revenue every month that I always dreamed of, right? So, you know, part of me is like, shit, this is actually our kind of first real existential threat, I think, you know, and, you know, I went to my like leadership team and I went to my board and I was like, I think there's like a”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“So I just need my stuff to be simple. That's kind of the mindset behind the companies let's make these kind of simple clear things that actually help people.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“That you end up like with research is always the research code bleeds into production, and every company. So I think a better way is to give researchers and ML people tools to just make their stuff more reliable. And it has to be simpler maybe, or it's just a slightly different audience. Like you can't just give someone like Docker. You can't just like, you could, I mean, a lot of people are like, hey, why don't you use like the get large file system stuff to version your data? And like there actually are some reasons it doesn't work well with like object stores. So there's some like ergonomics reasons, but it's also just like, man, Git is like complicated. I'm like willing to use it for code, but if you start making me like version of my data with Git, like I just want to like cry, you know what I mean? So like give me something like simple, you know what I mean where I don't have to like think about it or I'm just going to start like renaming my data sets like latest latest really latest really for sure June 27th.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I just don't know. I don't know it. And it's like my co founders just find it baffling that I wouldn't understand it. But I think it's. Know it's like they're like, wow, this guy needs some basic tools, you know, like, because you know, they're like, okay, like reproducibility, like, why don't you just use Docker? I think that's sort of the ops mindset. But I'm like, man, I don't understand Docker. Guys, I feel like I installed on my laptop. And then it's always like taking up memory and stuff. I like, I don't really know what it's doing. And I'm kind of scared of it. And like, I don't know. So it's like, I just feel like it's adding weird complexity to understand. And so I think the tool kind of exist in a way, but they just weren't made in a way that like ML people could really use them because like, you know, if you're like me, you kind of come from a mathy background or like a research background, you kind of didn't really learn to do like industrial style coding. And so, you know, I think companies have this idea that the researchers are just going to throw the thing over the fence and then it's going to be in production. But it doesn't really work, actually. I think that's a bad pattern that people sort of like imagine they're going to do and they don't ever really do.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“He's like a really good engineer, and I'm actually a really bad engineer. I'm really lazy and try to write the people my co-founders make fun of me all the time for like, you don't really know how Git works. And I just openly, I have no idea how Git works. I just sort of mash the keyboard until I kind of like, you know, get in a bed state and then I call Chris and beg him to like”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“That hadn't adapted to machine learning at the time because rule based systems were kind of all the rage when a different generation was growing up. And I was like, wow, you know, I am actually getting out of date myself. Like I'm saying these kind of wrong things that retrue 10 years ago and are not true now. And I honestly felt like really bad about myself. And so I did a couple projects to try to, you know, get up to speed. I started teaching free machine learning classes and deep learning classes to kind of force myself to learn the material. And I actually like interned briefly at OpenAI where I was just like, look, I will just do whatever work you want. I want to be like, I need, I know that I need an accountability partner, essentially, to force me to learn stuff, even though I love to learn stuff. It's like my favorite thing, but I always need accountability partners for anything I do. So I sort of use the students as an accountability partner and open AI. And then what was happening was I was showing my old co-founder, Chris, like all the cool stuff.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Sure. Yeah, so it's kind of constantly evolving, right? Because we're saying it's like a set of tools for people doing machine learning. We're best known for our first thing that does experiment tracking, which keeps track of how your models perform over time as they learn and train. We also have a lot of stuff around kind of data versioning, data lineage, you know, production monitoring, model registry, kind of the sort of end-to-end stuff that you'd need to do machine learning reliably. And I think the thing that happened to me was I had been running Crowdflower for years and I always loved machine learning, but I was really starting to get out of date. Like deep learning came along. And at first I was kind of skeptical of it because people are always saying, oh, I have a better model that's like magically better. And they're like wrong, wrong, wrong, wrong, wrong. It's just really like data. But then they were right, right? So there actually was a sort of a better modeling approach that worked. And I kind of realized, you know, when I was in my early 20s, I was really judgmental of, you know, the people in their late 30s.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Just like skyrocketing and revenue is like, oh man, like, I wish we had just maybe held on a little bit longer. But then it gave me the space to start weights and biases. So, you know, who knows? I want to be like Daphne Culler and evaluate my decisions accurately and critically, but it also does seem like, you know, I've had some good luck along the way.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Because there was no chasm to cross, right? There was like nowhere else to go. So we tried all these different things to build more complete solutions for our customers. And it just didn't work. And then kind of all of a sudden, you know, autonomous vehicles got popular. And that really actually suddenly caused our revenue to, you know, start to grow really fast again. But it was like an eight-year lull of like, you know, really no growth, right? So it's hard because we start off fast, got everyone really excited, you know, kind of got like whomped for just like years and years and years. Actually, we had all these competitors. They all went away. So at some point, we had like no competitors left, right? Because like everyone had gone out of business. And then it was a funny experience because scale came along and totally ate our lunch in the self-driving market, which is a market like I knew and loved. And so, you know, I was so excited to sell the company after, you know, so many years of struggle, you know, but then like right after that, we see like scale.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Not because he wanted to buy anything, but he just thought it was awesome. And so I'm just like, you know, I pick up my phone. And then there's just this guy in the other and just be like, oh man, like this is so cool. You know, I'm like, okay, like, who are you? You know, it's like, do you want to like get coffee? And that actually turned out to be incredibly helpful. But then I think the thing that was so different back then is that the people doing ML, there just weren't that many. There were people heavily investing in ML, but it wasn't that many. And so what happened was, you know, we got eBay as a customer, which has really mattered at the time. And we got, you know, Google is a customer and Bloomberg. And then there just wasn't anywhere else to go. So my board was always recommending reed crossing the chasm. And we tried like a million different ways to like, you know, grow the company. And, you know, I don't know. I hope this doesn't sound defensive. I mean, maybe I was just a bad CEO, but we had like years of strong.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Well, I mean, not only was AI not popular, but like startups weren't popular, right? Like my family didn't, you know, understand about startups. And I had graduated Stanford. You'd think I'd have all these great connections, but it didn't feel like that. I had no one who knew how to raise money from VCs. I didn't know any, you know, VCs or I didn't really know any entrepreneurs, honestly. And we had this website for Dolores Labs in the early days just trying to get customers. And it put my personal phone number. I actually remember I was like the first user of Twilio because I needed to make a phone tree. And so I used Twilio software. And then like all three of the founders came to my house to like help me make that phone tree work better, which is kind of amazing. It was like, you know, like, you know, one of those 20-something grunge apartments in the mission. And then Travis called in. But, you know, it's funny because the phone tree, we were just trying to pretend like we were a big company. And Travis called in because the phone numbers on the website.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“It was funny, right? Because back then I was coached actually quite a lot by Travis Callendick, who's famous now for doing Uber and other things. But he was like, don't tell anyone that it's like AI, like VCs, like, don't want to hear AI, which is actually good advice at the time and it's good advice in the early days of the company.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Oh, wasn't it a vault? And it'd be like, you'd make like a 50 page document. And like, you know that the people doing the labeling are not reading that document, but you kind of need that to cover your ass if they labeled something, you know, not the way you want. And it would have been so much better to be like, look, we're trying to write search results, like put yourself in the mindset of someone, you know, who's looking at this. Like, is it good or bad versus trying to lay out in excruciating detail what makes something relevant or not relevant?”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“And really get like visibility into it. Because, you know, at the time, I think the thinking was like, oh, this is sort of like a manual task. That's more of like an operations team should deal with this. And they would like to do this thing where you'd like, Make this giant requirements document. It was so like waterfall. Like it would be like”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“An ML model. And I was sort of the messenger here. So sometimes it would work and sometimes it wouldn't. And so people either really happy with me when it did work or they'd be really pissed at me when it didn't work. But I kind of realized actually the model that I'm building is like the same for each country. The training data though is different. So some countries would take the training data collection process really seriously and they'd get a great model and some would just like really half asset or like, you know, have these crazy issues in the data collection. And then the model wouldn't work. And so I just really kind of viscerally felt how much the training data process mattered. And I kind of felt like, you know, why don't they let me get involved in the training data process? Like that would be a better use of my time than building these models. And so I wanted to make a company where the people doing the ML could actually have control over the training data collection process.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Know, I was so frustrated by that that I just really wanted to work on something that people cared about. I actually turned down an offer from Google because they didn't tell me what I would be working on to go to Yahoo because they were like, okay, you can work on search rank ranking in different languages. But that actually turned out to be incredibly fun, right? Because it was super applied. It's actually a task that works really well. And Yahoo is kind of in the infancy of switching from hand-toon weights to machine learned weights. And they really had no one. Not many people actually like working on deploying this stuff. So I was like writing code to translate machine learning algorithms into C code and then check it. We would check it into our little code base and run this kind of like semi-hand generated C code in production. So that was super fun. But the thing I learned there actually, which I think I'm not the only one that learned this, but I just felt it. I would go from like country to country trying to switch from hand tune weights to”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Curious. Oh, you are? Oh, me too. Yeah, I love Go. Yeah, Daphne was very not interested. She really was practical. And so I worked on a task that you really don't do now called word sense disambiguation, where you're trying to find out like, okay, I have the word plant actually, if you look in most corpuses because they're government generated often at the time, plant typically will mean like the power plant sense of plant or cabinet often means the sort of president's cabinet sense of cabinet. And so you're kind of trying to figure out like what is the meaning here of these words and then apply it to translation. It's a cool task. I mean, and actually it turns out I think that these, again, nobody kill me, but my general sense is that these sort of like linguistic oriented strategies really don't work that well. It's kind of like by feeding more data in and sort of like working on outcomes, you can figure these things out much better. So a little bit of a dead end. And actually.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“No, Daphne is not interested in games, let me tell you. And it's actually another, I kind of admire that perspective too as much as I love games.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“It felt honestly kind of pointless and sad. Like, I love the idea of computers learning to do things, but it's hard to sort of sustain the enthusiasm for that when everything you try just completely doesn't work. And even the things that do work, you kind of wonder if you're like p-value hacking. Like, okay, I tried a thousand things, you know. So I guess something's going to be like a little bit more accurate than a baseline.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, you know, it's funny. I actually really struggled doing research with Daphne. Basically, the things that I tried just barely, barely worked. Like, you know, I published a couple papers that I feel kind of ashamed of where it was sort of like go from like 68% accuracy to 70% accuracy in a task nobody cares about by throwing like a thousand x to compute And by the way, like kind of guessing the most likely answer is probably like 64% accuracy. So, you know, it just”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“I think her personality has mellowed a little bit over time, but I kind of miss, I just miss that sort of aggressive, clear thinking, and I really admire it.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“At the time was the thing that was making them work. And I think later became clear that machine learning was a big part of that. But really, when I was doing ML, it was like searching for applications that were working. And Daphne was actually really obsessed at the time with a thing called BayesNets, which you don't hear about too much anymore because I don't think they ever really worked for many applications. I hope I'm not offending anyone, but that's my understanding. I actually think the thing that I really took away from Daphne that really lasted with me was I mean, she just one of the smartest people I've ever encountered and she had this incredible clarity of thought and an intolerance for sloppy thinking that that's just like really served me well. I think that's sort of separate from machine learning. You'd see like other professors would come and give like guest talks and, you know, they would say something's kind of lazy and like we'd all just be sitting there just like waiting for deaf to like eviscerate them.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, totally. As a kid, I was obsessed with playing games and I got really into Go, and I was super into the idea of or thinking about how would computers win at these games. And so I actually sent Daphne an email maybe as a freshman being like, hey, can I work with you? Like, I'm really interested in games. I want to learn how to beat Go. And Daphne wrote me actually a pretty polite email being like, that's not what I do. Go away. A few years later, I took her course and I was actually, I studied math at Stanford. And I have to say, Daphne cared about a thousand times more about teaching than even the best professor in the math department. And so it was really just eye-opening. I just loved how much she actually cared about teaching. And it got me really excited about the AI that was working there. And I went on to be a research assistant for her. And the funny thing at that time was like, nothing really worked. Like it was just before kind of, you know, Google was thought to be really like page rank.”
2023-08-03 · No Priors · Eradicating Machine Learning Pain Points with Weights & Biases CEO Lukas Biewald · IDENTIFIED FROM THE TRANSCRIPT