YouSaid · the spoken record

Kanjun Qiu

lines on the record: 18
first: 2023-11-16
most recent: 2023-11-16
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“I mean, I did this over the weekend actually for mid-journey. I got really sick of typing out mid-journey prompts on my phone in Discord. You can't keep iterating the prompts. So I just made a little thing that interacts with it via the API. Well, my version of the API. But yeah, and so, but I think everyone will be able to do this. Like, it didn't actually take that much code when we have agents that can write code, someone else who wants to use it in a different way. Great. You can just ask the agent to do that, come back five minutes later, and you have your own perfect way of interacting with this. I think that's just going to make our computers feel so much nicer to interact with.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“I think it's not only just more software, but also better software. If already we're having our agents kind of look at our pull requests, you know, fix the type errors, okay, but we can extend this to adding new unit tests to fixing the existing unit tests, to looking for security flaws. I'm very excited about agents that can go out and help all sorts of organizations improve the quality of their code base. How can we simplify this, refactor it, fix security flaws? I think there'll just be a huge flourishing of much higher quality, better software as a result, not just more software, but just taking the existing software and making it so much better, which will make it so much nicer and more fun to interact with as programmers as well.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, well, imbue is so much smaller in this other company, blah, blah, blah. It's like, no, no, we can write way more code than anyone else. So I think this is kind of a pretty interesting thing that over time, I think we'd like to work towards. And so that's another reason for code as well.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“We're using these coding agents to write the coding agents. And I think this is kind of the recursive self-improvement thing that people have always been sort of worried about or excited about in AI. But I think what it really looks like in practice is not this scary, like, oh, you leave your computer on overnight and all of a sudden it's a super, super god thing the next day. Instead, it's like this slow grind of making things a little bit better every day. But a 1% improvement every day over a year is huge, right? And so I think that's the kind of thing that we're really excited about with code is that not only can we apply it to our own workflows, but also as we start to actually get coding agents that can really write code, now we're in a very unlimited, like very interesting space. Right now the bottleneck for most companies is the ability to hire software engineers that can write really robust code, right? But if you can just turn compute into really good code, now this is a totally different world. Now there's none of this like, oh,”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“Yeah. So there's a bunch of different reasons for focusing on coding. One of them is that the evaluate, we talked about before, the evaluations are much easier to do and subjective. Another one is that coding is part of reasoning. Another one is that coding really helps us accelerate both our own work and the agents that we end up building. So as we're making the tools for ourselves, we already are starting to see this kind of leverage from the systems that we've built where like we can run this agent now, I think probably within the next year we'll probably not be hiring as many recruiting coordinators because, oh, we're going to do some of the scheduling with the agent that we've built, right? But we also can do the same thing on the software engineering side. We're writing unit tests literally right now automatically. Okay. And that's just helping accelerate us, helping, you know, remove the bugs. It's additive. It's incremental. It's like, okay, we get a 5% gain, a 10% gain here. But as we make more and more tools, those things compound. And I think over time, it's going to be possible to make much more robust systems, much more quickly.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“I mean, we actually have a lot of JPUs. Yeah, we have one may or may not apply, but we do have a lot of GPUs. We have enough compute to be able to train models that are as large as the largest models that have been trained today to date.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“I mean, I think actually a significant fraction of that is going to go to compute. I think I can't speak to other companies how they should deploy it. But I think for us, given that our goal is to make agents, what we really want actually as a company is not to become a huge company. We don't want tens of thousands of people. We want to make our product actually work so that we can make agents, so we can have some huge impact and have a relatively small close-knit team where the communication is much easier. It's really hard to communicate with 10,000 people. It's much easier to get 100 people in a room and know what the heck you want to do and agree on things. And so I think we're trying to ideally leverage ourselves. And we're already starting to do that today. And what that looks like is by spending a bunch on compute. Today, you know, we don't have AI agents that are running off and doing all sorts of things on their own, but we do have the beginnings of those. We do have our internal, you know, hyperparameter optimizer, for example, which saves us a ton of time. Instead of our researchers manually deciding like, oh, this learning rate, I should do this experiment, we just say, go, we come back after the night. And it's like, oh, great. Everything is optimized. This is really nice.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, one thing that our recruiter mentioned yesterday that I thought was kind of funny is she's been describing to Ken and it's like, we're actually sort of a software dev tooling company, but the idea is that in the future, everyone is going, like as we make these things easier and easier to program, really everyone's going to be at like sort of software engineer in that sense. Like we'll be able to make our own agents, right? Just by sort of working in natural language and describing what we want to do and how we want it to be done and interacting at that level. And so since we're going to be working on these agents, we're kind of making, we're like trying to move towards that kind of tooling. And so I think the goal in five years is for people to be able to really specify some huge range of possible agents that do exactly what they want. Like they can interact with their computer in whatever way they want.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“I think a year from now, we're going to start to see some of these use cases actually work that to date, you can write these. Like we have the capabilities. You can make some kind of agent to triage your email or to do scheduling or many of these workflows like we really should like, why don't we have that today? That definitely can be done, right? Like there's nothing stopping us. And I think five years from now, we're going to have something where it's not just, okay, we have a scheduling bot, we have this other thing, but we really have these more general, more robust systems where each of us can individually say like, I want a thing that does this. I want to do this particular weird research workflow and I want it to work like this and blah, blah, blah, blah, and just specify it in language. I think one thing that personalized agents.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“Until you get to a point where it's like, okay, I mean, a regular language vault or even just a person looking at this, like, there's an objective answer. One of the reasons why we work on code is that there are objective answers to a lot of these questions, either the test pass or they don't. Either the function is correct or it isn't. Those kind of things are much easier to evaluate. And so we're starting a lot more of our tests are in that zone as we sort of build up eventually to the ones that are a little bit more qualitative because the evaluation is so much harder there.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“The evaluations, I think, are actually one of the most important parts and one of the places where we spend the most time and think about it kind of the most, there's a lot of work in specifying exactly what you want from the to-do agent, for example, right? Like how do you know, like it gives you back some code. Okay, is that good? There's sort of a spectrum, but like if it's faster, it's better. If it gives you less code, that's better. But if there's bugs that's not good, so you really need to take it and break down what did I really want to happen here? And I think when you start to break this down, you start to say, okay, there's some things that are kind of qualitative, like do I trust it? Did it come back with tests? Can I run this code immediately? Like the kind of feel of it. There's other things that are just for the code itself. There are different attributes. Is it in the same style? Does it have good variable names? Like, is it a minimal change or did it change all sorts of stuff that it didn't really need to change? Each of those things are actually some that you can measure a little bit more easily than the overall task. So you can make another kind of metric that's like, okay, how good are the variable names? Well, all right, how similar are they? You can break that down. You can kind of keep breaking it.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“Yeah. So we're purposely trying to pick somewhat some diversity. Like we have one agent that will just go do a random to do in your code base. And so this can be super, super general. It can take a really, really long time to do this. And we have another opposite end of that spectrum is we have an agent that will look at every single pull request and run Linter against it and ask like, okay, are there any type errors? Okay, how do I fix them? All right, great. Here's like a PR with me fixing the type errors for you. But very, very specific. But really, you can imagine how, you know, you can invoke the to-do agent to fix a specific type error and you can expand the type error fixer to do unit tests and to do security flaws and to do renaming these variables. And they sort of meet in the middle as you kind of make these things both more capable. And so there are just different ways of kind of looking at the problem of how do we make a useful coding agent.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“We pick tasks kind of depending on a bunch of different factors one, like how useful, how frequent, how possible is this going to be to do, right? How generally applicable is it? How much is it going to help push the techniques that we want to push forward?”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I think for us, code is. Useful when we're thinking about reasoning. One way that we're sort of making collectively reasoning agents today is founders are just hard coding their reasoning process of like, okay, if there's a customer support complaint about this thing, then I do this. If it's like this, then I do that. And so you have this very special case version of the thing, right? And there's a spectrum between code and language or more kind of general reasoning abilities, but it's a spectrum. It's not a binary thing, I think. And so you can have code now that we have these language models that kind of mixes the language models and the code layer, right? where it's like sometimes you're using the language model to decide what to do sometimes you're using an if statement and so it's more about like a fusing or like melding of these two different things and being able to like be in the right place on that spectrum and so code is actually like a really important part of this and as you do things that you want to do more robustly and you want to do in a more repeatable way then you want to move it more towards code right and so to the extent that you've never seen this task before maybe you should be doing it in this more kind of nebulous intuition”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“In what is that other higher level system? How do we decide what is the right next step to take? When should I go collect more information? Am I certain about this? All of these kinds of other things, those are, I think, the questions that are much more interesting. And I think there's actually a lot of work to be done there. I think we're still very early in the days of creating these systems.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“I think there's a different process. Like, language models are great. They're really good at predicting the next word. They're good at making a very easy classifier. They're good at all sorts of things. But there are obvious limits. Like we know even in theoretical senses, like they cannot learn to do multiplication in the general sense because it literally doesn't fit in the context window. Multiplication, they can learn to do addition in a modular sense and they can learn to do it actually almost perfectly if you train them in the proper way, but they're not learning the general algorithm for addition. Instead, if you want something to actually execute the general algorithm for addition, you need to have a thing that works in a different way, that has some sort of outer loop about what step should I take next, right? That's just this kind of like definitional thing. There has to be some other sort of wrapper. There has to be a different sort of outside process. Everyone at OpenAI and Imbu and Anthropic, we all like know how this works. I don't think anyone is proposing like it's just, you know, shove it all in the language model. You can get really far, but I think we're interested.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“I think that's the right way of thinking about it. I think when Kenjun was saying the spectrum from a more specialized to more generalizable, I think we're talking about the ability to solve more general problems, like the ability to do these problems you've only seen once or twice. I think even as that ability goes up, we're still going to see kind of a thing coming behind that, a force that takes each of those things. Like maybe you start out by doing your plane booking with GPD-4, but eventually you realize like, oh, actually, like this is so expensive and slow. Like, I just want the thing to be really good at it. What you can do is you can apply these agents. And this is part of the reason why we're interested in agents that code. You can apply those agents to the original general system to have it go make a more specialized version of that. So it's kind of specializing the things that you're doing a lot. And you can look at like each of those things like, okay, I'm making 10,000 calls of this. This is super expensive. Can I just write a piece of Python code that does this? As you have more general capabilities, you actually can use those more general capabilities to kind of do that specialization.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT
“I think we've always been interested in agents, in not just recommender systems or classifiers or things like that, but in systems that are going to go do a real work for us, right, that are going to actually be useful in the real world. Right now, you can ask some kind of chat bot something, and it'll give you back a response, but the burden is sort of on you to go do something with that to verify whether it's correct or not. I think the real promise of AI is if we can get systems that can actually act on our behalf and kind of accomplish goals and kind of do these larger things and sort of free us up to focus on the things we're interested in.”
2023-11-16 · No Priors · AI Agents That Reason and Code with Imbue Co-Founders Kanjun Qiu and Josh Albrecht · IDENTIFIED FROM THE TRANSCRIPT