YouSaid · the spoken record

Karina Nguyen

lines on the record: 72
first: 2025-02-09
most recent: 2025-02-09
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“You definitely want to measure progress the model, and this is what Ebal says is because you can have prompted model as a baseline already. And if the most robust evals is the one where prompted baselines get the lowest score or something. And then because then you know, if you trained a good model, then it should just hill climb in that eval all the time while not also regressing on other intelligence evals. So I think it's more, that's what I'm saying, it's more of an R than science. It's like, okay, like if you optimize the model for this behavior, you kind of don't want to bring down damage in other areas of intelligence. And this is happening all the time in every lab, in every research team. I would say prompting is like also a way to prototype new product ideas.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Like some of the deterministic evaluations, then it's like, okay, like if the user says like 7 p.m., it should like the model should say 7 p.m. So if you can like have a deterministic evals, whether it's like pass or fail. So yeah, and like the way it works is sometimes I ask prog managers to like go create a Google sheet like have different tabs and like what's the current behavior, what's like the ideal behavior and like why or like some notes and sometimes you usually use it for eval sometimes we use it for training because like if you give the spreadsheet to like a one model it can probably figure out like how to teach itself a good behavior and I think there are certain type of like evals that is kind of more prevalent is like human evaluations and you can have specific chainers”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Certainly, it depends on what you're developing, but there are various types of evaluations. So sometimes I do ask product managers or there's also new role that we have like model designers to kind of go through some of the user feedback maybe or like think of various user conversations that should have triggered under these circumstances it should trigger canvas and then you have this like ground truth label of like okay with this conversation it should look to ground canvas and that this conversation it should not trigger canvas and you have this like very binary deterministic kind of like eval that for like decision binary behaviors is like this when we were launching tasks for example like how do you make correct schedules is like actually really hard for the model but we built out”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Like writing code, like, you know, changing models, writing evals, working with PMs and designers to learn, teach them how to even think about evaluations. I think that was really cool experience. And I think it was just an adoption of how do we really do this product management of AI features or like AI models. Yeah, but now it's like mostly management and like mentorship. I'm still like doing, I see research code after like 4 p.m. although. But yeah, it's just kind of like changed.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Sometimes I do that, sometimes I do set first. Actually, I think I learned this so much from Adabac. It's like people spend so much time just like prompting models and qualitatively bug bash all the time. And you actually get a lot of new ideas, how do you make the model better? It's like, oh, like this response is kind of weird. Like, why is it doing this? And you start debugging or something, or you start figuring out new methods of how do you teach the model to respond in a different way, like have better personality, let's say. So it's the same thing of like how personality is made in the models within those, like very similar methods. But yes, I think my time, ArabDa have changed. I think when I first came, I was like mostly like research IC work. So I was like building a lot of like”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“But then we used O1 to like produce the document. And then we kind of injected user prompt to be like, oh, make some comments, critique my piece of writing, or critique this piece of writing that you just made. And then we taught the model to make comments on the document on very specific talking documents. This is like also what kind of comments you want the model to make? Like, do they make sense or not? Like, how do you teach the quality of that? And it all came down to like measuring progress via very robust evals. But yeah, this is how you use a one and a synthetic data generation for like the straining.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Like completely rewriting the document versus having very specific targeted edits. So that's another layer of decision boundary within edit itself. It's like select the entire document that we write completely or you want to have very targeted custom behavior. And when we first launched the model, we would bias the model towards more rewrites because we thought the quality of the rewrites were much higher. But over time, you're kind of shifting based on user feedback and what's the learning from iterative deployment. Lastly, the third behavior that we taught synthetically, the model is how do you make comments on any document? So the way we use it is like we would use a one model to produce, to like simulate music conversation. Let's say write me a document about X-ray.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“How do we teach the model to update the document when the user asks? So, one of the behaviors that we taught the model is actually have some agency and autonomy to literally go to the document and select specific sections and either delete it or edit. So highlight it and rewrite certain sections. So sometimes the model sometimes the user would just like say change the second paragraph to be something friendlier and we would have to teach the model to literally find the second paragraph in the document and change it to a friendly tool. So basically you teach both like how to trigger edit itself but also how do you teach the model to get higher quality edit for that document. In case of like coding for example there's also like the question of like how good the model is.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Like, what are the most core behaviors that you wanted this product feature to do? And for Canvas, for example, it came down to three main behaviors. It was how do you trigger Canvas for prompts like Write Me Along SA when the user intention is mostly like iterating over long documents or write me a piece of code? When to not trigger Canvas for prompts like, can you tell me more about? President, like, I don't know, some of the general questions. So, you don't want to elect trigger canvas because the user intention is mostly getting answered, not necessarily iterate always a long document. The second behavior”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Like the first project of OpenAI, where researchers and applied engineers started working together from the very beginning of the product development cycle. And I think there's a lot of things that we have learned on the way. I definitely came with the mindset of like, we need to do like a really rapid model iteration such that it will be much easier for engineers to work with the latest model possible, but also learn from user feedback or early internal dog food, how to improve the model very rapidly. And it's really hard to kind of figure out how people, when you deploy a product, how people would be able to use it. And so the way you synthetically train the model is basically figuring out”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“So when I first came to OpenAI, I really had this idea of like, okay, it would be really cool for ChatGPT to actually change the visual interface, but also change the way it is with people. So going from being a chatbot to more of a collaborative agent and a collaborator is like a step towards more gent existence that become innovators ultimately. And so the entire team of applied engineers, designers, product, like research kind of got formed in the air almost out of nothing. It's just like a collection of people who just got together and we rapidly started iterating with each other. Actually, like Kevis is one of the”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“It's like a rapid model iteration for similar product outcomes. And we can dive more into, but the way we made Canvas and tasks and new product features for HTTP was mostly done by synthetic training.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Some tasks are synthetically curated, so this is like an active research area is like, how do you synthetically construct new tasks model to learn? Sometimes, you know, like when you develop products, you get a lot of data from the product and user feedback and you can use that data too and like this cross-chaining world. Sometimes you still want to use human data because actually some of the tasks can be really, really hard to achieve experts only know certain knowledge about some chemicals or like biological knowledge. So like you actually need to tap into the expert knowledge a lot. So yeah, I think to me like synthetic data training is more for”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Via reinforcement of learning. So any task, for example, like how to search the web, how to use the computer, how to write well, like all sorts of tasks that you're trying to teach the model, all the different skills. And that's why we think there's no data wall or whatever, because there will be infinite amount of tasks. And that's how the model becomes extremely super diligent. And we are actually getting saturated in all benchmarks. So I think the bottleneck is actually in evaluations that we don't have all the frontier, like EWAs, like, I don't know, GPGA, which is a Google proof question answering PhD level intelligence patchwork is getting to like, I don't know, more than 60, 70%, which is what PhD gets. It's like literally hitting the wall and like evol”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Drive basically, and you only have like a few words that will match that a car. So the model actually learns about the world in itself. So it's like it's modeling human behavior. Sometimes it's modeling. And when you talk to like pre-trained models, which are very, very large, they're actually extremely diverse and extremely creative because you can talk to almost any Reddit user through Prachine model. But I think what's happening right now with new paradigm of O1 series is of the scaling and post-chaining itself is not hitting the wall. And that's because basically we went from raw data sets from pre-trained models to infinite amount of tasks that you can teach the model in the post-”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“There are two questions here we can unpack one at a time, but people say we are hitting the data wall. I think people think more in the terms of pre-trained large models that are trained on the entire internet to predict the next token. But what actually the model is learning during that process is actually how do you compress the compression algorithm here. The model learns to compress a lot of knowledge and it learns how to model the world. So the next prediction of the world teach me how to”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Like extremely confused about whether it can set an alarm, but it doesn't have a body in the physical world. So it's like the model gets confused and sometimes it's like over refused. So sometimes it says like, I don't know, like, sorry, I cannot help you. And so there is always a balanced trade-off between how do you make the model to be more helpful for users, but also not being harmful.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Model training is more an art than a science and in a lot of ways like we as model trainers think a lot about like data quality is like it's one of the most important things in model training is like how do you ensure the highest quality data for certain like interaction model behavior that you want to create but the way you debug models is actually very similar the way you debug software so one of the things that I've learned early days at Anthropic was like we've discovered especially with like cloud 3 training when you taught the model some of the self-knowledge of like hey like you actually don't have a physical body to operate like in the physical world but then at the same time we had data that kind of taught the model some of the function calls which is like this is how you set the alarm and so the model would get”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Sounds great. Thank you so much. Yeah, I was extremely lucky to join early days on topic and kind of learned a lot of things there. And I joined OpenAI around like eight months ago. So yeah, I'm excited to dance more into this.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“When you taught the model some of the self knowledge, you actually don't have a physical body to operate in the physical world, the model would get extremely confused.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“Thinking you kind of want to like generate a bunch of ideas and filter through them and not just load the best product experience. I think it's actually a really, really hard to teach the model how to be aesthetic with really good visual design or like how to be extremely creative in the way they write.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source
“I first came to Andarby, and I was like, oh, that I really love from an engineering. And then the reason why I switched to research is because I realized, oh my god, cloud is getting better at front end. Cloud is getting better at like coding. I think COD can like develop new apps.”
2025-02-09 · Lenny's Podcast · OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · IDENTIFIED FROM THE TRANSCRIPT · source