YouSaid · the spoken record

John Schulman

lines on the record: 95
first: 2024-05-15
most recent: 2024-05-15
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“So, for example, you're like learning how to shoot baskets. I think you probably like that takes maybe thousands of tries to get more accurate. And I think you probably, there's probably something that's like a policy grading algorithm underneath. But that's not going to be the fastest way to learn. you have a model trying to do a project or some kind of task. So I would think we would want to rely more on in-context learning where you effectively have a learned algorithm, like you've learned how to explore, like you've learned how to try all the possibilities exhaustively. And instead of doing the same thing over and over again, making the same mistake. So yeah, I would say we'll be able to do things that look more like learned search algorithms. And that'll be. The kind of thing that gets used in a particular task.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I think policy grading algorithms are not the most sample efficient algorithms, so that's probably not what you want to do at test time if you want to learn really fast. But though, who knows? I mean, maybe it's not that bad. So I think something like motor learning in animals is probably something like a policy grading algorithm.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I would expect this capability would start to become like the need for it would start to become clear when we start to look at long horizon tasks more and to some extent just putting a lot of stuff into context will take you pretty far because we have really long context now. But you probably also want things like fine tuning. As for like introspection and the ability to active learning that might automatically fall out of the model's abilities to know what they know because they have some models have some calibration regarding what they know. And that's why models don't hallucinate that badly because they have some understanding of their own limitations. So I think that like same kind of a Could be used for something like active learning.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I would say if you're doing some kind of long horizon task, uh, Well, you're learning while you do the task, right? So the only way to do something that involves a lot of steps is to have learning and memory that gets updated during the task. So there's a continuum between short-term memory between short-term and long-term memory. I would say. I would expect.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Would probably also want to supplement that by some kind of fine tuning, like the capabilities you get from fine-tuning and in-context learning are probably somewhat complementary. So I would expect as to want to build systems that do some kind of online learning and also have some of these cognitive skills of introspecting on their own knowledge and seeking out new knowledge that fills in the holes.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“People haven't really pushed too hard on this middle ground between large scale training, like where you produce this snapshot model that's supposed to do everything, like a deployed model. And then on the other hand, in context learning. And I think part of that is that we've just been increasing context length so much that there hasn't been an incentive for it. So if you can go to like 100,000 or a million context, then that's actually quite a lot. It's not actually the bottleneck in a lot of cases. But I agree that.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I see. So it's not just about finding, I don't know, training on a bunch of sources that are relevant, fine tuning on some special domain. It's also about like reasoning about developing some knowledge through your own reasoning and also using some sort of introspection and self-knowledge to figure out what you need to learn. Yeah, I would say that does feel like something that's missing from today's systems. I mean, I would say”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I see. So it's not just memory, but it's also somewhat like specializing to a task that specializing to a certain task or putting a lot of effort into some particular project.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so do you mean models having some kind of medium term memory? So too much to fit in context, but like much smaller scale than pre-training.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I would say you could define reasoning as tasks that require some kind of computation at test time or maybe some kind of deduction. So by definition, reasoning would be tasks that require some test time computation, like step-by-step computation. On the other hand, I would also expect to gain a lot out of like doing some kind of training time computation or practice at training time. So I would think that you get the best results by combining these two things.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I would say there are probably some analogies, though it's, I don't know exactly how close it is, but I would say to some extent it is the models do have drives and goals in some meaningful way. And in the case of RLHF, where you're trying to maximize human approval as measured by a reward model, the model is just trying to produce something that people are going to like and they're going to judge us correct.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I would say there are probably some analogies with a driver a goal in humans. So, in that you're trying to steer towards a certain set of states rather than some other states. And so I would think that our concept of a drive or a goal has other elements like the feeling of satisfaction you get for achieving it. And those things might be more like have more to do with the learning algorithm than what the model does at runtime when you just have a”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“But of course, yeah, if you had a task like make money, then maybe that would lead to some nefarious behavior as a instrumental goal.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Piece of code at the end, like if you ask it to write you a flask app, it'll be like, oh, yeah, first I need to take over the world. And then I need to, I don't know. But at a certain point, it's a little bit hard to imagine why for some fairly well specified task like that, you would want to first take over the world.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“That does feel even though the models are very smart, it does feel very safe because the model is just trying to produce a message that is pleasing to a human. And it has no concern about anything else in the world other than whether this text it produces is approved. So obviously if you were doing something where there's where the model has it's carrying out a long sequence of actions which involve tools and everything, then it might have some incentive to do a lot of wacky things that wouldn't make sense to a human in the process of producing its final result. But I guess it wouldn't necessarily have an incentive to do anything other than produce a very high quality output at the end. So I guess you have these old points about like instrumental convergence. Like the model's going to want to take over the world so it can produce this awesome.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think we would want to have to take some of these questions seriously and we would want to have a lot of evals that sort of test them for misbehavior in the most or I guess that's like for the alignment of the models we want to check that they're not going to sort of turn against us or something, but you might also want to look for discontinuous jumps and capabilities you'd want to have lots of evals for the capabilities of the models. I mean also I guess you'd also want to make sure that whatever you're training on doesn't have any reason to make the model turn against you, which itself I think isn't I would say there's like that doesn't seem like the hardest thing to do I mean if Like the way we train them with RLHF.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“You'd want to be pretty careful when you do this kind of training if you see a lot of potentially scary capabilities if those seem close. I mean, like I would say it's not something we would want to we have to be scared of right now because right now it's hard to get the models to do anything like coherent. But if they started to get really good, I think.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“You're pretty confident that it's extremely resistant to any kind of takeover attempt or something or like severe misuse. And then you would also want to have like really good monitoring on top of it. So yeah, you could detect any kind of any trouble.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“That you expect, so red teaming of sorts, like you'd want to do that in a way that you feel is much less favorable than or much more likely to fail than the thing you're planning to do in the real world. You'd want to have a really good monitoring system so that you can start to go wrong with the deployed system, you can feel like it's going to be detectable immediately. Like you've got maybe you've got something watching over the deployed AIs and what they're doing and looking for signs of trouble. So I would want to, yeah, I would say just you'd want some defense in depth. Like you'd want to have some combination of like the model itself seems to be like really well behaved and have like impeccable moral compass and everything.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Would say if there's more of a discontinuous jump and the question is how do you know if the thing you've got is safe to release Would say. I can't give a generic answer. Like, I would want to, but like the type of thing you might want to do to make that more acceptable would be you would want to do a lot of testing like simulated deployment.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I would say if we We can deploy systems incrementally that are successively smarter than the ones before, then I think that's safer. So I hope the way things play out is it's not the scenario where everyone has to coordinate and lock things down and safely release things because it would lead to this big buildup in potential energy potentially. So I would rather some scenario where we're just continually releasing things that are a little better than what came before. And then we while making sure we're confident that each diff is Like improving the safety and alignment to the improvement and capability. And if things started to look a little bit scary, then we would be able to slow things down. So that's what I would hope for.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I would say if we had everyone reasonably coordinated, we could figure out some, and we felt like we had solved the technical problems around alignment well enough to be able to deploy really smart AIs that can act as an extension of people's will, but also prevent them from being misused in some way that would cause a catastrophe. I think then that would be great. Like we could go ahead and safely deploy these systems and it would usher in a lot of prosperity and a new more rapid phase of scientific advancement and so forth. So I think that would be what the good scenario would look like.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I don't have a good answer to that. I mean, I would say if we can, if everyone is going to coordinate like that, I think we would be, that would be an okay scenario. That would be a pretty good scenario because I do think building these models is very capital intensive and there are a lot of complex pieces. So it's not like everyone's going to go and recreate this stuff at home. So I think it is possible to do given the relatively small number of entities who could train the largest models, it does seem possible to coordinate. So I'm not sure how you would maintain this equilibrium for a long period of time. But I think if we got to that point, we would be in an okay position.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Like pause either further training, pause deployment, like avoid certain types of training that we think might be riskier. So just like setting up some reasonable rules for what everyone should do to having everyone somewhat limit these things.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Everyone's like, and that might require compromising on safety. So I think you would probably need some coordination among the larger entities that are doing this kind of training.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, the game theory is a little tough to think through. So, oh yeah, so first of all, I don't think this is going to happen next year, but it's still useful to have the conversation. Maybe it's like two or three years instead. But yeah, I guess two or three years is still pretty soon. I do think you probably need some coordination. Everyone needs to agree on some reasonable limits to deployment or to further training for this to work. Otherwise, you have the race dynamics where everyone's trying to stay ahead and like.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I would say just like maybe not training the even smarter version, not being really careful when you do train it, that it's not properly sandboxed and everything, maybe not deploying it at scale or being careful about what scale you deploy it.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Well, I would say that if AGI came way sooner than expected, we would definitely want to be careful about it. And we might want to like. Slowed down a little bit on training and deployment until we're pretty sure we know we can deal with it safely and we have a pretty good hand along what it's going to do, what it can do. So I think, yeah, we would have to be very careful if it happened way sooner than expected because I think our understanding is rudimentary in a lot of ways still.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“It's hard to say exactly what will be the deficit. I mean, I would say that when you talk to the models today, they have various weaknesses besides long-term coherence in terms of also thinking hard about things or paying attention to what you ask them. I would say I wouldn't expect just improving the coherence a little bit to be all it takes to get to AGI. But I guess I wouldn't be able to articulate exactly what the main weakness is that'll stop them from being a fully functional colleague.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Where if you do text only fine tuning, you also get reasonable behavior with images. Early on in ChatGPT, we were trying to fix some issues in terms of the model understanding its own limitations. Early versions of the model would think they could send you an email or call an Uber or something. The model would try to play the assistant and it would say, oh yeah, of course I sent that email and obviously it didn't. So we started collecting some data to fix those problems and we found that a tiny amount of data did the trick, even when you mixed it together with everything else. So I don't remember exactly how many examples, but something like 30, 30 example. Well, we had us, I don't know, pretty small number of examples showing this general behavior of explaining that the model doesn't have this capability.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“There's definitely been some interesting instances of generalization in post training. Like one well-known phenomenon is if you do all your fine-tuning with English data, you'll automatically, you'll have the model also behaving well in other languages. So if you train the assistant on English data, it'll also do something reasonable in Spanish, say. And sometimes you might get the wrong behavior in terms of whether it replies in English or replies in Spanish, but usually you get the right behavior there as well. You get it to respond in Spanish to Spanish queries. So that's one kind of interesting instance of generalization that you just sort of latch onto the right helpful persona and then you automatically do the right thing in different languages. We've seen some versions of this with multimodal data.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I guess I wouldn't expect the web to get totally redesigned to have APIs everywhere because I would expect that we can get models to use the same kind of UIs that humans use.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, that's an interesting question. I mean, I would expect that models will be able to use websites that are designed for humans just by using vision, like when the vision capabilities get a bit better. So there wouldn't be an immediate need to change them. On the other hand, some websites that are going to benefit a lot from AI is being able to use them will probably want to design to be better UXs for AIs. I'm not sure exactly what that would mean, but probably like assuming that our models are still better in text mode than like reading text out of images, you'd probably want to have a good text-based representation for the models. And also just a good indication of what are all the things that can be interacted with.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Maybe there's some other experience that human experts bring to different tasks, like having some taste or dealing with ambiguity better. So I could imagine that if we want to do something like research, like those kind of considerations come into play. Yeah, obviously they're going to be just sort of mundane limitations around affordances of the model, like whether it can use UIs and obviously the physical world or having access to things. So I think there might be a lot of mundane barriers that are probably not going to last that long but would initially like slow down progress.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, it's not totally clear what we're going to see once we get into that regime. And you have fast progress will be. So that's still uncertain. I would say I would expect there to be, I wouldn't expect everything to be immediately solved by doing any training like this. I would think there'll be other miscellaneous deficits that the models have that cause them to get stuck or not make progress or make worse decisions than humans. So I wouldn't say I expect that this one little thing will unlock all capabilities, but yeah, it's not clear, but it might like some improvement in the ability to do long horizon tasks might go quite far.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Or like 100 years from now, it's so we're not actually doing some kind of reinforcement learning where we need to worry about a discount factor that covers that time scale and so forth. So I think using language, you can describe all of these different timescales. And then you can do things like plan in the moment you can try to make progress towards your goal, whether it's a month away or 10 years away. So I might expect the same out of models where there some kind of, I don't know if it's a phase transition, but there's some capabilities that work at multiple scales.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I would say at a high level, I would agree that longer horizon tasks are going to require more model intelligence to do well and are going to be more expensive to train for. I'm not sure I would expect there to be a really clean scaling law unless you set it up in a very careful way or design the experiment in a certain way because I would say there might end up being some phase transitions where once you get to a certain level, you can deal with much longer tasks. So for example, people I think when people do planning for at different timescales, I'm not sure they use completely different mechanisms. So we probably use the same mental machinery if we're thinking about.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“If you have models that are weaker, you might be able to get them to do almost anything with enough data, but you might have to put a lot of effort into a particular domain or skill. Whereas for a stronger model, it might just do the right thing without any training data or any effort.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“They're not directly connected. So I would say you usually have a little bit of data that does everything. So, I mean, if you have collected diverse data set, you're going to get a little bit of everything in it. And if you have models that generalize really well, even if there's just a couple examples of getting back on track, I see. Or even like maybe in the pre-training there's examples of getting back on track, then like the model will be able to generalize from those other things it's seen to the current situation. So I think like.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Doing this kind of training. So I'd say that's one thing. Also, I would expect that as the models get better, they're just better at recovering from errors or they have just. They're better at dealing with edge cases or when things go wrong. They know how to recover from it. So the models will be more sample efficient. So you don't have to collect a ton of data to teach them how to get back on track just a little bit of data or just their generalization from other abilities will allow them to get back onto track. Whereas current models might just get stuck and get lost.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I would say this will come from some combination of just training the models to do harder tasks like this. So just like I'd say the models aren't particularly like Most of the training data is more like doing single steps at a time. And I would expect us to do more for training the models to carry out these longer projects. So I'd say any kind of training, like doing RL to learn how to do these tasks, however you do it, whether you're supervising the final output or supervising it like each step. I think any kind of training at carrying out these long projects is going to make them a lot better. And since the whole area is pretty new, I'd say there's just a lot of low-hanging fruit. Interesting.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I think even in one or two years, we'll find that a lot of you can use them for a lot of more involved tasks than they can do now. So, for example, right now, Like you could imagine having the models do carry out a whole coding project instead of maybe giving you one suggestion on how to write a function. So you could imagine the model like you giving it sort of high level instructions on what to code up and it'll go and it'll go and write many files and test it, look at the output, iterate on that a bit. So just much more complex tasks.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“Oh, yeah, five years. Yeah, I think the models will get quite a bit better. But in what way, in the course of five years.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“All these different kinds of content. And then when we do post training, we're usually targeting a narrower range of behavior where we basically want the model to behave like this kind of chat assistant. And it's a more specific persona where it's trying to be helpful. It's not trying to imitate a person. It's answering your questions or doing your tasks. We're optimizing on a different objective, which is more about producing outputs that humans will like and find useful, as opposed to just trying to imitate this raw content from the web.”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source
“In pre training, you're basically training to imitate all of the content on the internet or on the web, including websites and code and so forth. So you get a model that can basically generate content that looks like random web pages from the internet. And the model is also trained to maximize likelihood where it has to put a probability on everything. So the objective is basically predicting the next token given the previous tokens. Tokens are like words or parts of words. And since the model has to put a probability on it and we're training with to maximize log probability, it ends up being very calibrated. So it can not only generate all the content of the web, it can also assign probabilities to everything. So the base model can effectively take on all of these different personas or generate”
2024-05-15 · Dwarkesh Podcast · John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI · IDENTIFIED FROM THE TRANSCRIPT · source