YouSaid · the spoken record

Shane Legg

lines on the record: 54
first: 2023-10-26
most recent: 2023-10-26
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“For this, but yeah, we don't really test for that kind of a thing. So there is all sorts of bits and pieces, but it is a difficult question because you really need to, as I said, intelligence, the generality of human intelligence is very, very broad. So you really have to start going into the weeds of trying to find there's specific types of things that are missing from existing benchmarks or different categories of benchmarks that don't currently exist or something.”
2023-10-26 · Dwarkesh Podcast · Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, now that hot grease, these are quite big areas. So they don't measure things like understanding streaming video, for example, because these are language models and people can do things like understanding streaming video. They don't do things like humans have what we call episodic memory, right? So we have a working memory, which are things that have happened quite recently. And then we have sort of cortical memory. So these are things that have sort of been in our cortex that have been, but there's also a system in between, which is episodic memory, which is a hippocampus. And so this is about learning specific things very, very rapidly. So some of the things I say to you today, if you remember them tomorrow, that'll be your episodic memory epigammas. Our models don't really have that kind of thing, and we don't really test for that kind of thing. We just sort of try to make the context windows, which is, I think, more like a working memory, longer and longer to sort of compensate.”
2023-10-26 · Dwarkesh Podcast · Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures · IDENTIFIED FROM THE TRANSCRIPT · source
“Or not we're there, it's difficult because you'll never have a complete set of everything that people can do because it's such a large set. But I think that if you ever get to the point where you have a pretty good range of tests of all sorts of different things that people do, cognitive things people can do, and you have an AI system which can meet human performance and all those things. And with some effort, you can't actually come up with new examples of cognitive tasks where the machine is below human performance, then at that point it's conceptually possible that there is something that the machine can't do that people can do. But if you can't find it with some effort, I think it will be practical purposes. You now have an AGI.”
2023-10-26 · Dwarkesh Podcast · Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures · IDENTIFIED FROM THE TRANSCRIPT · source
“That's a hard question, usually. AGI, by its definition, is about generality. So it's not about doing a specific thing. It's much easier to measure performance when you have a very specific thing in mind because you can construct a test around that. Well, maybe I should firstly explain what do I mean by AGI? Because there are a few different notions around. When I say AGI, I mean a machine that can do the sorts of cognitive things that people can typically do, possibly more. But that's to be an AGI, that's kind of the buyer you need to meet. So if we want to test whether we're meeting this threshold or we're getting close to the threshold, what we actually need then is a lot of different kinds of measurements and tests that spans the breadth of all the sorts of cognitive tasks that people can do and then to have a sense of what is human performance on these sorts of tasks and that then allows us to sort of judge whether”
2023-10-26 · Dwarkesh Podcast · Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures · IDENTIFIED FROM THE TRANSCRIPT · source