YouSaid · the spoken record

François Chollet

lines on the record: 161
first: 2020-08-31
most recent: 2020-08-31
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“The assumptions in ARC is that every task should only require core knowledge priors, should not require any outside knowledge. So for instance, no language, no English, nothing like this, no concepts taken from our human experience like trees, dogs, cats and so on. So only tasks that are, reasoning tasks that are built on top of core knowledge priors. And some of the tasks are actually explicitly trying to probe specific forms of abstraction. Part of the reason why I wanted to create arc is a big believer in”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, Raven's privacy matrices. I mean, if you've done IQ tests in the past, you know, whether it is probably, at least you've seen it, even if you don't know what it's called. And so you have a set of tasks, that's what they're called. And for each task, you have training data, which is a set of input and output pairs. An input or output pair is a grid of colors, basically. The grid, the size of the grids is variables. The size of the grid is variable. And you're given an input and you must transform it into the proper output. And so you're shown a few demonstrations of a task in the form of existing input output pairs, and then you're given a new input. And you must provide, you must produce the correct output.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So, in the paper I outlined all these principles that a good test of machine intelligence and humanities should follow. And the arc challenge is one attempt to embody as many of these principles as possible. So I don't think it's anywhere near a perfect attempt, right? It does not actually follow every principle, but it is what I was able to do given the constraints. The format of arc is”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Like a turntable with a fixed speed. And if you want to know if two objects are stated versions of each other, you put the object on the turntable, you let it move around a little bit, and then you stop when you have a match.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Interesting, it's very likely that there is speaking about rotation that there is in the brain a hard-coded system that is capable of performing rotations. One famous experiment that people did in the, I don't remember it was exactly, but in the 70s was that people found that if you asked people, if you give them two different shapes and one of the shapes is rotated version of the first shape and you ask them, is that shape a rotated version of the first shape or not? What you see is that the time it takes people to answer is linearly proportional, right, to the angle of rotation. So it's almost like you have somewhere in your brain”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Cunning, like cunning one, two, three ish, then maybe more than three. You can also compare quantities if I give you three dots and five dots, you can tell the side with five dots as more dots. So this is actually an innate prior. So that said, the list may not be exhaustive. So Spelke is still pursuing. The potential existence of new knowledge systems, for instance, knowledge systems that we deal with social relationships.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Like, for instance, grid cells and place cells. So it's something that's literally hard coded at the neural level in our hypocampus. And the last prior would be the notion of numbers. Like numbers are not actually a cultural construct. We are intuitively innately able to do some basic counting and to compare quantities. So it doesn't mean we can do arbitrary arithmetic.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Moving in roughly synchronized fashion, you will intuitively infer that one of the dots is pursuing the other. So that one of the dots is, and one of the dots is an agent, and its goal is to avoid the other dot. And one of the dots, the other dot is also an agent, and its goal is to catch the first dot. has shown that babies, you know, as young as three months identify agentness and goal directedness in their environment. Another prior is basic geometry and topology, like the notion of distance, the ability to navigate in your environment and so on. This is something that is fundamentally hardwired into our brain. It's in fact backed by very specific neural mechanisms.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“This phenomena as well to human And the second prior that's also fundamental is agentness, which is not a real world. So agentness. The fact that some of these objects that you segment your environment into, some of these objects are agents. So what's an agent? Basically, it's an object that has goals. That has what? That has goals. They're capable of pursuing goals. So for instance, if you see two dots.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Four different core knowledge systems, like the first one is the notion of objectness and basic physics. Like you recognize that something that moves coherently, for instance, is an object. Physical coherence. And in terms of telemetry physics, there's the fact that objects can bump against each other and the fact that they can occlude each other. These are things that we are essentially born with or at least that we are going to be acquiring extremely early because we already hardwire to acquire them.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, so a researcher has done a lot of work on what exactly are the knowledge priors that are innate to humans is Elizabeth Spelke from Harvard. So she developed the core knowledge theory which outlines four different core knowledge systems, so systems of knowledge that we are basically either born with or that we are hardwired to acquire very early on in our development. And there's no strong distinction between the two. primed to acquire a certain type of knowledge in just a few weeks you might as well just be born with it it's just it's just part of who you are and so there are”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Pregenerate every possible question and answer. So it should be tasks that cannot be anticipated not just by the system itself, but by the creators of the system, right?”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So in the paper I outline a number of requirements that you expect of such a test. And in particular, we should start by acknowledging the priors that we expect to be required in order to perform the test. So we should be explicit about the priors, right? And if the goal is to compare machine intelligence and human intelligence, then we should assume human cognitive priors, right? Secondly, we should make sure that we are testing for skilled acquisition ability, skilled acquisition efficiency in particular, and not for skill itself, meaning that every task featured in your test should be novel and should not be something that you can anticipate. So for instance, it should not be possible to brute force the space of possible questions, right?”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I mean, novelty is a requirement. You should not be able to practice for the questions that you're going to be tested on. That's important. Because otherwise what you're doing. Is not exhibiting intelligence, which you're doing is just retrieving what you've been exposed before. It's the same thing as deploying model. If you train a deep learning model on all the possible answers, then it will ace your test in the same way that a stupid student can still ace the test if they cram for it. They memorize a hundred different possible mock exams and then they hope that the actual exam will be a very simple interpolation of the mock exams. And that student could just be a deep learning model at that point. But you can actually do that without any understanding of the material. And in fact, many students pass the exams in exactly this way. And if you want to avoid that, you need an exam that's unlike anything they've seen that rebes their understanding.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Basically, yes, meaning that so the test I'm interested in creating are not necessarily difficult for humans because human intelligence is the benchmark. They're supposed to be difficult for machines in ways that are easy for humans. Like I think an ideal test of human and machine intelligence is a test that is actionable that highlights the need for progress and that highlights the direction in which you should be making progress.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, I am personally not a fan of ambiguity in test questions, actually. But I think you can have difficulty without requiring ambiguity simply by making the test require a lot of extrapolation over the Tring examples.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Say the approach is implicit in the training examples. So here it is a training examples, it's over. Is why in arc, for instance, there is a test set that is private, and no one has seen it”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Requires improvisation is intrinsically hard, right? Because maybe your quantum physics expert. So when you take the exam, this actually stuff that despite being new to the students, it's not new to you, right? So it can only be difficult with respect to what the test taker already knows and with respect to the information that the test taker has about the task. So that's what I mean by controlling for priors, what you the information you bring to the table. And expense, which is the train data. So in the case of the quantum physics exam that would be all the course material itself and all the mock exams that students might have taken online.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“If you wanted to kind of test the understanding that students have of the material, you would come up with an exam that's very different from anything they've seen like on the internet when they were cramming. On the other hand, if you wanted to make it easy, you would just give them something that's very similar to the mock exams that they've taken something that's just a simple interpolation of questions that they've already seen. And so that would be an easy exam. It's very similar to what you've been trained on. And a difficult exam is one that really probes your understanding because it forces you to improvise. It forces you to do things that are different from what you've exposed to before. So that said, it doesn't mean that the exam that you”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“For instance, let's say you have a rotation problem, you must rotate a shape by 90 degrees. If I give you two examples and then I give you one prompt, which is actually one of the two training examples, then there is zero generalization difficulty for the task. It's actually a trivial task. You just recognize that it's one of the training examples and you produce the same answer. Now, if it's a more complex shape, there is a little bit more generalization, but it remains that you are still doing the same thing at this time as you were being demonstrated at training time. At difficult tasks that will require some amount of test time adaptation, some amount of improvisation. So consider, I don't know, you're teaching a class on quantum physics or something.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So, the thing to keep in mind is that there's no such thing as a question that's intrinsically difficult. It has to be difficult to respect to the things you already know and the things you can already do, right? So in terms of an IQ test question, typically we'd have, it will be structured, for instance, as a set of demonstration input and output pairs, right? And then you would be given a test input, a prompt, and you would need to recognize or produce the corresponding output. And in that narrow context, you could say a difficult question is a question where the input prompt is very surprising and unexpected. Given the training examples.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, limiting. So we evolved our cognition and our body evolved in very specific environments because our environment was so variable, fast changing, and so unpredictable. Part of the constraints that drove our evolution is generality itself. So we were in a way to be able to improvise in all kinds of physical or cognitive environments. And for this reason, it turns out that the minds and bodies that we ended up with can be applied to much, much broader scope than what they were evolved for, right? And that's truly remarkable. And that's a degree of generalization that is far beyond anything you can see in artificial systems today, right? It does not mean that human intelligence is anywhere universal.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Where the mind was initially supposed to do, which is why we can play music and write novels and go to mass and do all kinds of crazy things. But it's not universal in the same way that human morphology and our body is not appropriate for actually most of the universe by volume in the same way you could say that the human mind is not really appropriate for most of problem space, potential problem space by volume. So we have very strong cognitive biases actually that mean that there are certain types of problems that we handle very well and certain types of problems that we are completely inadapted for. So that's really how we'd interpret the g-factor. It's not a sign of strong generality. It's really just a broader conditionability. But our abilities, whether we are talking about sensory motor abilities,”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“And remarkable is that our morphology generalizes far beyond the environments that we evolve for. Like in a way, you could say we evolved to run after prey in the savannah, right? That's very much where our human morphology comes from. And that said, we can do a lot of things that are completely unrelated to that. We can climb mountains, we can swim across lakes, we can play a table tennis. I mean table tennis is very different from what we were evolved to do, right? So our morphology, our bodies, our sense and motor affordances have a degree of generity that is absolutely remarkable, right? And I think cognition is very similar to that. Our cognitive abilities have a degree of generality that goes far beyond”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Characteristics like maybe lung volume is correlated with being a fast runner, for instance, in the same way that there are neurophysical correlates of cognitive abilities. And at the top of the hierarchy of physical abilities that you would be able to observe, you would have a G-factor, a physical g-factor, which would map to physical fitness, right? And as you just said, that doesn't mean that people with high physical fitness can fly doesn't mean human morphology and human physiology is universal. It's actually super specialized. We can only do the things that we evolved to do, right? Like we are not appropriate to, you could not exist on Venus or Mars or in the void of space or the bottom of the ocean. So that said, one thing that's really striking.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Even if your very fit, that doesn't mean you can do anything at all in any environment. You obviously cannot fly. You cannot surve out the bottom of the ocean and so on. And if you were a scientist and you wanted to precisely define and measure physical fitness in humans, then you would come up with a battery of tests like you would have running 100 meter, playing soccer, playing table tennis, swimming, and so on. And if you run these tests over many different people, you would start seeing correlations in test results. For instance, people who are good at soccer are also good at sprinting, right? And you would explain these correlations with physical abilities that are strictly analogous to cognitive abilities, right? And then you would start also observing correlations between biological”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Intelligence because it's a useful concept. It's something you can intuitively understand. Some people are fit, maybe like you, some people are not as fit. Maybe like me.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“But they all describe a hierarchy with a kind of g factor at the top. And you're right that the g factor is, it's not quite real in the sense that it's not something you can observe and measure, like your height, for instance. But it's really in the sense that you sit in a statistical analysis of the data, right? One thing I want to mention is that the fact that there is a G-factor does not really mean that human intelligence is general in a strong sense does not mean human intelligence can be applied to any problem at all and that someone who has a high IQ is going to be able to solve any problem at all. That's not quite what it means. I think one popular analogy to understand it is the sports analogy. If you consider the concept of physical fitness, it's a concept that's very similar to”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so human connectivities have a structure, like the most mainstream theory of the structure of connectivities is called CHG theory. It's a cattle, horn, carol. It's named after the three psychologists who contributed key pieces of it. And it describes cognitive abilities as a hierarchy with three levels. And at the top, you have the G-factor. Then you have broad cognizabilities, for instance, through the intelligence, right? that encompass a broad set of possible kinds of tasks that are all related. And then you have narrow cognitive abilities at the last level, which is closer to task-specific skill. And there are actually different theories of the structure of cognitivity. They just emerge from different statistical analysis of IQ test results.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“That gives you information about the structure of the human mind, in particular about the structure of human cognitive abilities. So at scale, psychometrics paints a certain picture of the human mind. And that's interesting. And that's what's relevant to AI, the structure of human cognitivities.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“But it's there. It's there at scale. And that's also one thing I want to mention about psychometrics. When you talk about measuring intelligence in humans, for instance, some people get a little bit worried. They will say, you know, that sounds dangerous. Maybe that sounds potentially discriminatory and so on. And they're not wrong. And the thing is, personally, I'm not interested in psychometrics as a way to characterize one individual person. Like if I get your psychometric personality assessment or your IQ, I don't think that actually tells me much about you as a person. I think psychometrics is most useful as a statistical tool. So it's most useful at scale. It's most useful when you start getting test results for a large number of people and you start cross-correlating these test results because”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“In statistical terms, you would explain them with a latent variable. And the latent variable that would, for instance, explain the relationship between being good at math and being good at physics would be cognitive ability, right? And the g factor is the latent variable that explains the fact that every test of intelligence that you can come up with results on this test end up being correlated. So there is some single unique variable that explains this correlation. So that's the g-factor. So it's a statistical construct. It's not really something you can directly measure, for instance, in a person.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So, right, there are many different kinds of tests of intelligence, and each of them is interested in different aspects of intelligence. Some of them will deal with language, some of them will deal with spatial vision, maybe mental rotations, numbers, and so on. When you run these very different tests at scale, what you start seeing is that there are clusters of correlations among test results. So for instance, if you look at homework at school, you will see that people who do well at math are also likely statistically to do well in physics. And what's more, there are also people do well at math and physics are also statistically likely to do well in things that sound completely unrelated, like writing and English essay, for instance. And so when you see clusters of correlations,”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“The English language, then you have to be aware that this creates a bias against people who have English as their second language or people who can't speak English at all. So of course these principles for creating psychometric tests are very much an ideal. I don't think every psychometric test is really either reliable, valid, or offer from bias, but at least the field is aware of these weaknesses and is trying to address them.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So it's a field with a fairly long history. So, you know, psychology sometimes gets a bad reputation for not having very reproducible results and psychometrics is actually some fairly solidly reproducible results. So the ideal goals of the field is tests should be reliable, which is a notion tied to reproducibility. It should be valid, meaning that it should actually measure what you say it measures. So for instance, if you're saying that you're measuring intelligence, then your test results should be correlated with things that you expect to be correlated with intelligence, like success in school or success in the workplace and so on, should be standardized, meaning that you can administer your tests to many different people in some conditions. And it should be free from bias, meaning that, for instance, if your test involves”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Psychometrics is the subfield of psychology that tries to measure, quantify aspects of the human mind. So in particular, cognitive abilities, intelligence, and personality threats as well.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes. So in particular, I think if your goal is to measure a human-like form of intelligence, then you should clearly establish that you want the AI you're testing to start from the same set of priors that humans start with”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, exactly. And other board games can also share some similarities. And if you've played these board games, then with respect to the game of go, that would be part of your priorities about”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Expense is acquired right. So, for instance, if you're trying to play go, your experience with Go is all the Go games you've played or you've seen or you've simulated in your mind, let's say. And your priors are things like, well, Go is a game on a 2D grid and we have lots of hard-coded priors about the organization of 2D space.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Of a human, you have to control for priors. You have to start from the same set of knowledge priors about the task. And you have to control for experience, that is to say, for training data.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Hard could it prior knowledge into the system via rules and so on that come from the human mind, from the minds of the programmers? And also buying higher levels of skill just by training on more data. For instance, you could generate an infinity of different Go games and you could train a Go playing system that way, but you could not directly compare it to human goal playing skills because a human that plays go had to develop that skill in a very constrained environment. They had a limited amount of time, they had a limited amount of energy. And of course, this started from a different set of priors. They started from innate human priors. So I think if you want to compare the intelligence of two systems, like the intelligence of an AI and the”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So if your goal is to create AIs that are more human-like, then it will be super variable, obviously, to have a test that's universal, that applies to both AIs and humans so that you could establish a comparison between the two that you could tell exactly how intelligent in terms of human intelligence a given system is. So that said, The constraints that apply to artificial intelligence and to human intelligence are very different. And your test should account for this difference. Because if you get artificial systems, it's always possible for an experimenter to buy arbitrary levels of skill at arbitrary tasks, either by injecting”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So, a very smart agent can make efficient uses of very little information about a new problem and very little prior knowledge as well to cover a very large area of potential situations in that problem without knowing what these future new situations are going to be.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“So you can think of intelligence as a measure. Of an information conversion ratio. Like imagine a space of possible situations. And you've covered some of them. So you have some amount of information about your space of possible situations that's provided by the situations you already know. And that's, on the other hand, also provided by the prior knowledge that the system brings to the table, the prior knowledge that's embedded in the system. So the system starts with some information about the problem, about the task. It's about going from that information to a program, what we would call a skill program, a behavioral program that can cover a large area of possible situation space. And essentially the ratio between that area and the amount of information you start with. is intelligence.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, absolutely no. Deep plan industrialized a little bit. Like journalization is not binary. It's more like a spectrum.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Likewise, in order to acquire a skill you need, a human engineer to write down a bunch of rules that cover most or every possible situation. Likewise, the system is not intention. The system is merely the output artifact of a process that happens in their minds of the engineers that are creating it. It is encoding an abstraction that's produced by the human mind and intelligence would actually be The process of producing, of autonomously producing substruction. If you take an abstraction and you encode it on a piece of paper or in a computer program, that abstraction itself is not intelligent. What's intelligent is the agent that's capable of producing these abstractions.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Which you can adapt to new situations, to truly new situations, not situations you've seen before, right? Not situations that could be anticipated by your creators, by the creators of the system, but three new situations. The efficiency with which you acquire new skills. If you require, in order to pick up a new skill, you require a very extensive training data set of most possible situations that can occur in the practice of that skill, then the system is not intelligent. It is mostly just a lookup table.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“Driver and asks it to learn to pilot a commercial airplane, for instance. And then you would look at how much human involvement is required and how much strain data is required for the system to learn to pilot an airplane. That gives you a measure of how intelligent that system is.”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“With engineering an explicit model of the surrounding of the cause and you bridge the two in a clever way, your model will actually start generalizing much earlier and more effectively than the end-to-end deep training model. So why would you not go with the more manual engineering oriented approach? Like even if you created that system either the end-to-end deep learning model system that's infinite data or the slightly more human system, I don't think achieving L5 would demonstrate general intelligence or intelligence of any generality at all. Again, the only possible test of generality in AI would be a test that looks at skill acquisition over unknown tasks. For instance, you could take your L”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source
“A real amount of intelligence. For instance, if you really did have infinite training data, you could just literally train an end-to-end deep learning model that does driving, provided infinite training data. The only problem with the whole idea is collecting data sets that's sufficiently comprehensive, that covers the very long tail of possible situations you might encounter. And it's really just a scale problem. So I think there's nothing fundamentally wrong with this plan, with this idea. It's just that it strikes me as a fairly inefficient thing to do because you run into this scaling issue with diminishing returns. Whereas if instead you took a more manual engineering approach where you learning modules in combination,”
2020-08-31 · Lex Fridman Podcast · #120 – François Chollet: Measures of Intelligence · IDENTIFIED FROM THE TRANSCRIPT · source