YouSaid · the spoken record

Yann LeCun

lines on the record
182
first
2024-03-07
most recent
2024-03-07
sittings or episodes
1
sources
podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

  1. “And we can put guardrails in open source systems. I mean, if we eventually have systems that are built with this blueprint, we can put guardrails in those systems that guarantee that there is sort of a minimum set of guardrails that make the system non-dangerous and non-toxic, et cetera. Basic things that everybody would agree on. And then the fine-tuning that people will add or the additional guardrails that people will add will kind of cater to their community, whatever it is.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  2. “I mean, there are some limits to what the same way there are limits to free speech. There has to be some limit to the kind of stuff that those systems might be authorized to produce, some guardrails. So, I mean, that's one thing I've been interested in, which is in the type of architecture that we were discussing before where the output of a system is a result of an inference to satisfy an objective. That objective can include guardrails.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  3. “Thinks about questions about religion and things like that, right? Or cultural issues that people from different communities would disagree with in the first place. So there's only kind of a relatively small number of things that people will sort of agree on, you know, basic principles, but beyond that, if you want those systems to be useful, they will necessarily have to offend. Number of people inevitably.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  4. “Physically. Yeah. I mean, Mark is right about a number of things that he lists that indeed scare large companies. Certainly congressional investigations is one of them. Legal liability, making things that get people to hurt themselves or hurt others. Big companies are really careful about not producing things of this type because they have, you know, they don't want to hurt anyone, first of all. And then second, they want to preserve their business. It's essentially impossible for systems like this that can inevitably formulate political opinions and opinions about various things that may be political or not, but that people may disagree about moral issues and, you know.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  5. “And can be offensive for some people as well, right? So it's going to be impossible to kind of produce systems that are unbiased for everyone. So the only solution that I see is diversity.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  6. “Going to be, you know, you push it in one way, one set of people are going to see it as biased, and then you push it the other way, and another set of people is going to see it as biased. And then in addition to this, there's the issue of if you push the system, perhaps a little too far in one direction. It's going to be non-factual, right? You're going to have Black Nazi soldiers in”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  7. “No, I don't think it has to do. I don't think the issue has to do with the political leaning of the people designing those systems. It has to do with the acceptability or political leanings of their customer base or audience, right? So a big company cannot afford to offend too many people. So they're going to make sure that whatever product they put out is safe, whatever that means And it's very possible to overdo it. And it's also very possible to, it's impossible to do it properly for everyone. You're not going to satisfy everyone. So that's what I said before. You cannot have a system that is unbiased, that is perceived as unbiased by everyone.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  8. “Our ability to meta's ability to derive revenue from this technology is not impaired by the distribution of base models in open source.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  9. “Well, no, the So it's going to be useful to them. Whatever we offer them is going to be useful. And there is a way to derive revenue from this. And it doesn't hurt that We provide that system. The base model, right? The foundation model in open source for others to build applications on top of it too. If those applications turn out to be useful for our customers, we can just buy it from them. It could be that they will improve the platform. In fact, we see this already. I mean, there is literally millions of downloads of Lama 2 and thousands of people who have provided ideas about how to make it better. This clearly accelerates progress to make the system available to sort of a wide community of people. And there is literally thousands of businesses who are building applications with it.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  10. “Okay, so you have several business models, right? The business model that Meta is built around. You are for a service And the financing of that service is either through ads or through business customers. So for example, if you have an LLM that can help a mom and pop pizza place by talking to the customers through WhatsApp. And so the customers can just order a pizza and the system will just ask them, what topping do you want or what sites? Blah, blah. The business will pay for that. Okay, that's a model. And otherwise, if it's a system that is on the more classical services, it can be ad supported or there's several models. But the point is. If you have a big enough potential customer base and you need to build that system, Anyway, for them, Don't hurt you to actually distribute it in open source.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  11. “Of which any group can build specialized systems. So the direction of inevitable direction of history is that the vast majority of AI systems will be built on top of open source platforms.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  12. “Technical abilities in various domains. And you can have an industry, an ecosystem of companies that fine-tune those open source systems for vertical applications in industry, right? You have, I don't know, a publisher has thousands of books and they want to build a system that allows a customer to just ask a question about any content of any of their books. You need to train on their proprietary data, right? You have a company. We have one within Meta is called Metamate. And it's basically an LLM that can answer any question about internal stuff about the company. Very useful. A lot of companies want this, right? A lot of companies want this not just for their employees, but also for their customers to take care of their customers. So the only way you're going to have an AI industry, the only way you're going to have AI systems that are not uniquely biased is if you have open source platforms on top of”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  13. “Very important for people in India. I was talking to a former colleague of mine, Mustafa Sice, who used to be a scientist at FAIR, and then moved back to Africa, created a research lab for Google in Africa, and now as a new startup called CERA. And what he's trying to do is basically have LLM that speak the local languages in Senegal so that people can have access to medical information because they don't have access to doctors. It's a very small number of doctors per capita in Senegal. I mean, you can't have any of this unless you have open source platforms. So with open source platforms, you can have AI systems that are not only diverse in terms of political opinions or things of that type, but in terms of... Language, culture, value systems, political opinions.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  14. “We're going to have a very large diversity of different AI systems that are specialized for all of those things, right? So I'll tell you, I talked to the French government quite a bit, and the French government will not accept that the digital diet of all their citizens be controlled by three companies on the west coast of the US. That's just not acceptable. It's a danger to democracy. Regardless of how well-intentioned those companies are, right? And so, and it's also a danger to local culture, to values, to language. I was talking with The founder of Infosys in India. He's funding a project to fine tune Lama 2, the open source model produced by Meta, so that Lama 2 speaks all 22 official languages in India.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  15. “It's very expensive and difficult to train a base model, right? A base LLM at the moment in the future might be something different. But at the moment, that's an LLM. So, only a few companies can do this properly. If some of those subsystems are open source, anybody can use them. Anybody can fine tune them. If we put in place some systems that allows any group of people, Whether they are. Individual citizens, groups of citizens. Government organizations, NGOs, companies, whatever, to take those open source Systems, AI systems, and fine tune them for their own purpose on their own data.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  16. “You speak different languages. So a lot of our interactions with the digital world are going to be mediated by those systems in the near future. Increasingly, the Search engines that we're going to use are not going to be search engines. They're going to be dialog systems that we just ask a question. And it will answer and then point you to perhaps appropriate reference for it Here is the thing we cannot afford those systems to come from a handful of companies on the west coast of the US. Because those systems will constitute the repository of all human knowledge. We cannot have that be controlled by a small number of people. It has to be diverse. For the same reason the press has to be diverse. So, how do we get a diverse set of AI assistants?”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  17. “Progress of ideas and even science, right? In science, people have to argue for different opinions. And science makes progress when people disagree and they come up with an answer and a consensus forms, right? And it's true in all democracies around the world. So there is a future. Which is already happening, where every single one of our interaction with the digital world will be mediated by AI systems, AI assistants, right? We're going to have smart glasses. You can already buy them from Mita, the Rayband Meta, where you can talk to them and they are connected with an LLM and you can get answers on any question you have, or you can be looking at a... Monument, and there is a camera in the system that in the glasses you can ask it, where can you tell me about this building or this monument? You can be looking at”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  18. “Different people may have different ideas about what constitutes bias for a lot of things. I mean, there are facts that are, you know, Indisputable, but there are a lot of opinions or things that can be expressed in different ways. And so you cannot have an unbiased system. That's just an impossibility. And so, what's The answer to this? And the answer is the same answer that we found in liberal democracy about the press. The press needs to be free and diverse. We have free speech for a good reason. It's because we don't want all of our information to come from a unique source because that's opposite to the whole idea of democracy.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  19. “I actually made that comment on just about every social network I can. Made that point multiple times in various forums. Here's my point of view on this. People can complain that AI systems are biased and they generally are biased by the distribution of the training data that they've been trained on that reflects biases in society and that is potentially offensive to some people. Potentially not. And some techniques to de-bias then become offensive to some people. Because of historical incorrectness and things like that. And so you can ask the question. The first question is, is it possible to produce an AI system that is not biased? And the answer is absolutely not. And it's not because of technological challenges, although they are technological challenges to that. It's because Bias is in the eye of the beholder.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  20. “I think it would be much more efficient to use it for planning. But currently it's used to fine-tune the parameters to the system. Now there are several ways to do this. Some of them are supervised. You just ask a human person, like, what is a good answer for this? And you just type the answer. I mean, there's lots of ways that those systems are being adjusted”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  21. “DH And then there are various ways to use human feedback, right? So you can ask humans to rate answers Multiple answers that are produced by WordMarl. And then what you do is you train an objective function to predict that rating. And then you can use that objective function to predict whether an answer is good. And you can backpropagate gradient through this to fine-tune your system so that it only produces highly rated answers. one way. So that's like. That means training what's called a reward model. So something that basically a small neural net that estimates to what extent an answer is good. It's very similar to the objective I was talking about earlier for planning, except now it's not used for planning. It's used for fine-tuning your system.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  22. “What's had the transformational effect is human feedback. There's many ways to use it, and some of it is just purely supervised, actually. It's not really reinforced by learning.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  23. “You don't want to do it for real because it might be dangerous, but you can adjust your world model without killing yourself, basically. So that's what you want to use RL4. When it comes time to learning a particular task, you already have all the good representations. You already have your world model, but you need to adjust it for the situation at hand. That's when you use URL.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  24. “There's two ways you can be wrong either your objective function does not reflect the actual objective function you want to optimize or your world model is inaccurate right so the prediction you were making about what was going to happen in the world is inaccurate so if you want to adjust your world model while you are operating the world or your objective function that is basically in the realm of RL. This is what RL deals with to some extent, right? So adjust your wall model. And the way to adjust your world model even in advance is to explore parts of the space where your wall model where you know that your wall model is inaccurate. That's called curiosity basically or play. When you play, you kind of explore parts of the state space that”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  25. “Yeah, now there's two things. You can use if you've learned a world model, you can use a world model to plan a sequence of actions to arrive at a particular objective. You don't need RL unless the way you measure whether you succeed might be inexact. Your idea of whether you were going to fall from your bike. Might be wrong, or whether the person you're fighting with MMA is going to do something and then do something else. So there.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  26. “I don't hate reinforcement learning, and I think I think it should not be abandoned completely, but I think its use should be minimized because it's incredibly inefficient in terms of samples. And so the proper way to train a system is to first have it learn good representations of the world and world models from mostly observation, maybe a little bit of interactions.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  27. “So there, the compatibility between two things is here's an image or a video, here is a corrupted, shifted, or transformed version of that image or video or masked. And then the energy of the system is the prediction error of the Representation The predicted representation of the good thing versus the actual representation of the good thing. So you run the corrupted image to the system, predict the representation of the good input uncorrupted, and then compute the prediction error. That's the energy of the system. So this system will tell you this is a good... This is a good image and this is a corrupted version, it will give you zero energy if those two things are effectively one of them is a corrupted version of the other. It gives you a high energy if the two images are completely different.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  28. “Indirectly, that gives it all probability to a high probability to sequences awards that are good and low probability to sequences awards that are bad, but it's very indirect. And it's not obvious why this actually works at all, but because you're not doing it on a joint probability of all the symbols in a sequence. You're just doing it kind of. Sort of factorize that probability in terms of conditional probabilities over successive tokens.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  29. “Similar way, but you have to have this way of preventing collapse, of ensuring that there is high energy for things you don't train it on. And currently it's very implicit in LLM. It's done in a way that people don't realize it's being done, but it is being done is due to the fact that when you give high probability to a word, Automatically, you give low probability to otherwise because you only have a finite amount of probability to go around. They have to sum to one. So when you minimize the cross entropy or whatever, when you train your LLM to produce the next word, you're increasing the probability your system will give to the correct word, but you're also decreasing the probability it will give to the incorrect words.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  30. “All no, it depends on how the internal structure of the system is built. If the internal structure of the system is built in such a way that inside of the system there is a latent variable, let's call it Z that You can manipulate so as to minimize the output energy. Then that Z can be viewed as a representation of a good answer that you can translate into a Y that is a good answer.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  31. “Yeah, so you can do this with language directly by just x is the text and y is a continuation of that text. Or X is a question, why is the answer?”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  32. “Regularizer, a criterion, a term in your cost function that basically minimizes the volume of space that can take low energy. And the precise way to do this is all kinds of different specific ways to do this depending on the architecture. But that's the basic principle. So that if you push down the energy function for particular regions in the XY space, it will automatically go up in other places because there's only a limited volume of space that can take low energy. By the construction of the system or by the regularizerizing function.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  33. “Gigantic But people do this. They do this when you train a system with RLHF. Basically, what you're training is what's called a reward model, which is basically an objective function that tells you whether an answer is good or bad. And that's basically exactly what. This is so we already do this to some extent. We're just not using it for inference, we're just using it for training. There is another set of methods which are non-contrastive, and I prefer those. And those non-contrastive methods basically say, okay, the energy function needs to have low energy on pairs of XY's that are compatible that come from your training set. How do you make sure that the energy is going to be higher everywhere else? And the way you do this is by having a”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  34. “Okay, how do you train a system like this at a completely general level is you show it pairs of x and y's that are compatible, a question and the corresponding answer. And you train the parameters of the big neural net inside to produce zero. Okay, now that doesn't completely work because the system might decide, well, I'm just going to say zero for everything. So now you have to have a process to make sure that for a wrong y, the energy will be larger than zero. And there you have two options. One is contrastive method. So contrastive method is you show an x and a bad y. You tell the system, well, that's give a high energy to this, like push up the energy, right? Change the weights in the neural net that confuse the energy so that it goes up. So If the space of y is large, the number of such contrasted samples are going to have to show.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  35. “Right. So then we're asking the question of conceptually, how do you train an energy based model? So an energy-based model is a function with a scalar output, just a number. You give it two inputs x and y. He tells you whether Y is compatible with X or not. X, you observe, let's say it's a prompt, an image, video, whatever, and Y is a proposal for an answer, a continuation of video, whatever. And it tells you whether Y is compatible with X. And the way it tells you that Y is compatible with X is that the output of that function will be zero if Y is compatible with X and we'll be a positive number non-zero if Y is not compatible with X.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  36. “They do it, they do this optimization in a horribly inefficient way, which is generate a lot of hypothesis and then select the best ones. And that's incredibly wasteful in terms of computation because you basically have to run your LLM for every possible generated sequence. And it's incredibly wasteful. So it's much better to do an optimization in continuous space where you can do grain and descent as opposed to generate tons of things and then select the best. You just iteratively refine your answer to go towards the best, right? That's much more efficient. But you can only do this in continuous spaces with differentiable functions.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  37. “Not really only in a very simple way. I mean, basically, you can think of those things as doing the kind of optimization I was talking about, except they optimize in the discrete space, which is just space of possible sequences of tokens.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  38. “Gradium based. He's gradient based inference. So now you have a representation of the answer in abstract space. Now you can turn it into text. And the cool thing about this is that the representation now can be optimized through gradient descent, but also is independent of the language in which you're going to express the answer.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  39. “It's an optimization process. You can do this if the entire system is differentiable that scalar output is the result of running through some neural net Running the answer, the representation of the answers to some neural net, then by gradient descent, by backpropagating gradients, you can figure out how to modify the representation of the answers who has to minimize that.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  40. “Representation. Abstract representation. So you have an abstract representation inside the system. You have a prompt. The prompt goes through an encoder, produces a representation, perhaps goes through a predictor that predicts a representation of the answer of the proper answer. But that representation may not be a good answer because there might be some complicated reasoning you need to do, right? So then you have another process that takes the representation of the answers and modifies it. so as to minimize a cost function that measures to what extent the answer is a good answer for the question. Now we sort of ignore the fact for the issue for a moment of how you train that system to measure whether an answer is a good answer for a question.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  41. “By minimizing an objective function basically, and this is, we're talking about inference, we're not talking about training. The system has been trained already. So now we have an abstract representation of the thought of the answer, representation of the answer. We feed that to basically on the two executive decoder, which can be very simple, that turns this into a text that expresses this thought. That, in my opinion, is the blueprint of future dialogue systems. They will think about their answer, plan their answer by optimization before turning it into text. And that is Turing complete.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  42. “Well, so really what you need to do would be to not search over possible strings of text that minimize that energy. But what you would do is do this in abstract representation space. So in sort of the space of abstract thought, you would elaborate a thought, right? using this process of minimizing the output of your model, which is just a scalar. It's an optimization process. So now the way the system produces its answer is through optimization.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  43. “Okay, so think of it as some gigantic neural net, but it's got only one output. And that output is a scalar number, which is, let's say, zero. If the answer is a good answer for the question and a large number, if the answer is not a good answer for the question Imagine you had this model. If you had such a model, you could use it to produce good answers. The way you would do is. Produce the prompt and then search through the space of possible answers for one that minimizes that number. Call an energy business model.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  44. “How do we get them to do this, right? How do we build a system that can do this kind of planning or reasoning that devotes more resources to complex problems than to simple problems? And it's not going to be autography prediction of tokens. It's going to be more something akin to inference of latent variables in what used to be called probabilistic models or graphical models and things of that type. So basically the principle is like this. The prompt is like observed variables. And what the model does is that it's basically a measure. It can measure to what extent an answer is a good answer for a prompt.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  45. “That's system one. So all the things that you do instinctively without really having to deliberately plan and think about it. And then there is other tasks where you need to plan. So if you are Not to experience a chess player, or you are experienced, you play against another experienced chess player, you think about all kinds of options, right? You think about it for a while, right? And you're much better if you have time to think about it than you are if you play Blitz with limited time. So this type of deliberate planning, which uses your internal role model, that's system two. This is what LLMs currently cannot do.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  46. “It's the same difference as the difference between what psychology school system one and system two in humans, right? So system one is the type of task that you can accomplish without deliberately consciously think about how you do them. You just do them, you've done them enough that you can just do it subconsciously without thinking about them. If you're an experienced driver, you can drive without really thinking about it and you can talk to someone at the same time or listen to the radio, right? If you are a very experienced chess player, you can play against a non-experienced chess player without really thinking either. You just recognize the pattern and you play.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  47. “Okay, whether it's difficult or not, the near future will say because a lot of people are working on. Reasoning and planning abilities for dialogue systems. I mean, if we restrict ourselves to language, just having the ability to plan your answer before you answer. In terms that are not necessarily linked with the language you're going to use to produce the answer. So this idea of the semantic model that allows you to plan what you're going to say before you say it. That is very important. I think there's going to be a lot of systems over the next few years that are going to have this capability. But the blueprint of those systems will be extremely different from auto-regressive LLMs.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  48. “Be exactly estimated. It's like, you know, it's the size of the prediction network with its 36 layers on 92 layers or whatever it is multiplied by number of tokens. That's it. And so essentially it doesn't matter if the question being asked is simple to answer, complicated to answer, impossible to answer because it's decidable or something. The amount of computation the system will be able to devote to the answer is constant or is proportional to number of token produced in the answer, right? This is not the way we work. The way we reason is that when we're faced with a complex problem or a complex question, we spend more time trying to solve it and answer it, right? Because it's more”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  49. “So the problem is that there is a long tail. This is an issue that a lot of people have realized in social networks and stuff like that, which is there's a very, very long tale of things that people will ask. And you can fine-tune the system for the 80% or whatever of the things that most people will ask. And then this long tail is so large that you're not going to be able to fine-tune a system for all the conditions. And in the end, the system ends up being kind of a giant lookup table, right? Essentially, which is not really what you want. You want system second reason. Certainly they can plan. So the type of reasoning that takes place in LLM is very, very primitive. And the reason you can tell is primitive is because the amount of computation that is spent per token produced is constant. So if you ask a question and that question has an answer in a given number of token, the amount of competition devoted to computing that answer can be”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source

  50. “Yeah, some people have done things like you write a sentence in English that has, and, or you ask a question in English and it produces a perfectly fine answer. And then you just substitute a few words by the same word in another language. I don't know if a sudden, the answer is complete nonsense.”

    2024-03-07 · Lex Fridman Podcast · #416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI · IDENTIFIED FROM THE TRANSCRIPT · source