YouSaid · the spoken record

Dr. Percy Liang

lines on the record: 50
first: 2023-03-09
most recent: 2023-03-09
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Of no open minded, I would say. I was asking earlier about what no priors, where that comes from. And I think it's a fitting way to think about the world because I think everyone, including scientists, often get sort of drawn into a particular worldview and paradigm. And I think that the world is changing both on the technical side, but also how we conceive of AI and maybe even humans at some level. And I think we have to be open-minded to how that's going to evolve over the next few years.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“I must say that my opinions about EGI have changed over time. I think that for a while, it was perceived by most of the community as laughable. I will say that in the last 10 years, I have been aware of, you know, there's a kind of a certain community who think about AGI and also existential risk and things like that. So I've been in touch with people who think about these. I think I see the world maybe differently. I think of perhaps certainly these are powerful technologies and could have extreme social consequences. And there's a lot of more near-term issues. I focus a lot on kind of robustness of ML systems in the last five years. But one thing I've learned about foundation models because of their emerging qualities, I've learned to be very kind.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Trying to produce a finished product and putting it out there. This goes back to the point about involving the spirit of open source and involving the community to build these foundation models together as opposed to someone unilaterally building them.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so we are developing a client that will be made available. We both from the perspective joining the together cloud so that you can contribute your compute, but also where we have an API that we're developing so that people can use together infrastructure to do inference and fine-tuning models. We are also training some open models. So we have this something called open chat kit that we're really seeing soon. And this is built on top of Eleuther AI's NeoX model, but improved to include various different types of capabilities. It's still, you should think about it as really a work in progress. What we've tried to do is open it up so that people can play with it, give feedback, and have the community improve this together rather than us.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“I like to think about nutrition labels or any sort of specification sheets on electronic devices. There's some sort of obligation, I think, that producers of some product should have to make sure that their product is used properly and has some bounds on it.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Noticing that, well, a lot of this is just kind of completely a black box. developing norms starting with transparency i think transparency is necessary but not sufficient you need some level of transparency to even have a conversation about any of the the policy issues so making sure that the public can understand how these models are are built at least some notion of like what the data is what are the instructions that are given to align the models we're trying to advocate for greater transparency there and i think this will be really important as these models really get deployed at scale and start impacting our lives you know a kind of a analysis”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, actually, I'm glad you asked that because we've been thinking a lot about the social implications of these models and sort of the, not the models themselves, which we focus a lot on talking about, but the environment in which these models are built. So there are a few players in this space with different opinions about how models should be built. Some are more closed, some are more open. There's also, again, this sort of lack of transparency where we have a model that's produced and it's aligned apparently to human values. But then once you start kind of questioning, you can ask the question, okay, well, which values, which humans are we talking about who determines these values, what legitimacy does that have, and what's the sort of accountability, then you start”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Sort of jailbreak your own model, but you can jailbreak other people's model and get them to do various things. And then so that could lead to sort of a cascade of errors. So some of these are the concerns that we hope to also capture with the model. I should also mention we're also trying to look at multimodal models, which I think is going to be pretty pertinent. So lots to do.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Increasing capabilities. Now they can do a lot more. They can write an email or give you life advice on XYZ if you put in a scenario or write an essay about XYZ. And I think what we need to do with the benchmark is also add the scenarios that accordingly improve or capture these capabilities, as well as kind of new risks. So we are definitely interested in benchmarking how persuasive are these language models, which governs and also how secure they are. One thing I'm actually also worried about is given all that the jailbreaking that is extremely common with these models where you can basically bypass safety controls if these models start interacting with the world and accepting external inputs. Now you can not only just”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“So that you could see the top level statistics and accuracies, but also you can drill down into on this particular benchmark, what are the instances, what are the predictions that these models are making all the way down to what prompts are you using for the language models. So the idea here is that we're trying to provide transparency to this space, right? We know that these models are powerful. They have some deficiencies. We're trying to lay that all out in a kind of a scientific manner. So I'm pretty excited about this project. The challenging thing about this project is since we put out the paper maybe three months ago, a bunch of different models have come out, including ChatGPT, LLAMA, Coherent AI 21 have updated their models, GPT-4 might come out at some point.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Focus on, but also issues of how robust it is, how well it's calibrated, meaning that does the model know what it doesn't know, whether the models are fair according to some definition of fairness, whether they're biased, whether they spear out toxic content, how efficient they are. And then we go and we basically grab every language model that's prominent that we could access, which includes open source models like OPT and Bloom, but also getting access to APIs from Cohare AI 21, OpenAI, and also Anthropic and Microsoft. So overall, there were 30 different models, 42 scenarios and seven metrics, and we ran the same evaluations on all of that. We've put all the results on the HELM website.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so Helm stands for Holistic Evaluation of Language Models, which is this project that happened over the last year. And the goal is to evaluate language models. So the trouble is that language models is a very generic thing. It's like saying evaluate the internet. What does that mean? The language model takes text in and text out. And one of the features of a language model is that it can be used for a myriad different applications. And so what we did in that paper is to be a systematically and as rigorous as we could in laying out the different scenarios in which language models could be used and also measure aspects of these uses, which include not just accuracy, which a lot of benchmarks”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“So there's foundation models is a pretty broad category of, and many of the sort of the core center is large language models that are trained on lots of internet data. We've trained a model here at CRFM in collaboration with Mosaic on a bot called Biomed LM. It's not a huge model, but it's trained on PubMed articles and it exhibits pretty good performance on various benchmarks. For a while, we were able to be state of the art on the US medical licensing exam. Google did come up with a model that was, I think, 200 times larger and they beat that model, so scaled doesn't matter. But I think there are many cases where efficiency reasons, maybe you do want a smaller model since cost, I think, is a big concern.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Many different ones for different kind of use cases as this space takes off. Many of them will be derived from maybe existing foundation models, but many of them will also be perhaps trained on from scratch as well.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Able to use the decentralized cloud for doing training when they're not using it. And when they are using it, they can use much more of it. So the hope is that it provides a much more efficient use of the compute because you're spreading across a larger set of people. And then on the commercial side, the hope is that the open models that are developed in the open source ecosystem can together platform can allow people to fine-tune and adapt these models to various different use cases. One thing I think is noteworthy is that we think of foundation models today as maybe there's a few foundation models that are very good and exist. But I think in the future there's going to be”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“So photography at home is, I think, a great inspiration for a lot of this work. At some point during the middle of the pandemic, they actually had the world's largest supercomputer in terms of flop count because it was used to do molecular dynamic simulations for COVID. The main challenge with foundation models is that there's a lot of big models and big data that needs to be shuffled around. So the task decomposition is much harder. So that's why many of the technical things that we're doing about scheduling and compression enable us to overcome these hurdles. And then there's the question of incentives. So I think there's two aspects of what together is building. One is sort of what I will call research computer, which is for academic research purposes where people can contribute compute and in the process of contributing compute.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Is a central bottleneck in foundation models. On the other hand, there's a lot of compute that's decentralized, that's maybe underutilized or idle. And if we could harness Then we could actually do a lot more. There are some pretty hefty technical challenges around doing that because foundation models are typically trained in very high-end data center environments where they interconnect between devices is extremely good. Whereas if you just grab your average desktop or home interconnect, it's 100 times or more slower. Chris Ray and Sejan and others really they did deserve most of the credit for this. We've developed some techniques that allow you to leverage this weekly connected compute and actually get pretty interesting training going. So hopefully with that type of infrastructure, we can begin to unlock a bit more of compute both for academic research but also for other startups and so on.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, it really depends on the gaps that you're seeing. Right now in academia, you can train one billion parameter models. I mean, it's not cheap by academia standards, but you can do it. And here at CRFM, we're training $6 or $7 billion parameter models. And I think it's enough to be able to try out some ideas. But ultimately because of emergent properties and importance of scale, you can only make a hypothesis. You can find something like, oh, this seems promising at smaller scales. You still have to go out and test whether it really pans out or the gap just closes. And maybe this is a good segue to talk about to compute together. So we found it together on the premise that compute was”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, you would not use Alice Ham. Transformer is strictly dominates an LSTM from the perspective of given a fixed compute budget. So this question of like, what if I could scale up LCM, it becomes a little bit sort of irrelevant?”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“What can we learn from transformers? What is it trying to do? And how can we incorporate them in a much more principled way? At some level, it's still going to be about compute, right? So people have shown that LSTMs, scaling laws for LSTMs show that if you're able to scale up LSTMs, maybe they would work pretty well as well. But the amount of compute is many times more. And given a fixed compute budget, we're always in a compute-constrained environment.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I really hope that in 10 years we won't be able using the transformer because I think the transformer is, I mean, it's a very good architecture. People have tried to improve it, but it's sort of like kind of good enough for people to press ahead. Scientifically, there's no reason to believe that this is the one. And there have been some efforts. So one of my colleagues, Chris Ray, and his students have developed other architectures which are actually at smaller scales competitive with transformers and actually don't require the central operation of attention. And I would love to see much more research exploring other alternatives to transformer. This is something, again, that academia, I think, is very well suited to do because it involves kind of challenging the status quo. You're not really trying to just get it to work and get it out there, but you're trying to reflect on what are the principles.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so I think there's pre training which is predicting the next word and developing a world model, so to speak. And with those capabilities, then you still have to say don't hallucinate, but it will be much easier to control that model if it has a notion of what hallucination even is.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“What's happening, and of course, with more context, then maybe it can use that context to actually know that, well, okay, well, I don't know. Maybe I should ask where he went.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Then you sort of have a model of what's happening in the world, at least the world that you've captured in text. And so while the notion of truth might be ambiguous in many cases, I think the model can get an idea of what certain parts of internet are maybe reliable and what parts of the internet are not and what kind of the idea of having entities and dates and locations and what activities there are. I think that will maybe become more salient in the model. Like if you think a model language model, that's just predicting the next word and it's only trained to do that. And you say Elad travel to. Of course it's going to mix something up without further context. But if it has a better understanding of”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Capability that I think is very striking The hope is that language models currently make stuff up, they hallucinate. And this is clearly a big problem. And almost in some ways a very difficult problem to crack. The hope is that as models get better, that some of this will actually go away. I don't know if that will happen. I guess the way I think about these models is that they're doing some sort of, if you think about predicting the next word, it seems very simple, but you have to really internalize a lot of what is going on in this context. What are the previous words? What's the syntax? Who's saying them? And all of that information and context has to get compressed. And then that allows you to predict the next word. If you're able to do this extremely well,”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“So, I can give you an example of something I think is emerging and I So, what we're seeing today is the ability to instruct a model using natural language to do certain things. You see a lot of this online with ChatGPT and BingChat where you can just and some of Anthropics work as well. You can instruct a model to be succinct, generate three paragraphs in the style of and so on. You can lay out these guidelines and have the model actually follow. So this instruction following ability is getting extremely good. Now I will say that how much is emergent and how much is not, it's hard to tell because a lot of these models, it's not just a language model that's trained to predict the next word. There's a lot of secret sauce that goes under the hood. So and if you define emergence of it was not intended by the designers, I don't know how much of that is emergent, but at least it's a case.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Because up until now, again, remember the AI dream tops out at humans, but now we can actually go beyond in many ways. And I think that unlocks a lot of possibilities.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Know much, much better than what many people could come up with, which means that it has learned different concepts of what Shakespeare and what Quicksort are and is able to fuse them. So if you think about creativity, I think this is sort of an example of creative use. People say that sometimes all language models just memorize because they're so big and train on clearly a lot of text, but these examples, I think, really indicate that there's no way that these language models are just memorizing because this text just doesn't exist. And you have to have some creative juice and invent something new. I think to kind of go on riff on that a little bit, I think the creative aspects of these language models with the potential for scientific discovery or doing research or pushing the boundaries on what humans can do, I think is really, really fascinating.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“So, the idea is if I have a question that's presented to a language model, the language model could just answer and it will maybe get it right or wrong. But if you ask a language model to generate an explanation of how it would solve the problems, kind of thinking out loud, then it's much more likely to get the answer right. And this is very natural that it would be the case for humans as well, but the fact that, again, the chain of thought, the capability is something that emerges. The other thing I think is really wild is this, and I think it's maybe a general principle, which is the ability to mix and match. So you can ask the model to explain the quick sort algorithm in the silo of Shakespeare, and it will actually construct something that is semantically pretty on point, but also stylistically.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“I think going back to GPD3, I think in context learning is something that surprised many people, including me. So here you're prompting a language model with an instruction and input-output pairs. Here's a sentence. It's classified positive. Here's a sentence to classify negative. The model is somehow able to latch on to these examples and sort of figure out what you're trying to do and solve the task. And this is really intriguing because it's emergent. It wasn't hand-coded by the designers to, oh, I want to do in context learning this way. Now, of course, you could have done that, but I think the real sort of magic is you didn't have to do that and yet it still does something. It's not completely reliable, but it's sort of can get better with better models and better data. Then there's chain of”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Transformers are fixed, and there's ways to extend it, but fundamentally it's sort of a fixed model. Let's say advanced problem solving, for example, if you want to solve a math problem, you want to improve something. The language model generates sort of this chain of thought and generates token by token, and then it generates something. But we know that humans, when they solve a problem, it's much more You try different things, you backtrack, it's much more flexible, iterative. And it can last a lot longer, and then just going for a few iterations. And what is the architecture that can handle that level of complexity, I think is still an outstanding question.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that it's interesting to remark on what's happening because there are to a first order approximation larger models trained on the relevant data seem to do well on various benchmarks. I think that maybe there isn't enough emphasis on data efficiency and how quickly you can get and how robustly you can get to these points because we know it has been well documented that benchmarks can be gamable so even though you do want to benchmark doesn't mean you've necessarily solved the problem. So I think one has to be a little bit cautious about that. So obviously scale and more data is just one clear direction. But in terms of orthogonal directions, what are the methods several things have to happen? One is we have to have ability to handle greater context lengths. If you think about a long reason”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Maybe it's natural because there's certain things you can do in your head, certain things you can invoke a tool to use. But this has been also one of the classic debates in AI. There's neural versus symbolic. And for a while, symbolic AI was dominant. Now neural AI has come really taken off and become dominant. But some of those central problems of how do you do planning, how do you do reasoning, which was the focus and study of symbolic AI, are now again really relevant because now we've moved past just simple classification and just entity extraction, but now more to more ambitious tasks.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“To more like 2020 generation of these large foundation models. So I think there's certainly a place for that type of thinking. There are cases where you want to just map natural language into say people call it tool use, like you ask some question that's calculation. You should just use a calculator rather than try to sort of quote unquote do it in the transformer's head. But there's also a lot of aspects of reasoning which are not quite formal. We do this all the time. And a lot of that happens kind of natively in the language model. And I think it's still an interesting question how to kind of marry the two. I feel like the two are still in sort of jammed together in a way.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“And we developed the squad question answering benchmark to fuel progress in open domain question answering. And that in turn and many other data sets that were developed both at Stanford and elsewhere, I think led to the development of these powerful language models that then like Bird and Roberta and Elmo back in around 2018 to then many years ago. Ancient history now.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Natural language questions into essentially SQL queries, which obviously has many different applications as well. And what was nice about this framework is that to really do this, you had to understand how the words contribute to different parts of the SQL query. And then you could get something that was a program that you could execute and you deliver the results as opposed to many question answering systems, which you ask a question, maybe retrieve some document, you're retrieving the answer, or either that or making something up rather than computing it rigorously. So that was a paradigm I was working in maybe five to ten years ago. But the main problem is that the world isn't a database. A small part of the world is a database, but most of the world is unstructured. And then I started thinking about question answering.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“So computational semantics is the process where you take language, text, and compute unquote meaning from it. And that is something I'm not going to maybe try to attempt to define. There's a huge literature of linguistics and philosophy about what meaning is. I would say that a lot of my research in the past, maybe five to ten years ago, was adopting this view that language is a programming language. It computes. You can give orders, you can instruct, you can do things with language. And therefore, it was natural to model natural language as a formal language. So a lot of semantic parsing is about mapping natural language into a formal space so that machines could execute this. And so one concrete application of this that worked on for a while is”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think that's a great point for ages, human level has been the target for AI. And that has really been kind of a North Star that has fueled many dreams and efforts and so on over the decades. But I think we're getting to a point where many axes, it's superhuman or should be superhuman. And I think we should maybe define more of an objective measure of like what we actually want. We want something that's very reliable, is grounded. I often want more statistical evidence when I speak to doctors and sometimes fail to get that and have something that would be sort of a lot more principled and rational. This is more of a general statement about how we should think about technology, not just chasing after mimicking a human because we already have a lot of humans.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think it's a good question. I think there are a bunch of different issues that need to be resolved. For example, foundation models are trained on a lot of data. How do you deal with privacy? How do you deal with robustness? Because once you're talking about in the healthcare space especially, there are cases where we know that these models can still hallucinate facts and sound very confident in doing so. How do you see doctors like that too? Yeah, there you go.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Risks of disinformation, monitoring to what extent these tools are so persuasive, which they are getting increasingly. So, and what are the actual risks when it comes to, let's say, foreign state actors leveraging this technology? And there's also people at the center who are in medicine, and we're exploring ways of leveraging Foundation models and deployment in actual clinical practice.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“And this is going back to the question about what is CRFM's role. CRFM is a center with over 30 different faculty across 10 different departments at Stanford. So it's quite interdisciplinary. So we're looking at foundation models not just from a technical perspective of how do you get these models to work, but also thinking about their economic impact, the challenges when it comes to copyright and legality. We're working on a paper that explores some of those questions. We're looking at different questions of social biases and thinking through carefully how the impact of these models have on issues of homogenization where you have a central model that's making perhaps decisions for a single user across all the different aspects. So some of these are the types of questions. There are also people at the center looking at”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“I mean, I think industry and academia. And now today, I think it's the dynamic is quite different because it's no longer Academia's job isn't just to get things to work because you can do that in other ways. There's a lot of resources going into tech companies where if you have data and compute, you can just sort of scale and blast through a lot of barriers. And I think a lot of the role of academia is understanding because these models for all their impressive feats, we just don't understand what they work, how they work, what the principles are, how does this training data, how does moral architecture affect the different behaviors, what is the best way to weight data, how do you, what's the training objective, many of these questions, I think could benefit from a more rigorous analysis. The other piece, which is a different type of understanding, is understanding social impact.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“I think that this is very natural because these models take a lot of capital to train. There are enormous amount of congenering a lot of value and it's a competitive advantage. So incentives are to keep these under control. There's also another factor, which is safety reasons. I think these models are extremely powerful. And maybe the models right now, I think are, well, if they were out and open, it would be maybe okay. But in the future, these models could be extremely good and having them completely anything goes. We might have to think about that a little bit more carefully.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“And whoever just getting involved. And what we're seeing now is sort of a retreat of that open culture where models are now being only accessible via APIs. We don't really know all the secret sauce that's going behind them. And they're sort of limited access.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so the Center for Research on Foundation Models sorted two years ago is under the Human Centered AI Institute at Stanford. And the main mission of the center is, I would say, to increase transparency and accessibility to foundation models. So foundation models are becoming more and more ubiquitous, but at the same time, one thing we have noticed is the lack of transparency and accessibility of these models. So if you think about the last decade of deep learning, it has profited a lot from having a culture of openness with tools like PyTorch or TensorFlow, data sets that are open, people publishing openly about the research. And this has led to a lot of community and progress, not just in academia, but also in industry with different startups and hobby.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“A general substrate where you can ask the model to do various things. And the idea of a task, which is so central to AI, I think, begins to dissolve. And I find that extremely exciting. And that's the reason later in 2021, we founded the Center for Research on Foundation Models. We coined the term foundation models because we thought there was something that was happening in the world that somehow large language models didn't really capture the significance. And it was not just about language, it's about images and multimodality. It was a more general phenomenon. And we coined the term foundation models. And then the center started. And it's been sort of a kind of a roller coaster ride ever since.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, there was a very decisive moment, and that moment was when GPT-3 came out. That was in the middle of the pandemic. And it was just, it wasn't so much the capabilities of the model that shocked me, but it was a way that the model was trained, which was basically taking a massive amount of text and asking the model to predict the next word over and over again, billions of times. What rose from it was not only a model that could generate fluent text, but also a model that could do in-context learning, which means that you can prompt the language model with instructions, for example, summarize this document, give it some examples, and have the model on the fly in context figure out what the task was. And this was a paradigm shift, in my opinion, because it changed the way that we conceptualize machine learning and NLP systems from these bespoke systems where it's trained to do question answering, to train to do this, to just”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Not a way that I would necessarily expect, but with it coming out of large language models such as GPT-3, it's truly kind of astonishing how much of the structure of language and the world that these models can capture. In some ways, it kind of harkens back when I actually first started in NLP, I was training language models, but of a very different type. It was based on hidden Markov models. And there the goal was to discover hidden structure and text. And we were, I was very excited by the fact that it could learn about tease apart what words were like city names versus days of the week and so on. But now it's kind of on a completely different level.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so I've been in the field of machine learning and natural language processing for over 20 years. I started getting into it in undergrad. I was undergrad at MIT. I liked theory. I had a fascination with languages. I was fascinated by how humans could just be exposed to just strings of text, I mean speech, and somehow acquire very sophisticated understanding of the world and also syntax and learn that in a fairly unsupervised way. And I wanted to, my dream was to get computers to do the same. So then I went to grad school at Berkeley. And then after that started at Stanford. And ever since I've been pursued of developing systems that can really truly understand natural language. And of course, in the last four years, this once upon a time kind of dream has really taken off in a sense. Maybe in a”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source
“For ages, human level has been the target for AI. And that has really been kind of a North Star that has fueled many dreams and efforts and so on over the decades. But I think we're getting to a point where many axes, it's superhuman or should be superhuman. And I think we should maybe define more of an objective measure of like what we actually want. This is more of a general statement about how we should think about technology, not just chasing after mimicking a human because we have a lot of humans.”
2023-03-09 · No Priors · What is the role of academia in modern AI research? With Stanford Professor Dr. Percy Liang · IDENTIFIED FROM THE TRANSCRIPT · source