YouSaid · the spoken record

John Jumper

lines on the record: 55
first: 2024-05-01
most recent: 2024-05-01
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Which previously were not there. Of course, there are challenges with understanding uncertainty and sort of hallucination and all these sort of technical problems need to be addressed. But once that is done, I think the impact that's going to have on models for scientific discovery would be amazing. That's another reason to be excited for the future.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“I think one sort of thing that has been very exciting in the last few years is the rise, of course, there's a lot of excitement about LLMs and foundational models and so forth. And if you look at the impact that's going to have on science, now most of the projects that I was talking to you about, we were working with structured data, data either which was collected or in the case of some of our fusion work, data that was simulated. But with the rise of foundation models and LRMs, that opens up the possibility of now using unstructured data to feed these models. And so that really opens the door for a large scale ingestion of scientific knowledge into the models. And that is a very exciting direction that will, I think, bring a number of other problems now in the feasibilities.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“End point for me is like as we talked about, we're kind of in the middle of this journey and this technological journey, this cultural journey, these cultural shifts, and that it's going to feel like the big goals that I've laid out, let's say a clinical trial things and systems biology, that's so far.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Stable compounds, which expands the number of stable compounds known by more than order of magnitude, right? But how do you now take those sort of compounds and then reason about their specific properties that would be useful in a particular application? So in any of these disciplines, we are not targeting one specific milestone. We are just saying here is a topic and the long-term sort of roadmap is to bring about a paradigm shift in how science is done in that area and move towards a more rational modeling-based approach and tackling some of the problems that are encountered here. So there's a lot that needs to be done and we are just trying to focus on specific areas and then new areas come up if the raw materials are there in terms of data and if we are clear on the valuation metric we are constantly reviewing them as well.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“I think what is fascinating about science and in any of these fields is that there's so much more to work on. I mean, even the one structural prediction, I just mentioned that the latest version of APAFOL, the work there is on extending it to general biomolecules like DNA, understanding RNA, understanding the interactions between small molecules, begins, and proteins, like bigger complexes, antibodies. There's so many things that we can extend in genomics. We have worked on both gene expression, the coding part of the genome, like with the Wisconsin variants, and the non-coding part of the genome, right? Or like predicted gene expression, we have made progress, but we are not completely at the end of it, right? So there's a lot that we are doing in all these areas in material science mentioned this model gnome, which was able to predict 400,000 novels.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Think that's where there's really going to be a tipping point and a point where we can just move much, much more rapidly, where we can sort of not get stymied with having to run these animal models, which takes a long time and is very expensive. And even there's crazy things. Like right now, there's a monkey shortage because monkeys are in such high demand to run these experiments. So I think there's probably a long road to get there where these models of humans are more predictive than the alternatives. But I think once we get there, that will be a major inflection point.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Fun thing about CASP, this critical assessment of structure prediction is that I think it also inspired all these other prospective trials and prospective studies. So there's a ton of that stuff to do. And I think there's tests for predicting binding of small molecules. I think we'll see in time these types of methods do extremely well in those assessments. But the Holy Grail is, in my mind, being able to predict clinical trials. It's something where to understand how a drug works in human biology. And that pushed me to the point is that's a systems biology problem at the largest scale. And so that is the holy grail. And I think we'll probably do it in parts. You could imagine even like models for specific organs or models for specific parts of the body, and then we put them together. Mixtures of experts is pretty common these days, and maybe that would be one approach. But however it gets done, once that gets done to the point where these models are better than the animal models,”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“To highlight the data issue, I think one of the biggest differences between AI for, let's say, language models or AI for video and AI for biology or for healthcare is that I think most of the interesting data in biology and healthcare is either dark, that there's all these medical records and so on, that you just can't access on the internet, which would be very useful for understanding the healthcare side, for trial side, and so on. It's either dark or it's never been measured and that we need to do the experiments. I think having the data could be paramount. And that I think that's going to be different than other places. But other places, maybe the algorithms can really drive things because everyone has the same data more or less. I think here people will be differentiated by their data. And so the innovations will be innovations in AI combined with innovations and data collection. And there are obviously things at that interface where active learning and how can you use the data more efficiently and so on.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“The problem, we are confident that we have a good evaluation metric to track progress. And we have the raw material, the data or a simulator to get good data. Only then do we make that long-term commitment towards a specific topic.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“One specific area that I would love to have impact on, right? And I think AI would eventually have impact on is systems biology. It's an incredibly important sort of problem to really understand at the system level how biological systems behave. It's just the data and the evaluation is not at a place where it is for maybe genomics, functional genomics or for structural biology. Before we actually start an initiative in any of these areas, there is a huge due diligence process that we need to undergo because essentially you are making a very long-term commitment and the careers and the impact of some of the best scientists and engineers that we have are being committed to that area. So we take that responsibility very seriously. And only when the impact, when we are confident of the impact”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Or a biologist would be interested in. So we just released the predictions rather than the model because the model had many other sort of users. You could run it on different organisms. There were other sort of commercial considerations. So it was felt that we could release the predictions, we could share the methodology, but we will not sort of open source the approach.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“There are a number of different factors, both what will be the social impact, the scientific impact of releasing things versus what is the commercial cost of releasing something while leveraging it for commercial purposes or even the safety sort of argument. So just to give you an example, one of our recent models that we announced last year was alpha medicines. And this is a model for predicting effect of misense variants. And what the model does, it produces state-of-the-art accuracy in making predictions about whether Missins variants are denied. or could be pathogenic. And in this particular case, we felt that the predictions of the model for the human genome, for the human misense variant like the 71 million of them, if we release that, that would serve most of the purposes that a clinician”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“by the impact that our whole two has had in the community. Now, of course, that's not true for all the different models. In fact, subsequently, we have had models, which we have not open sourced. But I think in the case of alcohol too, the decision was very, very clear in favor of sharing it with the world in the most free way possible.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“We would not have come up with, right? If we had kept it closed, someone basically interacting with the model in the community figured that out. So when we were thinking about it, there was, of course, how to maximize social impact and scientific impact of the model. The second one was responsibility. And we consulted a number of experts from structural biology, from chemistry, from drug discovery to figure out what is the right and responsible and safe approach here and even considering the malicious sort of use cases. And after we had done all the due diligence that we felt that this was safe to release and the impact of releasing it and open sourcing it in a wider sort of way would outweigh any costs that we would need to sort of model. It was decided that we should open source it. And I think the decision has been”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“There was a lot of deliberation within the team and within the company on this. I think there were a few different things that went into that final decision. One was we wanted to ultrafold was that foundational there. It was so foundational. If we had kept it closed source, the impact of it, like fully leveraging the impact for society, I mean, that would have been difficult. It was because it's so fundamentally sort of foundational and it's very hard to even predict what are the potential sort of applications of it. Just to give you an example, when we launch Alpha Cold, a couple of days later, somebody did an analysis on the uncertainty associated with the alpha-fold predictions and figured out that in fact alpha fold was even though it was not trained for that was the best predictor for predicting disorder in proteins. So that was something that”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Hopefully, not be toxic at all. Turns out to be toxic. So it's actually very easy to make toxic things. And Google will teach you actually how to get rice in and how to get all this other stuff for better or worse. So I think there, the asymmetry is that if we get rid of AI for drug design, you lose all the good and you don't prevent any of the bad, which is already here.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Parts won't be open source. I think you unfortunately can't open source a drug compound because then no one's going to pay for the trial and certain things like that economics doesn't make sense given these hundreds of millions of billions of dollars and so on. So certain parts will be closed source and there's hundreds of startups and AI in biology and AI drug design that will maybe take advantage of what's been done, develop their own methods and build on top and then that's where I think the drugs will come from You talked about also the concern for how, because this is so powerful we could maybe do sort of dangerous things with it and that's where I think there's a bit of a misconception because actually there's a huge asymmetry between the complexity of drug design for treating disease and that's a really hard problem to do but actually turns out to be really easy to come out with chemicals that actually are dangerous and toxic. In fact, that's why we have phase one trials because like even the things that you thought would really”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“I think the beauty of open source, and we see this open source for AI and biology, but AI more broadly, is that people could build on top of each other. And I think what's really remarkable about the AI field, I would say over the last five, maybe possibly 10 years, is that it feels like an amazing result comes out like once a week. And that the key part of that is that it comes out with code or GitHub repo. And that you can check out immediately. You don't even have to just believe the results. You can run it yourself. People have even open sourced various tests of things. So essentially we're building like a skyscraper. Each person builds a new floor and we're going up really fast. And that's what open source can do. In the past, if it wasn't open source, I'd have to read the paper. I'd have to code it myself. And sometimes the paper may be a little vague for some detail. So I might not bother, right? And I'll just go do my thing. And so I think what open source allows us to do is to build on top of each other and build rapidly.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“How do you do the ligand design? How do you solve the co-holding problem? At the same time, we are continuing and work on the foundational sort of side of things and have now released an announcement or an update on the next generation of Alpha Fold, which goes beyond proteins to other biomolecules, to nucleic acids like DNA, RNA, PDMs, small ligands, and so on.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so for the Alpha Pole database, it's literally free. You just go to the Alpha Pole database, sort of find the protein that you're interested in out of the 250 million proteins and lick it. And it's there. It's for free for everyone on the planet to use. So really, it has democratized things in a way that scientists in Latin America or India who is working on sort of neglected tropical diseases, for instance, who had no way they could get a structural of a protein that they were interested in can now get access to these structures at the sort of click of a button. Of course, a lot of research needs to be done to take that work and towards a more focused outcome and a lot more investment is needed if you are trying to finish and accomplish the vision that Vijay sort of outlined.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Expectation is that AI for biology and understanding targets and so on based on human data, that would also help on the trial side in addition to anything else there. So I think put together, I think we can get to these therapeutics faster, cheaper, and hopefully better.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“One of the sort of fantasies that one of my former colleagues talked about was what we call beach biotech, where you have, let's say, one person and a laptop, presumably on the beach or wherever you want to be. And you've got CROs, these contract research organizations, to do the experiments. You have some AWS cloud or whatever, some GCP cloud somewhere to run your calculations, and that one person with AI, I think we're not quite there yet, but I think that's an intriguing fantasy to think about. And I think on the way to the one person sort of aspiration is smaller teams doing way more with much less capital outlays and building startups, I think, much more efficiently and where they get to results much more rapidly. The challenge is going to be I mentioned before, is that the getting to the clinical trials, speeding that up will be nice, but I think the big financial return will be on the clinical trial side. But I think the...”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“I think there's also just within biology there's become a shift that I think people are sort of wrapping their heads around prediction a bit better. I think before experiment was the gold standard and that was all people want to hear about. I mean part of it's also just the zeitgeist of the time when you deal with large language models, you're basically dealing with predictions of what comes. And I think people have understood the pros and cons of predictions, but that there's massive value in having it. And I think it's, you know, it's funny that we talk so much about the technology, but I think it's the human shifts and the cultural shifts are the things that we're going to really need to push. And I think what gets me most excited about what push me has just been talking about is that the fact that I think that's the sign that we're seeing this cultural shift as well.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Which was using sort of Alpha Fold to develop and think about enzymes that could decompose plastics. So you have this whole spectrum of fundamental biology, drug discovery to even synthetic biology and enzyme development that has been impacted by AlphaFold. And so it was very difficult to even predict what would be the uses of the tool.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Of the impact, it's again like an amazing sort of spectrum. We saw alpha fold being used in bath breaking, fundamental biological discoveries. Like my personal favorite in that domain is the nuclear pore complex, the structure of basically the pore complex, like the way a nucleus controls how a material gets into the nucleus and out. I mean, that fundamental structure of that complex was not known. And researchers used alpha fold two structures to be able to piece together the whole complex. A recent paper from the Feng Lab showed how you could develop a molecular syringe. And again, they used alpha pole to in designing that. And there are so many other sort of areas where people have been using it for developing new vaccines and working on new antibiotics against antimitrobial resistance. and synthetic biology like one of the key partners at the early stages was a university paired in the uk”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so this was another sort of fascinating journey of growth. As I told you, I was not from the natural sciences. So working on Alpha Hold was a learning experience, but then actually releasing Alphafold to the community was even a bigger sort of a learning experience. So Alphafold database, when we were sort of building it up, we wanted it to be available everywhere in the planet to all the sort of scientists. But the scale of science was unprecedented. I was not aware of it. The Alpha Fole database today has been accessed in 190 countries. And there have been 1.6 or 7 million users of the alcohol database. Now, if that is not a positive statement about the planet, then I don't know what it is. There are 1.7 billion people interested in protein structure prediction. I'm really happy about that. I mean, all the things that are happening in the world.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and what can we do? I mean, one of the most amazing things, I think, is that DeepMind, for the most part, has given these models or the results of them to the community. And so researchers have their hands on them. And so maybe we could talk about that. How are researchers leveraging these new breakthroughs? There's all kinds of stats around we don't have enough cancer drugs or there are shortages and those are very real things we want to fix. So push me, maybe we'll start with you. What are you seeing and your team seeing in terms of this technology being deployed and how are researchers using it?”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“And we saw with Go and we saw with all these other things. So I think that's just a cultural shift. But I don't think that's a bad thing. I mean, forklift can lift much more than the strongest weightlifter. And we view that as a positive thing. It's always going to be us and them. And I think the interesting question will be is once it could do these things that we can't do, well, what do we do together with that?”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“I think amongst biologists, especially maybe 10 years ago and further back, I think there was often a belief that biology is just so complex that it's just incomprehensible that there's no way to even understand it. The only thing you can do is run the experiment and see what happens.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“We work with mathematicians with then not only make that conjecture but actually prove that there was a very elegant nice relationship between those two characterizations. So this is like completely fundamental discoveries in mathematics that were completely unknown to mathematicians now being uncovered by a machine learning and AI model. And we are seeing this across the board in any of the scientific areas that we are looking at. We are discovering new insights, new sort of patterns that were not expected just because the techniques to analyze the raw scale of data did not exist.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Definition. And mathematicians understood these categorization, but never understood the connections between that, right? And what we showed in one of our sort of works is basically we generated a lot of data for knots in these two characterizations and somehow asked the neural network, can you make predictions about one characterization from the other? The idea was well the answer should be no. But in fact, it couldn't make predictions, and when we drill down, we found a very nice conjecture that nobody had encountered.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Essentially, what we have entered is basically an age where a single human mind cannot comprehend the data that we are gathering about the universe. And this is true in any field you now encounter. It is true in biology. No biologists can reason and analyze all the biological data that is being gathered. No physicists can look at and analyze all the high energy physics data that is being gathered. And even mathematicians cannot sort of look and analyze all the large scale mathematical simulation data that we can now compute and simulate and find out. And I think what's happened is AI is not sort of nice to have. It's basically almost a necessity for us to make sense and reason about any problem that we are now looking at. I have examples in pure mathematics where work on topology, you describe a knot in two different sort of definitions. There is an algendric definition and there is a”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Definitely. We talked about Alpha Fold, which is very exciting and maybe the most familiar to folks, but Pashmit, your team has also created a bunch of other papers that touch this intersection of AI and science, or you could say AI and math or AI and physics. And those are things like materials, graph cast, which has to do with weather forecasting, fun search, alpha geometry. And so I'd love to hear from you again on this video. Probing of are we moving the frontier forward with these different models? What are you seeing from some of these other projects? Your team is working on in terms of AI helping us actually uncover new science.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“As a whole area. And then finally, I think AI for clinical trials is going to be really where maybe the biggest impact financially will be clinical trials could cost hundreds of millions to billions of dollars. Even a 10% improvement on a billion dollar enterprise is huge. And that's where maybe some of the toughest problems to work on. But I think as we make impact there, I think clinical trials will be better, will be probably more easily powered, and will be hopefully more successful because we'll be picking the right ones to do. And then that turns into eventually AI for personalized medicine, which is in a sense the extension of that trial. And so we're now, I don't want you to do an experiment on me as a mouse or a rat, but I would love to make sure I get the best drugs for me. And you and I are different and will respond different to drugs to be able to have that predicted will be huge. So I think there's the arc of that. And I think we're just at the very beginning.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Pushmeat said well that structure prediction is a foundational problem, but if you take, for instance, just the sort of arc of drug design, where first you have to come up with understanding the biology, the AI for biology is a very interesting area where we can maybe start to understand the nature of pathways and do this on human biology in ways that don't require experiments on human beings, which has always been one of the biggest limitations. I think we understand mouse biology really well because of all the experiments we can do, but we could never do that on human beings directly. But AI models for humans, as they become more predictive, and especially this more predictive than a mouse is predictive of human, the mouse is a model in a sense, that gets super interesting for unraveling biology. And so AI for biology is a thing. We can talk about AI for chemistry, and I think Alphafold is in that category where now we're trying to understand biophysical chemistry, we want to try to understand how can we quickly drug on drugable proteins, how can we come up with new antibodies and design proteins.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“He's so good. But apparently there was one particular sort of scientist who had submitted a protein but did not know the structure. They had hoped that the structure would be obtained by the time the competition ended, but this structure was not known to anyone, literally anyone. And Alpha Fole could give them an initial starting point which can solve the structure for that particular protein. So they were totally amazed that such a system now existed in the CAS competition and we later on sort of released alpha fold and not only was it very accurate it was also very efficient. So we decided to in fact find the structures for almost all the proteins that are known to scientists around 250 million of them and put them in a database with our partners, the European Microbiology Laboratory, MEBI, and then made that as a resource that anyone can access.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“When the second CASP competition ended, we got this email from the organizers who wanted to chat. And that was unprecedented. We were sort of surprised, why did the organizers want to sort of chat so early on? And they were super surprised at how good the predictions were. In fact, some of them speculated. Maybe the team has cheated in some way.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“The actual importance of the problem because we were all sort of sitting in our homes sort of shielding and there were scientists out there who said if you have the structure of the different SARS-CoV-2 proteins, it would be really helpful. Now the community very quickly found the structure of the spike protein because it also was sort of similar to SARS code 1. But the SSLE proteins of the virus the structure for those was not known. And so the fact that we could compute these predictions, share it with experts who are trying to deal with the pandemic and think about designing inhibitors and so on, it really brought to the team the real world impact and relevance that this fundamental problem has and around September 2020.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Cross that AT GTT sort of threshold, then that was like unprecedented. And of course, that also motivated us to push it even further and later on to a 90 GTT and beyond, right? Which we thought is what we needed to do. And so the pandemic happened and it really sort of brought home to the whole team.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“So out of all two, we start this long journey where we start making progress on Alpha Fold 2 with a much lower sort of performance from Alpha Fold 1 even. We have this internal leader board where everyone in the team can propose ideas and try out their ideas on a central leaderboard to see how much of a delta each idea or each change sort of makes. And we were making steady sort of progress. And then there were times where progress would stagnate and sometimes even for months it would stagnate and people would ask the question, well, have we reached the limit? But over time, and I think around when the pandemic started, we caught some really, really big deltas where we thought we are making real progress. And if you look at the metrics as to how do you quantify protein structure prediction accuracy, it's called GTT. And we had.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Able to properly train the model, we need it end to end. We needed a model which could go directly from the sequence to the structure. And that was one critical sort of element and a change that needed to be made, but it was a difficult change to make because you are starting from a much lower baseline when you are sort of building up that second end-to-end network.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Also Fold I had adopted a classical approach. This classical two-stage approach, what the machine learning model's job was, it given a sequence, it does not predict the 3D coordinates of the amino acids directly. What it predicts is basically the distance between amino acids. And then there's the second stage, which was supposed to take that distance matrix and recover the 3D coordinates. So the machine learning neural network's job was restricted to find the distances between amino acetues. And this two-stage sort of model was very effective, but it was not very elegant in the sense that if you made certain errors, you will not be able to backpropagate back to the neural networks. Because you found the results after the second stage and the neural network would not get that supervision. So we believe that in order to be”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“What gave you the indication that alpha fold one couldn't take you to the next level? Because I think even in the AI space outside of science, there are a lot of questions around can we just depend on the scaling laws? Do we need some sort of new unlock to get to, you know, insert problem here, could be AGI, could be something else? What gave you the indication that this was great? We're so happy with our results, but we actually need to throw this out and start anew.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“We started around 2017 and we took part in the critical assessment at the end of 2018. And when we entered Alpha Fold 1 in 2018, we were not really sure where would it be And that validated our sort of hypothesis. The basic research philosophy at DeepMin has been the multidisciplinary nature of the teams. So we had brought in some really good structural biologists and biophysics people, John Jumper being the lead of Alpha Fold, was part of the team at that time. And that gave us a lot of confidence. Now we were the best in the world, but the model was still not useful, right? It was producing good results, but it was nowhere close to solving the problem. And then we had to sort of make a bet. Can we really go after it and solve it once and for all? Or this is it. And so the first thing we had to do was start from scratch. We had to throw alpha fold one from the table and said this approach that we have started is not going to work.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“These models are extremely good at sort of cheating. And if you give them any sort of way to cheat, they will cheat. So the protein folding community and the protein structure prediction community had this annual biannual sort of competition called CASP, the critical assessment for structure prediction. And they would run this blind assessment, like Olympics of protein structure prediction where people would be given protein sequences whose structure was not known by anyone. Only like one experimentalist who has deposited it and then they would be tested. And the true generalization ability of the model would be exhibited. So we thought this problem really checked a number of key criteria which we use for taking up a problem for the very long term. So we started with a team which investigated how much progress we can make on this. We were hopeful, optimistic that machine learning.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Cases would diligently deposit that 3D structure in this database. And so at that time, when we started that field, 150,000 odd structures, both from X-ray crystalography and cryoem. And that was like an amazing sort of data set to start with. And not only that, the other big problem in machine learning as to how do you evaluate the machine learning model. Because in machine learning, one of the easiest things that you can do is basically fool yourself.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Started working on this problem because we thought it sort of satisfies one of our key requirements when we look into problems. That is its real foundational root node problem. Once you solve it, it has so many different implications, disease understanding and biology and so synthetic biology as well. And not only that, it is a classic sort of machine learning problem. You require reasoning in this problem because you are working with a very expanded solution space as well as you have access to raw material which is data. And the structural biology community had done an amazing job in sort of curating a very good data set in the form of the PDB. So scientists all across the world had whenever they found the structure of a protein which sometimes took almost five years or even a decade in some cases.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“So Al-Shufur was telling how I started my journey with the science program at DeepMind. And at that time, we had these two small scale sort of projects. One was protein structure prediction, another one was quadruple chemistry. And Alfold sort of rose from that protein structure prediction project. In its simplest form, it's a very simple problem where given an amino acid sequence which constitutes a protein, we want to understand the 3D coordinates of those amino acids. And that's really important because if you understand the 3D structure of the protein that informs and gives you an idea about what the function would be of that protein. And that has implications for drug discovery, for understanding basic cellular biology and so forth.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Largely has not been permeated by technology, not by IT to a great deal. And healthcare and life sciences collectively, it's almost like becoming 25% of U.S. GDP, these trillions and trillions of dollars going through this, and none of it, or very little of it being sort of revolutionized by tech. So this revolution, I think, is happening because of AI. AI is allowing this industrialization to happen. And especially turning these bespoke artisanal processes into something that is engineered and industrialized. AI is one aspect of it, and there's many others. I talked about robotics. And that's the arc that's, I think, exciting. And it's something where I think we saw hints of it in 2015. It's probably a 25-year arc, maybe a 30-year arc that we're 10 years into. And industrial revolutions don't happen overnight. But when you look back, the whole world's going to be changed. And so we're living in the middle of it. And I was actually always jealous about people living in the 1920s and people going from nothing to steam trains and all this stuff. And actually now we're the ones that I think are in the center of it. It's such an exciting time.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source
“Apply machine learning in impactful problems. And I think the most impactful area that you could work on is science. And that was a complete left field sort of suggestion. Class was in school. So I was quite skeptical, to be honest. I told him, like, you caught the wrong guy. Like, I have no background in biology or physics or chemistry. But he said, no, I mean, the way you are approaching these things, it's good. Let's sort of give it a try and see where it goes. And so we started the science program with six or seven people working on two projects. And now it's almost 140 person team. And we have 10 different initiatives banning many areas of biology from structural biology to genomics to protein design to cell genomics to quantum chemistry to meteorology to fusion to pure mathematics to computer science. So it's a long journey, but it started with sort of an accident.”
2024-05-01 · a16z Podcast · Can AI Advance Science? DeepMind's VP of Science Weighs In · IDENTIFIED FROM THE TRANSCRIPT · source