YouSaid · the spoken record

Daniela Amato

lines on the record: 36
first: 2024-06-06
most recent: 2024-06-06
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“It really is tantamount to saying we barely just invented the printing press, or we've barely just invented the Model T Ford car. And now what we should immediately do is try to rush and prevent future improvements to cars or to the printing press by largely putting the responsibility of any accidents that happen from people irresponsibly driving the car out on the streets on Henry Ford or the inventors of the printing press.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“That are increasing at speed and scale, but fundamentally these safety concerns that are largely science fiction and theoretical are a complete distraction at the moment. And lastly, we have no choice but to absolutely accelerate open source innovation. We should be investing in open source collaboration between America and our allies to keep our national competitiveness from falling behind our adversarial countries. And so the three big policy principles I would look for from regulators would be to regulate and focus and target misuses, not models to prioritize AI security over safety and to accelerate open source. But the current legislation is absolutely prioritizing the wrong things and is rooted in a bunch of arbitrary technical definitions that will be outmoded, obsolete, and overreaching fairly soon.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Non negotiable, really, here should be zero liability at the model layer, right? What you want to do is target misuses and malicious users of AI models, not the underlying models and not the infrastructure. And that's the core battle here. I think that's the fundamental flaw of this bill is it's trying to regulate the model and infrastructure and not instead focus on the misuses and malicious users of these models. And so over time, I think it would prove out that the right way to keep the US at the frontier of responsible, of secure AI innovation is to actually focus on the malicious users and misuses of models, not slow down the model and infrastructure layer. We should focus on concrete AI security and strengthening our enforcement and our defenses against AI security.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“25% of the compute that the base model developer used to fine-tune a model, then it's no longer a derivative model and you're off the hook for it as the base model developer. Well, that's absolutely nonsensical as well. As some great researchers like Eon Stoika at Berkeley have shown, it takes an extremely small amount of compute to fine-tune a model like Vikuna where with just 70,000 shared GPT conversations, they fine-tune Lama to become one of the best open source models at the time, showing it really doesn't take much computer data to turn a car into a tank to borrow an analogy. And so like with the 10 to the 26 compute threshold issue we discussed earlier, this is just another arbitrary magic number the bill authors are pulling out of thin air to try and define model layer computing dynamics that are so early and changing that it's absolute over regulation and will kill the speed of innovation here.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes. So this is another kind of worms as well. Again, a symptom of the root cause of this build's flawed premise of regulating models instead of misuses. So in the current bill draft, the language says that these restrictions and regulations will extend to a concept of a derivative model, which is a model that is a modified version of another model, such as a fine-tuned model. So if someone makes a derivative model of my base model that's harmful, I am now liable for it. It's akin to saying that if I'm a car manufacturer and someone turns a car I made into a tank by putting guns on it and shoots people with it, I should get thrown in jail. The definition of what a derivative model is also super vague. And so now the bill sponsors are considering an amendment that says, oh, let's add a compute cap to this definition. And they've decided to pick 25%, which is quite arbitrary. And to say if somebody uses more than”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Yes, and the other thing about open source is I guess this is true of any model theoretically, but the idea if someone takes it and builds on it, right? In AI, in generative AI or foundation models, you would call that fine-tuning, right? Where you kind of retrain a model to your own purposes using your own data. And again, this bill would, as written, impose liabilities again on the original developers if someone is able to fine-tune their model to perform theoretically some sort of bad act, right? I mean, how realistic is it for someone to even build a model that would be resistant or resilient against these types of fine-tuning attacks or optimizations, for lack of a better term?”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“AI revolution is moving, will continue. It's certainly not in California and probably not in the United States. Open source is kind of the heart of software innovation, and this bill slows it down, has a chilling effect on open source by putting liability on the researchers and the builders pushing open source forward.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Oh, fundamentally, I don't think the current wave of modern generative scaling laws based AI would even exist without open source, right? If you just go back and look at how we got here transformers, kind of the atomic unit of how these models learn was an open source widely collaborated on development. In fact, it was produced at OneLab, Google, and allowed another lab after open publishing and collaboration, which was OpenAI, to actually continue that work. And there's no chance we'd be here without open source. The downstream contributions of open source continue to be massive today when a company like Mistral or Facebook open source models and release their weights, that allows other startups to then pick up on their investments and build on top of them. It's like having the Linux to the closed source Windows operating systems. It's like having the Android to the closed source iOS. And without those, there's no chance that the speed at which the”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Speaking of startups, that's to say nothing about open source projects and open source development, which have been like a huge driver of innovation over the past couple of decades. We're talking about very, very bootstrapped skeletal budgets on some of these things, but hugely, hugely important.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Prior to launch, demonstrate beyond any shred of reasonable doubt, which is, again, a completely ambiguous definition in the bill, that these users were known about, their risks were understood, that exhaustive safety testing had been done to make sure none of these things would be possible, just completely killed that engine. If we went back in time and this bill passed as currently envisioned, as much as I hate to say it, there's no chance I would have founded my company in California.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“And robotics to VFX and entertainment that we hadn't even considered. And so the whole engine and the beauty of platform businesses is that developers can focus on developing general and highly flexible technology and then just let the market figure out entirely new niche use cases at scale. And this is true of almost every great AI business I've either worked with directly or invested in, right? Whether it was mid-journey and image generation and anthropic and language models or 11 labs and audio models, great technologies always find their way.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Oh, yeah, look, I love California, and that's why I'm fighting so hard for this. I did my undergraduate and graduate work here in the Bay. I founded my first company here. I sold that to another California company. And over the last decade plus that I've been here, it's only become more and more clear to me that a huge part of what makes the entire startup AI ecosystem even work is the ability for founders to take bold technology risks without having to worry about the kinds of ambiguity and liability risks that this bill is proposing. When we first started Ubiquiti 6, my last company, the goal was to empower developers to use our computer vision pipeline for all kinds of new use cases that we hadn't even imagined. We had some idea of what people would do originally augmented reality applications, but after we launched it, we found millions of users who used our 3D mapping technology for entirely new kinds of uses from architecture.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Provisions will be to drive AI development underground or offshore. No rational startup founder or academic researcher is going to risk jail time or financial ruin just to advance the state of the art in AI. They'll simply move their operations to a jurisdiction with a more sensible regulatory environment, and the U.S. will lose out, period. The worst part, these liability provisions actually make us less safe, not more. By driving AI development into the shadows, you lose the transparency and open collab that's essential for identifying and battle-hardening vulnerabilities in AI models. What we need is more open source development, not less. So while the build sponsors may have good intentions, imposing blanket liability on model developers for hypothetical future misuse is the exact opposite of what we need.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“No, you're absolutely right. The idea of imposing civil and criminal liability on model developers when downstream users do something bad is so misguided and such a dangerous precedent. First off, the bill requires developers to prove that their models can't possibly be used for any of the defined hazardous capabilities. But as we just discussed, these definitions are way too vague, ambiguous, and subject to interpretation. can a developer prove a negative, especially when the goalposts keep moving? It's an impossible standard to meet. Second, the bill holds developers responsible for any misuse of their models, even if that misuse comes from someone else who's fine-tuned or modified the model. It's ridiculous. It's like holding car manufacturers liable for every accident caused by a driver who's modified their car. So it's an absurd standard that no other industry is held to. And the practical effect of these liability”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“The other thing that SB 1047 would do, which we haven't really touched on, is impose liability, civil and in some cases, criminal liability on model developers for the civil liability part. If they build a model that's covered by this bill, they need to be able to prove. With beyond reasonable assurance or whatever the language is that this could not possibly be used for any of these types of attacks. And also they have to be able to prove that no one else could come along and say fine-tune their model and use it for some sort of attack, right? So that's a whole new level to be on the hook for money as an individual or jail time as an individual for building this model and not making it quote unquote safe enough.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Exactly. I think what we really need is more investment in defensive artificial intelligence solutions. What we need is to arm our country, our defense departments, our enforcement agencies with the tools they need to keep up with the speed and scale at which these attacks are being perpetuated, not slowing down the fundamental innovation that can actually unlock those defensive applications. And look, the reality is America and her allies are up against a pretty stiff battle from adversarial countries around the world who aren't stopping their speed of innovation. And so it's almost an asymmetric warfare against ourselves that's being proposed by SB 1047.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Right, and to your point, even if a model made it marginally easier to learn, let's say, how to build a bioweapon. Like, one, people know how to do that today. We have all of these things. We have labs dedicated to all of these things. You still need to get materials to carry out these attacks. And there are regulations around acquiring those materials and databases around who's buying what. Yes, it does seem like the existing legal framework for some of these major threats is very robust.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Absolutely. I mean, basically, what we have is a complete over rotation of the legislative community around entirely non-existent concerns of what is being labeled as AI safety when what we should be focusing on is AI security. These models are no different than databases or tools in the past that have given humans more efficiency.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“To their advantage, while smaller companies, open source researchers and academics will be completely left out in the cold. It's almost like saying we've just invented the printing press, and now we're only going to let those folks who can afford $100 million budgets to make these printing presses decide what can and cannot be printed. It's just blatant regulatory capture, and it's one of the most anti-competitive proposals I've seen in a long time. And what we should be focusing on instead is regulating specific high-risk applications and malicious end users. That's the key to ensuring that AI benefits everyone, not just a few.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Burden on these smaller teams, many of whom just don't have the resources to navigate these super complex regulatory requirements. Plus, when you just look at the rapid pace of model engineering, these definitions would need to be updated constantly, which would be a major drain on innovation. So when you combine that ambiguity with the criminal and monetary liabilities proposed in the bill, as well as the broad authority they're trying to give to the new frontier model division, which is sort of like a DMV for AI models that they're proposing, which can arbitrarily decide these matters, the outcome is clear, right? Most startups will simply have to relocate to more AI-friendly states or countries, while open source AI research in the US will be completely crushed due to the legal risks involved. So in essence, the bill is creating this disastrous regressive tax on AI innovation. Large tech companies that have armies of lawyers and lobbyists will be able to shape the definition.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so this $100 million amendment to train might seem like a reasonable compromise at first, but when you really look at it, it has the same fundamental flaws as the original flop threshold. The core issue is that both approaches are trying to regulate the model layer itself rather than focusing on the malicious applications or misuse of the models. Generative AI is still super early, and we don't even have clear definitions for what should be included when calculating these training costs. Do you include the data set acquisition, the researcher salaries? Should we include the cost of previous training runs or just the final ones? Should human feedback for model alignment expenses count? If you fine-tune someone else's model, should the costs of the base model be included? These are all open questions without clear answers. And forcing startups, founders, academics to provide legislative definitions for these various cost components at this stage would place a massive”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“The other thing is, at the time we're recording this, there are some proposed amendments floating around to SB 1047, one of which would limit the scope of the bill to applying, again, only to models trained at that compute capacity. And additionally, that also cost more than $100 million to train. So what's your thought on that? And again, if we attach a dollar amount to this, doesn't it make the compute threshold kind of obsolete?”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“That's right. That's right. That's a great historical example. I think there was this 1979 export administration act that the U.S. had written in the Cold War era, in the 70s. And the definitions for what was dangerous in the Cold War became obstate so fast that a couple of decades later when the Macintosh launched, it was technically a munition. And so we've been here before and we know that when policymakers and regulators try to capture the state of a current technology that's dramatically improving really fast, they become obsolete, incredibly fast. And that's exactly what's happening here.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“I think it's a relevant touch point to remind people that a smartphone today, like an iPhone 15, has more flops, more performance than a supercomputer did about 20 years ago. Like the world's fastest supercomputers, your iPhone can do more flops than that. The Apple Macintosh G4, I think back in 1999, had enough computing power that it would have been regulated as a national security threat. So these numbers, these are very much sliding scales to your point”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Flop threshold. But as we've just discussed, that's a pretty serious overestimation. So even assuming a model costs a billion dollars to train to that level today, it's going to cost as little as 400,000 in just a decade. This is easily within the range for most small businesses who are going to then have to grapple with compliance and regulation and so on. And so look, the bottom line is that given the breakneck pace at progress in compute costs and efficiency, we can expect smaller companies and academic institutions to start hitting these benchmarks in the very near future.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Years, it would only take around $6 million worth of flops to achieve that same result. Just considering the algorithmic progress alone. Now, when you put these two trends together, it paints a pretty stunning picture because the cost to reach any given benchmark of reasoning of capability is dropping by about 50 times every five years. And so that means that if a model costs $100 million to train to some benchmark in 2024, by 2029, it will probably cost less than $2 million. That's well within a startup budget. And by 2034, a decade, that cost will drop to somewhere between $40,000, $50,000 putting it within the reach of literally millions of people. And despite these clear trends, the advocates for the bill seem to be overlooking or underestimating this rapid progress. Some folks are suggesting that, oh, these smaller companies might take 30 years or more to reach this 10 to the 26th.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Right. So that all comes down to two key trends. One, the falling cost of compute, and two, the rapid progress in algorithmic efficiency. Empirically, the cost per flop for GPUs is having roughly every two to two and a half years. And so this means that a model that costs about $100 million to train today would only cost about $25 million in about five years. And less than $6 million in a decade, just based on hardware trends alone, just Moore's law. But that's not even the whole story, right? Algorithmic progress is also making it dramatically easier to achieve the same benchmark performance with way less compute rapidly. And so when you look at those trends, we observe that the compute required to reach a given benchmark of reasoning or capability is decreasing by half about every 14 months or less. So if it takes 100 million worth of flops to reach some given benchmark today, in five”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“We see frequently is people in labs going out there and saying, We're going to build big state of the art models that. Seems like, and you could have this sort of performance for a fraction of the cost in a small number of years.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Are likely to sit above that 10 to the 26 flop threshold. In reality, most open source and academic models will soon be covered by that definition as well. This would really hurt startups. It would burden small developers. And ironically, it's going to reduce the transparency and collaboration around AI safety by discouraging open source development.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Model strained with this immense amount of computation are extremely large and capable to the point where they could pose social risks or harm inherently, if not developed carefully. But dying regulations to some fixed flop count or equivalent today is completely flawed because algorithmic efficiency improves, computing costs decline, and so models that take far fewer resources than 10 to the 26 flops will match the capabilities of a 10 to the 26 flop model of today within a fairly short time frame. So this threshold would quickly expand to cover many more models than just the largest most cutting-edge ones being developed by tech giants. It will basically cover most startups and open source too within a really short amount of time. And so while today in 2024, realistically only a handful of the very largest language models like GPT-4 or Gemini and other top models from the big tech companies.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Right. So flops in this context refers to the number of floating point operations used to train an AI model. And floating point operations are just a type of mathematical operation that computers perform on real numbers as opposed to just integers. And the amount of flops used is a rough measure of the computing resources and complexity that went into training a model. And so if models are like ours, flops might be the amount of steel used to make a car, to borrow an analogy. It doesn't really tell you much about what the car can and cannot do directly, but it's just one way to kind of measure the difference between the steel required to make sedan versus a truck. And this 10 to the 26 flop threshold is significant because that's how the bill is trying to define what a covered model is. It's an attempt to define the scale at which AI models become potentially dangerous or in need of additional oversight. And this all starts from the premise that foundation”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“A lot to get into with the proposed law and some of its shortcomings or oversights. But one place I want to start is both SB 1047 and President Biden's executive order from last year establish mandatory reporting requirements for models that are trained, and this is a little difficult to speak, but bear with me listeners, that are trained on 10 to the 26 integer floating point operations per second or flops is the acronym of compute power. Can you explain to listeners what flops are and why they're significant in this context?”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“And so, if it passes in California, it will set the precedent in other states. It will set a nationwide precedent. And ultimately, that'll have rippling consequences outside of the US to other allies and other countries that look to America for guidance and for thought leadership. And so what is happening here is this butterfly effect with huge consequences on the state of innovation.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Shock, disbelief. It's hard to understand just how blindsided startups, founders, the investor community that have been heads down building models, building useful AI products for customers, paying attention to what the state of the technology is, and ultimately just innovating at the frontier, these folks, the community broadly feels completely blindsided by this bill. When it comes to policymaking, especially in technology at the frontier, the spirit of policymaking should be to sit down with your constituents, startups, founders at the frontier builders, and then go solicit their opinion. And what is so concerning about it right now is that this bill, SB1047, was passed in the California Senate with a 32 to 1 overwhelming vote, bipartisan support. And now it's headed to an assembly vote in August, less than 90 days away, which would turn it into law.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Bill, whether a static threshold can realistically even hold up to exponential trends in algorithmic efficiency in compute cost, historical precedents that we can look to for comparison, the implications of this bill on open source, and the startup ecosystem at large, and most importantly, what you can do about it. Now this bill really is the tip of the iceberg with over 600 new pieces of AI legislation swirling in the United States today. So if you care about one of the most important technologies of our generation in America's ability to continue leading the church here, we encourage you to read the bill and spread the word.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source
“Senate Bill 1047 is designed to apply to models trained above certain compute and cost thresholds. The bill also puts developers both civilly and even criminally liable for the downstream use or modification of their models. By requiring them to certify that their models won't enable, quote, hazardous capability. The bill even expands the definition of perjury and could result in jail time. Third, the bill would result in a new frontier model division, a new regulatory agency funded by the fees and fines on AI developers. And this fairy agency would set safety standards and advice on AI laws. Now, if all of this sounds new to you, you're not alone, but today you have the opportunity to hear from A16Z General Partner Ajay Mitta and venture editor Derek Harris. Together, they break down everything the tech community needs to know right now, including the compute threshold of 10 to the power of 26 flops being targeted by this.”
2024-06-06 · a16z Podcast · California's Senate Bill 1047: What You Need to Know · IDENTIFIED FROM THE TRANSCRIPT · source