YouSaid · the spoken record

Boris Sofman

lines on the record: 198
first: 2021-11-16
most recent: 2021-11-16
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Well, we try to be careful because, man, when you change the hardware, it takes a while to retrain the models and everything. So we just went through that and going from the Pacificas to the Jaguars. And so the Jaguars and then the trucks have the same generation now. But yeah, the LIDAR is, it's incredible. And so Waymo has leaned into that as a strength. And so a lot of the near-range perception system that obviously kind of carries over a lot from the car side uses LIDAR as a very prominent kind of like primary sensor. But then obviously everything has its strengths and weaknesses. And so in the near range, LIDAR is a gigantic advantage. And it has its weaknesses when it comes to occlusions in certain areas, rain and weather, like things like that. But it's an incredible sensor and it gives you incredible density, perfect location precision, and consistency, which is a very valuable property to be able to kind of apply a mellow approach.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Publicly. So some parts iterate more than others. I'm trying to remember on the censor side. So the entire self-driving system, which includes sensors and compute, is fifth generation.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. So I'll start with LiDAR. So LIDAR has been like way most, one of Waymo's big strengths and advantages where we developed our own LIDAR in-house where many generations in, both in cost and functionality. It is the best in the space.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“That's right. And it's not just like they no longer give data, it could be they're fouled or they stop giving data where some electrical thing gets cut or part of your compute goes down. So what's neat is that like you have way more sensors, part of his field of view and equations, part of his redundancy, and then part of it is new use cases. So there's new types of sensors to optimize for long range and kind of the sensing horizon that we look for on our vehicles that is unique to trucks because it actually is kind of much further out than a car. But a majority are actually used across both cars and trucks. And so we use the same compute, the same fundamental baseline sensors, cameras, radar IMUs. And so you get a great leverage from all of the infrastructure and the hardware development as a result.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“But by having two on a side, there's actually multiple benefits. So, one is just beyond the trailer, so you can see fully flush with the trailer. And so you eliminate most of your blind spot except for right behind the trailer, which is great because now the software carries over really well and the same perception system you use on the car side, largely that architecture can carry over. And you can retrain some models and so forth, but you leverage it a lot. It also actually helps with redundancy where there's a really nice built-in redundancy for all the LiDAR cameras and radar where you can afford to have any one of them fail and you're still okay. And at scale, every one of them will fail.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Much occlusion, and so then you would add a lot of complexity to the software to make up for that and just unnecessary complexity.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Kind of on the cabin, not all the way in the front, but like kind of where the mirrors for the driver would be. And so those are the main sensor pods. And the reason they're there is because if you had one in the middle, the trailer is higher than the cabin and you would be occluded with this awkward wedge.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so it's more than the cars. So very loosely you can think of as like 2x, but it varies depending on the sensor. And so we have like dozens of cameras, radar, and then multiple LIDAR as well. You'll see one difference where the cars have a central main sensor pod on the roof in the middle and then some kind of hood sensors for blind spots. The truck moves to two main sensor pods on the outsides where you would typically have the mirrors next to the driver. It effectively goes as far out as possible kind of up to the boundaries of the front.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Augmentations of the existing tech stack because a majority of what we're solving is actually generalizable to freeway driving and different platforms. And over time, they all start to kind of merge ideally where the things that are unique as As minimal as possible, and that's where you get the most leverage. And that's why Waymo can do, you know, take on two trillion dollar opportunities and be nowhere near 2x the cost or investment or size. In fact, it's much, much smaller than that because of the high degree of leverage.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“We have this really cool test facility where we take it to the max and actually imagine a truck with these giant training wheels on the back of the trailer and you're pushing it past the safety limits in order to like try to actually see where it goes. And so you define this high dimensional boundary, which then gets captured in software to stay safe and actually do the right thing. But it's kind of fascinating to sort of challenges you have there. But then all of these things drive really interesting challenges from perception to unique behavior prediction challenges and obviously in planner where you have to think about merging and creating gaps with a 53 foot trailer and so forth. And then obviously the platform itself is very different where you have different numbers of sensors sometimes types of sensors and you also have unique blind spots that you have because of the trailer which you have to think about. So it's a really interesting spectrum and in the end you try to capture these special cases in a way that is queenly”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Quickly, you have roll risk and so forth. And so we spent a huge amount of time understanding those boundaries. And those boundaries change based on the load that you have, which is also an interesting difference. And you have to propagate through the algorithm so that you're leveraging your dynamic range, but always staying within a safety bounds, but understanding what those safety bounds are.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“But there are then unique challenges like you have a very different dynamics in your vehicle where you have to see much further out in order to have the proper response time because you have an 80,000 pound fully loaded truck. That's a very, very different type of braking profile than a car. You have really interesting kind of dynamic limits because of the trailer where you actually, it's very, very hard to physically flip a car or do something like physically like most risk in a car is from just collisions. It's very hard to like in any normal operation to do something other than like, you know, unless you hit something to actually kind of like roll over or something on a truck, you actually have to drive much closer to the physical bounds of the safety limits, but you actually have real constraints because you could have really interesting interactions between the cabin and the trailer. There's something called jackknifing if you turn too quick.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“And that's why over time these do kind of converge, where in a lot of ways, a lot of the challenges we're solving are freeway driving, which are going to carry over very well to the vehicles, to the car side.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“That's right. Constraints are just completely different. And yeah, there's definitely a property of that here. And we're also still learning about it because there is a lot of really fascinating and sometimes really elegant things that the industry has done where there's companies whose entire existence is around despite the constraints optimizing as much as they can out of it. And those lessons do carry over. But it's an interesting kind of merger of worlds to think about like, well, what if this was completely different? How would we approach it? And the interesting thing is that For a really, really, really long time, it's actually going to be the merger between how to use autonomy and how to use humans that leans into each of their strengths.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“All or nothing problem. You start to kind of unlock more and more functionality over time. Which functionality most enables this optimization ends up being kind of part of the discussion. But you're totally right. Like you fast forward to five years, 10 years, 15 years. And you think about very generalized capability of automation and logistics, as well as the ability to poke into how those handoffs work. The efficiency goes far beyond just direct cost of today's like unit economics of a truck. They go towards reinventing the entire system in the same way that you see these other industries that like when you”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“One of the biggest enablers is, yes, we have to solve autonomous trucking first. And by the way, that's not like an overnight thing. That's decades of continued kind of expansion and work. But the first checkpoint in the first route is not that far off. But once you start enabling and you start to learn about how the constraints of autonomous trucking, which are very, very different in the constraints of human trucking, and again, strengths and weaknesses, how do you then start to leverage that and rethink a flow of goods more broadly? And this is where like the learnings of like really partnering with some of the largest fleets in the US and the sort of learnings that they have about the industry and the sort of needs that they have and what would change if you just like really broke this one constraint that like holds up the whole network? Or what if you enabled this other constraint? That actually drives the roadmap in a lot of ways because this is not like.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“The handoffs to like, you know, well, even you get really ambitious, you start to expand this beyond like how does the fulfillment center work and like how does the transfer hub work? How does the warehouse work to, I mean, there's a lot of opportunities to start to automate these chains. And a lot of the inefficiency today is because like you have a delay, like Port of LA has a bunch of ships right now waiting outside of it because they can't dock because there's not enough labor inside of the port of LA. That means there's a big backlog of trucks, which means there's a big backlog of deliveries, which means the drivers aren't where they need to be. And so you have this like huge chain reaction and your feasibility of readjusting in this network is low because everything's tied to humans and manual kind of processes or distributed processes across a whole bunch of players. And so”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“The average kind of truck operation is like a one to five truck family business. And so there's just like a huge amount of fragmentation which makes for really interesting challenges in kind of stitching together through like bulletin boards and brokerages and some people run their own fleets. And this world's kind of like evolving. It is one of the less digitized and optimized worlds that there is. And the part that is optimized is optimized to the constraints of today. And even within the constraints of today, this is the $900 billion industry in the US, and it's continuing to grow.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Pretty fascinating. This is one of the most motivating aspects of all this where, like, yes, there's a mountain of problems that you have to solve the get to the first checkpoints and first drivelists and so forth. And inevitably, like in a space like this, you plug in initially into the existing kind of system and start to kind of learn and iterate. But that opportunity is massive. And so, you know, a couple of the factors that play into it. So first of all, there's obviously just the physical constraints of driving time, driver availability. Some fleets have a 95% attrition rate right now because of just this demands and kind of gaps and competition and so forth. And then it's also incredibly fragmented where you would be shocked at when you look at industries, like and you think of the top 10 players, like the biggest fleets, like the Walmarts and FedExes and so forth, the percentage of the overall trucking market that's captured by the top 10 or 50 fleets is surprisingly small.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“The goal is to have no human in the truck. Now, of course, right now we're testing with expert operators and so forth, but the goal is to, now there might be circumstances where it makes sense to have a human, and obviously these trucks can also be manually driven. So sometimes we talk with our fleet partners about how you can buy Waymo-equipped dimeward truck down the road and on the routes that are autonomous, it's autonomous. On the routes that are not, it's human driven. Maybe there's L2 functionality that adds safety systems and so forth. But as soon as they become, as soon as we expand in software, the availability of driverless routes, the hardware is forward compatible to just now start using them in real time. And so you can imagine this mixed use, but at the end of the day, the largest value proposition is where you're able to have no constraints on how you can operate this truck. And it's 100% autonomous with nobody inside.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“So most of it will be the truck, and everything is like connected. So we have our servers where we know exactly where every truck is. We know exactly what's happening at a hub. And so you can imagine like a large backend system that over time starts to manage timings, goods, delivery windows, all these sort of things. And so you don't actually need to, there might be special cases where that is valuable to equip some sensors.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“L4 can be applied to any operating domain or scope, but it's effectively for the places where we say we're ready for autonomous operation, we are 100% operating through as a self-driving truck with no human behind the wheel. That is L4 autonomy. And it doesn't mean that you operate in every condition. It doesn't mean you operate on every road. But for a particularly well-defined area operating conditions, routes kind of domain, you are fully autonomous. And that's the difference between L4 and L5, and most people would agree that at least any time in the foreseeable future, L5 is just not even really worth thinking about because there's always going to be these extremes. And so it's a race and almost like a game where you think of what is the sequence of expanded capabilities that create the most value and teach us the most and create this feedback loop where we're building out and unlocking more and more.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Five miles manually to get to the final kind of depot to avoid having to solve the general surface street problem, which is obviously very complex. Now, when the time comes, and we are increasingly already, we're pushing on some of this, but we will increasingly be pushing on surface street capabilities to build out the value chain to go all the way depot to depot instead of transfer hub to transfer hub. And we have probably the best advantages in the world because of all the Waymo experience on surface streets. But that's not the highest ROI right now where the highest ROI is hub to hub and get the routes going. And so when you ask what's self-four,”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Autonomous trucking. And that's what's going to happen, actually. You need to come out in January and check it out because it's going to be really cool. Not only is it our main operating headquarters for our fleet there, but it will be the first fully ground up design driverless hub for driverless autonomous trucks in terms of where do they enter, where do they depart, how do you think about the flow of people, goods, everything. It's like, it's quite cool and it's really beautiful on how it was thought through. And so early on, it is totally reasonable to do the last.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“That's right. And so one of the simplifications that we intentionally have put on is using what we call transfer hubs where you can imagine depots that are at the entry points to metropolitan areas, like let's say Dallas, like the hub that we're building, which does a few things that are very valuable. So from a first product standpoint, you can automate transfer hub to transfer hub. And that path from the transfer hub to the full freeway route can be a very intentional single route that you can select for the features that you feel you want to handle at that point in time.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“So, the goal is to get to market as fast as we can because the moment you get to market, you just learn so much and it influences everything that you do. And it is, I mean, one of the experiences that carried over from before is that you add constraints, you figure out the right compromises, you do whatever it takes because getting the market is so critical, right? But here with a town is drawn.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Come back. Yeah. But trucking is actually, Texas is one of the best places to start because of both volume, regulatory weather. There's a lot of benefits. On trucking, a huge opportunity is Port of LA going east. So in a lot of ways, a lot of the work is to start to stitch together a network and converge to port of LA, where you have the biggest port in the United States. And the amount of goods going east from there is pretty tremendous. And then obviously there's kind of channels everywhere and then you have extra complexities as you get into like snow and increment weather and so forth. But what's interesting about trucking is every single route segment that you add increases the value of the whole network. And so it has this kind of network effect and cumulative effect that's very unique. And so there's all these dimensions that we think about. And so in a lot of ways, Dallas is a really unique hub that opens up a lot of options has become a really valuable lever.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“San Francisco has these types of unique challenges dense pedestrians, all this like complexity, especially when you get into the downtown areas and so forth. And Phoenix has a really interesting kind of spectrum of challenges. Maybe other ones like LA kind of add freeway focus and so forth. You start to kind of cover the full set of features that you might expect and it becomes faster and faster if you have the right systems and the right organization to then open up the fifth city and the 10th city in the 20th city. On trucking, there's similar properties where obviously there's uniquenesses and freeways when you get into really dense environments and then the real opportunity to then get even more value is to think about how you expand with some of the service free challenges. But for example, right now we're looking, we have a big facility that we're finishing building in Q1 in Dallas area that'll allow us to do testing from the Dallas area on routes like Dallas to Houston, Dallas to Phoenix.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Right. So starting with the car side. Way more very intentionally picked the Phoenix area and the San Francisco area as a follow once we hit driverless where when you think of consumer transportation and ride sharing kind of economy, a big percentage of that market is captured in the densest cities in the United States and so really pushing out and solving San Francisco becomes a really huge opportunity and importance and places one dot on kind of like the spectrum of complexity. The Phoenix area starting with Chandler and like kind of expanding more broadly in the Phoenix metropolitan area, I believe the fastest growing city in the US it's a kind of a higher medium sized city but growing quickly and still captures a really wide range of kind of complexities and so getting to driverless there actually exposes you to a lot of the building blocks you need for the more complicated environment and so in a lot of ways there's a thesis that if you start to kind of place a few of these kind of dots where”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, it's a good question. So there's, and as you can expect, it's a mix. Some things become a lot easier or at least more flexible. Some things are harder. And so on the things that are like the tailwinds, the benefits, a big focus of automating trucking, especially initially, is really focusing on the long-haul freeway stretch of it, where that's where a majority of the value is captured. On a freeway, you have a lot more structure and a lot more consistency across freeways across the US compared to surface streets where you have a way higher dimensionality of what can happen, lack of structural, lack of consistency and variability across cities. So you can leverage that consistency to tackle at least in that respect a more constrained AI problem, which has some benefits to it. You can itemize much more of the sort of things you might encounter and so forth.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“We have some drivers that have four million miles of lifetime driving experience. It's pretty incredible. And yeah, learning from them, like some of them are on the road for 300 days a year. It's a very unique type of lifestyle.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Can be driving in a single day before they're not allowed to drive anymore, which is a very important safety constraint. What that does is it enforces limitations on how far jumps with a single driver could be and makes you very subject to availability of drivers, which influences where warehouses are built, which influences how goods are transported, which influences costs. And so you start to have an opportunity on everything from plugging into existing fleets and brokerages and the existing logistics network and just immediately start to have a huge opportunity to add value from a cost and driving fuel insurance and safety standpoint all the way to completely reinventing the logistics network across the United States and enabling something completely different in what it looks like today.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Network. And long term, there's actually, it's a pretty incredible enabler where today you have already a giant shortage of truck drivers. It's over 80,000 truck driver shortage. That's expected to grow to hundreds of thousands in the years ahead. You have really, really quickly increasing demand from e-commerce and just distribution of where people are located. You have one of the deepest safety challenges of any profession in the US where there's a huge, huge, huge kind of challenge around fatigue and around kind of the long routes that are driven. And even beyond kind of the cost and necessity of it, there are fundamental constraints built into our logistics network that are tied to the type of human constraints and regulatory constraints that are tied to trucking today. For example, our limits on how long a driver”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“So we're starting with developing L4 Autonomy for class 8 trucks. These are 53 foot trailers that capture like a big sizable percentage of the goods transportation in the country. Long term, the opportunity is obviously to expand a much more diverse types of vehicles, types of goods transportation, and start to really expand in both the volume and the route feasibility that's possible. And so just like we did on the car side, you start with a single route with a very specific operating kind of domain and constraints that allow you to solve the problem. But then over time, you start to really try to push against those boundaries and open up deeper feasibility across routes, across surface streets, across environmental conditions, across the type of goods that you carry, the versatility of those goods, and how little supervision is necessary to just start to scale this.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“It's a mix, so there's a huge amount of centralized kind of core teams that support all applications. And so you think of something like the hardware team that develops the lasers, the compute, integrates into vehicle platforms. This is an experience that carries over across any application that we have in an ebb and flow with both. Then there's like really unique perception challenges, planning challenges like other types of challenges where there's a huge amount of leverage on a Cortex stack, but then there's like dedicated teams that think of how do you deal with a unique challenge. For example, an articulated trailer with varying loads that completely changes the physical dynamics of a vehicle that doesn't exist on a car, but becomes one of the most important kind of unique new challenges on a truck.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Adds its own unique challenges, but the success case is that the challenges that you push on, they get leveraged across all platforms and all domains.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“It's a cool sunny name. I mean, when you think about it, it's just like, well, we're going to transport it via this and that. So it's just kind of like an illusion to the mechanics of transporting something. Yes, cool. And it is a pretty good grouping. And the interesting thing is that even the groupings kind of bore where Wayma 1 is like human transportation. And there's a FOIA autonomous service in the Phoenix area that every day is transporting people. And it's pretty incredible to just see that operate at reasonably large scale and just kind of happen. And then on the Via side, it doesn't even have to be like long haul trucking is a major focus of ours. But down the road, you can stitch together the vehicle transportation as well for local delivery also. And a lot of the requirements for local delivery overlap very heavily with consumer transportation.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“That's right. And so, in the same way that you have a driver's license and your ability to drive is a tied to a particular make a model of a car. And of course, there are special licenses for other types of vehicles, but the fundamentals of a human driver, very, very large you carry over. And then there's uniquenesses related to a particular environment or domain or a particular vehicle type that kind of add some extra additive challenges. But that's exactly right. It's the underlying systems that enable a physical vehicle without a human driver to very successfully accomplish the task that previously wasn't possible without 100% human driving.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Hardware, software, infrastructure, everything that goes into it to enable and to commercialize autonomous driving. This hits on consumer transportation and ride sharing and kind of vehicles and urban environments. And as you mentioned, it hits on autonomous trucking to transport goods. So in a lot of ways, it's transporting people and transporting goods. But at the end of the day, the underlying capabilities are required to do that are surprisingly better aligned than one might expect, where it's the fundamentals of being able to understand the world around you, process it, make intelligent decisions, and prove that we are at a level of safety that enables large-scale autonomy.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Sure. Waymo, by the way, has been eye opening on just how incredible the people and the talent is and how in one company you almost have to create, I don't know, 30 companies worth of like technology and capability to kind of solve the full spectrum of it. So I've been at Waymos since 2019, so about two and a half years. So Waymo is focused on building what we call a driver, which is creating the ability to have autonomous driving across different environments, vehicle platforms, domains, and use cases. As you know, got started in 2009. It was almost like an immediate successor to the Grand Challenge in Urban Challenges that were like incredible kind of catalyst for this whole space. And so Google started this project and then eventually Waymo spun out. And so what Waymo is doing is creating the systems, both”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. Creating a minefield of potential places to feel off. And then you're sidestepping where the biggest kind of functional AI challenges are to actually have really rich productivity that actually kind of justifies the higher price points. And that's part of the challenge. It's like, yeah, robots are going to get to like thousands of dollars, tens of thousands of dollars and so forth. But you can imagine what sort of expectation of value that comes with it. And so that's where you want to be able to invest the time and depth. And so going down the full human replica route creates a gigantic Distraction and really, really high bar that can end up sucking up so much of your resources.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and that's why you have these big pushes that for most of the last decade plus, like where you'd have full human replicas of robots, being down to like skin and like kind of in some places. My personal feeling is like, man, that's not the direction that's most fruitful right now.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“And you kind of had that with, I don't know, I guess data and some of the other ones. But yeah, so, and that becomes a constraint that lets you meet the bar.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Those kinds of personalities. Where you're like, you're intentionally kind of getting a toddler level of speech. So that's exactly right. So you can have. Tied into the experience where it is a more limited character, or you embrace the lack of emotion.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Way while we are 2D2 and kind of other characters are able to. It's quite powerful and it generalizes across cultures and across ages really, really well. I think we're going to be in that world for a little while where it's still very much an unsolved problem on how to make something it touches on the Uncanny Valley thing. So if you have legs and you're a big humanoid looking thing, you have very different expectations and a much narrower degree of what's going to be acceptable by society than if you're robot like Cosmo or some other form where you can kind of like reinvent the character. Speech has that same property where speech is so well understood in terms of expectations by humans that you have far less flexibility on how to deviate from that and lean into your strengths and avoid weaknesses.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“For a product, it's too deep right now. It's just not, you would immediately break the fiction because the state of the art is just not good enough. And that's on top of just narrowing down the demographic where like the way you speak to an adult versus the way you speak to a child is very different, yet a dog is able to appeal to everybody. And so right now there is no speech system that is like rich enough and subtly realistic enough to feel appropriate. And so we very, very quickly kind of like moved away from it. Now, speech understanding is a different matter where understanding intent, that's a really valuable input. But giving it back requires way, way higher bar given kind of where today's world is. And so that realization that you can do surprisingly much with either no speech or kind of tonal like the”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Stays feeling alive. And so, but then when you combine it, Permutations and kind of like the combinations of emotions stitched together sometimes surprised us because you see them in isolation. But when you actually see them and you see them live, you know relative to some event that happened in the game or whatnot, it was kind of cool to see the combination of the two. And not too different in other robotics applications where you get so used to thinking about the modules of a system and how things progress through a tech stack that the real magic is when all the pieces come together and you start getting the right emergent behavior in a way that's easy to lose when you just kind of go too deep into any one piece of it.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, we made it intentionally so that there would be some surprise and like a decent amount of variability in how you'd respond in certain circumstances. And so in the end, this isn't general AI. This is a giant spectrum and library of parametrized kind of emotional responses and an emotional engine that would map your current state of the game, your emotions, the world that people are playing with you, all so forth to what's happening. But we could make it feel spontaneous by creating enough diversity and randomness, but still within the bounds of what felt like very realistic to make that work. And then what was really neat is that we could get statistics on how much of that space we were saturating and then add more animations and more diversity in the places that would get hit more often so that you stay ahead of the curve and maximize the chance that it.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“You would just feel it that it's right and intuitively know, and then you'd also do, we did user testing. But it was very, very often that if we found it magical, it would scale and be magical more broadly. There were not too many cases where we were pretty decent about not getting to it, you know, geeking out or getting too attached to something that was super unique to us, but trying to kind of like, you know, put a customer hat on and does it truly kind of feel magical. And so in a lot of ways, we just give a lot of autonomy to the character team to really think about the character board and mood boards and storyboards and like what's the background of this character and how would they react. And they went through a process that's actually pretty familiar, but now had to operate under these unique constraints. But the moment where it felt right kind of took a fairly similar journey than as a character in an animated film.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source
“We stayed away from Uncanny Valley just by having such a different mapping where it didn't try to look like a dog or a human or anything like that. And so you avoided having like a weird pseudo-similarity but not quite hit in the mark. But you could like just fall flat where just like a personality or a character emotion just didn't feel right. And so it actually mirrored very closely to kind of the iterations that a character director of Pixar would have where you're running through it and you can virtually kind of like see what it'll look like. We created a plugin to where we actually used like Maya, the simulation, you know, the animation tools and then we created a plugin that perfectly matched it to the physical one. And so you could like test it out virtually and then push a button and see it physically play out. And there's like subtle differences. And so you want to make sure that that feedback loop is super easy to be able to test it live. And then sometimes like.”
2021-11-16 · Lex Fridman Podcast · #241 – Boris Sofman: Waymo, Cozmo, Self-Driving Cars, and the Future of Robotics · IDENTIFIED FROM THE TRANSCRIPT · source