Earlier this month, TechCrunch held its annual Mobility Sessions event down in San Jose, where leading mobility-focused auto companies, startups, executives and thought leaders joined us to discuss all things autonomous vehicle technology, micromobility and electric vehicles.
Extra Crunch is offering members access to full transcripts key panels and conversations from the event, including a mainstage conversation between Waymo CTO Dmitri Dolgov and TechCrunch mobility axe Kirsten Korosec. Dmitri and Kirsten dove into Waymo’s full product evolution, and dissect the path ahead for the company and the AV industry as a whole.
Dmitri Dolgov: So essentially, what makes this problem interesting in my mind is that it’s not one or two things where you say “there’s only this one challenge that remains and then you solve everything.” It’s really across a whole number of different areas — a whole number of different disciplines from hardware, to more advanced sensors, more powerful sensors, sensors you can manufacture at scale, cheaper sensors, more powerful compute, cheaper compute software.
You mentioned sensor fusion — this is actually where the software plays very nicely with the hardware and that connection is very deep. And our approach at Waymo is interesting in that we build our own software and hardware in house.
So this is where we have access to very powerful sensors, and not just having the increased range and increased resolution, but also having access to the raw sensor data, like the raw measurements that you get from lidars from cameras from radars, and doing sensor fusion at a later stage, where you can really bring to bear modern, deep learning algorithms to have models that learn to pick out the best signal from those different sensing modalities. That’s a very active area of improvement.
Dmitri also goes into more depth on the specific technical areas of improvements for AV technology and the importance of simulation miles when building a polished mobility product. Dmitri and Kirsten also talk through the regulatory road map and the impact it will play on AV rollouts, AV product quality, as well as on cities and society as a whole.
For access to the full transcription below and for the opportunity to read through additional event transcripts and recaps, become a member of Extra Crunch. Learn more and try it for free.
Kirsten Korosec: Thanks, everyone for coming out. And we’re all here to talk to the CTO [Dmitri Dolgov] of Waymo. So I’m going to just let you roll with it. I’m wondering if you showed up in a self-driving car today, How’d you get here?
Dmitri Dolgov: I took a self-driving car to work today. Then I met with some folks there. And we took another car to here.
Korosec: So manually driven car from Mountain View. But My understanding is that you actually use Waymo self-driving cars quite a bit. You’re one of the most prolific users of the vehicle, correct?
Dolgov: Yeah. Nowadays, this is my default mode of transportation around town, I take our cars to work pretty much every day. I take them to get around town to run some errands. And actually, recently, there was a change that happened in California – we got the permit from the CPUC, the California Public Utilities Commission, that allowed us to take passengers in cars that are not employees of Waymo.
So now I can take my family and I have to go to the grocery store and I actually did that recently. I had to go with my son and instead of me doing the silly old thing of me driving around in my own car, we hopped into a Waymo. And that was cool.
Image via Getty Images / metamorworks
Korosec: Does he think it’s cool? Does he understand what your job is and that this is a self-driving car is like no big deal for him
Dolgov: Oh, he’s five and a half. He is all about lasers, radars, cameras, like you can point them out he gets very excited. I have another one. A daughter who is two and she’s also starting to get into it.
The other day, I came home in a Waymo and I pointed it out and said “hey, this is a robot!” And she looked at me and said “no, it’s not a robot, Daddy, it’s a Waymo.”
Korosec: You’re starting them early, that’s good. Get the brand recognition going early. They’ll be future employees at Waymo, I’m sure.
You’ve been riding in this self-driving cars for at least three or four years. Over the years, are you still seeing changes and improvements in the vehicle today? Or is it pretty much consistent and the same?
Dolgov: I think for me, it’s been more like 10 years in self-driving cars. In, 2009, at Google, which was Project Chauffeur and then it became Waymo. And I’ve been using them, taking rides in them, throughout those years.
It kind of changed before in the early days. It was for testing in development and early days, I would be testing something or building some software and that was the mode that I was using the car in. Over time it kind of switched to being the thing that I do for my job while I’m developing the system, to now being kind of the way I get around.
And of course, an interesting thing changed, and while they don’t have this in California, but in Phoenix, Arizona you can go and take a ride with nobody in the car. So that was an interesting change that I noticed. And the US is still improving at a pretty high rate.
And now it’s things that we’re focusing on that have to do more with the product and the features for ride-hailing applications, where the cars are getting better at picking you up, and where you want to be picked up and dropping you off on the most convenient location. And all of the user-facing features, removing all of the friction from getting you from point A to point B.
Korosec: So technically speaking now, in your experience, and as you’re riding in these here in California, and also in Arizona, is there anything left to master? Technically?
Dolgov: Yeah, there’s still a lot of work. We’ve been doing this for a while and this problem has this interesting property that is kind of like that metaphor of climbing a mountain, you see a peak, you get to the peak and you think you’re on top, and only then do you realize that you’re just getting started. And the real peak is farther ahead.
So there’s still lots and lots of challenges on the fundamental research, on the fundamental engineering, and on the product side. So the way I think about it is it’s a very exciting phase right now, for us. And that we really, we have truly driverless cars on the roads.
So we’ve got that solid foundation of engineering, of science, we’ve solved the fundamentals. And yet there are plenty of opportunities left to really accelerate the speed of iteration and the speed of improvements going forward.
And you can bring fundamental research to bear especially in the areas of advanced AI, to really accelerate the speed at which you improve the capabilities and make it a better product and scale. So things that I had for us are things like going to new places and making it really simple and really easy and really cheap, kind of like making that a turnkey solution where you go from place A to place B, and it requires basically no additional engineering.
Image via Getty Images / iam-Citrus
Korosec: There are probably a number of engineers in the audience. So I’m going to let you get even more technical for just a moment.
What are the critical areas? Is it sensor fusion, for example? Or is it some other aspect on the technical side where there’s still a lot of work and also opportunity.
Dolgov: So essentially, what makes this problem interesting in my mind is that it’s not one or two things that you say “there’s only this one challenge that remains and then you solve everything.” It’s really across a whole number of different areas, a whole number of different disciplines from hardware, to more advanced sensors, more powerful sensors, sensors you can manufacture at scale, cheaper sensors, more powerful compute, cheaper compute software.
You mentioned sensor fusion, this is actually where the software plays very nicely with the hardware and that connection is very deep. And our approach at Waymo is interesting in that we build our own software and hardware in house.
So this is where having access to very powerful sensors, and not just have the increased range and increased resolution, but also having access to the raw sensor data, like the raw measurements that you get from lidars from cameras from radars, and doing sensor fusion at a later stage, where you can really bring to bear modern, deep learning algorithms to have models that learn to pick out the best signal from those different sensing modalities. That’s a very active area of improvement.
This is just step one, perception, right? Seeing the world and then reasoning about it, and the object level of detecting different objects, understanding what they are classifying pedestrians, cyclists, cars, background, vegetation, and so forth, and so on.
But it really gets super interesting beyond that. This is kind of like the step one, self-driving one to one, you have to see the world and the different types of objects.
But really where it gets interesting, and where we’ve been doing a lot of fundamental work on the research front and engineering is reasoning about the world at kind of the same level, not just their prerogative level, and really deeply understanding the interactions between the different objects in the world.
How does one car affect the pedestrian or cyclist? How do you fit into that? And this is very deep, very dynamic, very social interactive aspect of this problem is where a lot of really interesting work happens.
Korosec: And are you doing that mostly in simulation?
Dolgov: So we do a lot of work in simulation. This is, in my mind, one of the critical tools that we have. In fact, we’ve recently crossed a new threshold. We’ve driven over 10 billion miles in simulation.
But simulation is an accelerator, right? In this space, it all comes down to your ability to improve the system and iterate as fast as possible. And in order to iterate fast, you need to be able to measure it with high accuracy, and low latency. And there is really no way to do that without simulation.
Imagine every time you make a system, you have to put it on a real fleet and drive. That’s just a very slow iteration cycle. So this is where simulation comes in.
But of course, simulation has to be grounded in reality. You have to convince yourself that your simulation is realistic, right? And this is why driving a lot in the physical world and in simulation, kind of doing the two together, in my mind is really important.
And at Waymo, we’ve driven more than 10 million miles in the real world, and over 10 billion miles in simulation. And the amount of driving you do in both of those is really a function of the maturity of your system, and the capability of the system.
If you’re just getting started, it doesn’t matter. You’re working on the basics, you can drive a few miles or a few thousand or tens of thousands of miles in the real world.
And that’s plenty to tell you and give you information that you need to know to improve your system and keep making the basics work slowly. It’s okay to just have a basic simulator where you can test your system in just simulation and you know when it’s not doing the right thing, and kind of the basic cases.
But as you as the system matures, you have a system that works great 99.99% of the time, then it really requires very large scale simulation, and very large scale driving in the real world. So you can convince yourself that your simulation is realistic, and that you can actually trust the results that you’re getting out of simulation and that they’re predictive of what you will encounter in the real world.
Image via Getty Images / Jackie Niam
Korosec: So a number of the bigger AV companies, your competitors, have simulation as well. And so two questions for you: One, if you’re a startup and you don’t have a simulation and you want to launch, should you just hang it up now?
And If you compare your simulation to techniques used over at Cruise and other bigger companies, what is it that makes Waymo’s so different or so special? Isn’t a simulation a simulation? Does it just come down to scaling up and doing as many miles as possible? Or is there some other special sauce, if you will?
Dolgov: I think what makes it a good simulator and what makes it powerful is two things. One, fidelity. And by fidelity, I mean, not how good it looks but how well it behaves, and how representative it is of what you will encounter in the real world.
And then scale. As you asked about, startups and companies that are just getting started can make a lot of progress, by just driving a little bit in the physical world and the simulator. But as you said, if you don’t have a simulator, and you want to launch, then you should just pack it up and go home.
I think you really need to drive in either the physical world or the simulation for two reasons. One is for testing. The other one is for validation. And if you’re developing the system, and especially in the early days, testing, driving for testing is sufficient.
And you don’t need to do much in either the simulated or the physical world. Right? Once you get into validation, that’s orders of magnitude more.
And whether you do it in simulation or not you need to convince yourselves that the results you’re getting out of simulation are realistic and valid. But then you need to ground that in a lot of driving and a lot of empirically observed data that only comes from the physical world.
Korosec: We were talking a little bit about hardware before and Waymo does a lot of it in house. So have you done anything differently for the Jaguar I-PACE, which is now on the road publicly being tested in the Mountain View area. Has that vehicle changed at all from the Pacific hybrid, which is sitting out front?
Dolgov: Yeah, which is very exciting. We’re starting to see those I-PACEs on the roads. This is a partnership that we announced just over a year ago in March of 2018.
Since then, our teams have been collaborating on engineering to make sure that the vehicle, and our self driving system are engineered to play nice together. And we’ve got a lot of work to make sure that everything from thermal to mechanical to electrical to acceleration fits together nicely, because those are targeted to be manufactured at scale, and deployed in a truly driverless fashion.
So we want to make sure you get all that right. But also put some effort and some thought and some design into the interior. So we can tailor it specifically for the ride-hailing application.
So things like the materials in the car, and the screens for the user interface, they’re specifically designed for application. So it’s really cool that these cars are rolling off the production line, ready for the Waymo driver, like our software and hardware to just plug right in.
Waymo-branded Jaguar I-Pace electric vehicles in San Francisco via Waymo
Korosec: So that’s not a bit of a different retrofitting that you were doing before. It’s more from the ground up at this point.
Dolgov: It’s more mature, right. So you know, we did some kind of work on – this is our fifth platform. We’ve been on Priuses, Lexuses and I-PACE is the new one.
And what makes exciting is that as much as the car is different and is supposed to be fifth-generation self-driving hardware, it’s our most advanced sensors, lighters, radars, cameras, and then a new generation of compute platform. And when we designed this hardware suite, we really put the emphasis on two areas.
One is capability. So all of the sensors are more capable. They have longer range, they have high resolution, they produce richer raw data than we can use in our software to do better sensor fusion and better perception. But It’s also designed to be a manufacturer at scale, right? Manufacturability and robustness.
And unit economics is not something you fix after the fact, you really have to design things from the ground up. So It’s very exciting to see many of these multi-year projects that are now coming together as this new platform.
Korosec: So in your mind now, are you finally at a point where you can really actually economically speaking, and technically speaking, go into volume production for a Robotaxis service there? Is there another iteration?
Dolgov: That platform is designed to do it at scale. And it will continue to improve, but that platform is —
Korosec: So you’re going to be rolling out any day, then?
Dolgov: The early days of the new platform as to what you’re seeing right now are engineering prototypes. We’re just bringing it up. So not probably not tomorrow.
Korosec: Okay. But maybe Friday.
Dolgov: Let me get back to you.
Korosec: Okay. And you were talking earlier about your children now being able to ride in your car with you, and there’s a safety driver in the vehicle now. But you’ve been working on these systems for a really long time and I’m wondering, would you feel comfortable with you and your two kids in the car pulling the safety driver out today?
Dolgov: I have taken, along with our employees as well as some members of the public have taken rides in cars that have nobody behind the wheel in Phoenix, Arizona. And yes, you know, I’ve taken many rides, and it’s an incredible experience. And yes, absolutely, I would take my kids on those rides.
Image via Getty Images / nadia_bormotova
Korosec: So what to you then is the limiting factor here? Is this a regulatory issue? If regulations didn’t exist today would we be seeing Waymo rolling around San Jose, and we’d be all taking them here today? What’s the limiting factor?
Dolgov: In Arizona there are no regulatory hurdles. So what we are doing is ramping things up in an incremental fashion. So since 2017, we’ve made this really big jump to having cars that are completely driverless, like truly driverless with not a test driver behind the wheel, that are operating on public roads.
I still remember the first time we did it we drove a hundred miles in a fully driverless fashion on that one day. And then since then, we’ve driven orders of magnitude more, we have our early rider program that is running in Phoenix, we have launched a commercial service that we call WaymoOne last year, and we’re learning.
We’re still learning in all these different deployment regimes about what makes it an awesome product. So we have more than 1000 people using our cars to get around in Phoenix, Arizona, and they’re giving us incredibly valuable feedback on what they like and what they don’t like.
And it’s a huge difference to go from technology that just works and having a center to them turning it into a product that people use and can get around in, in a seamless fashion to replace most transportation. So we’re using Phoenix to make sure we get that right. So we want to make sure that we have an awesome product and we’ve got that really down before we start expanding and replicating.
Korosec: Is that product going to change in terms of how cities are different and how people are different in each city?
I think of the difference between say San Francisco and Phoenix — once you’re rolling out the product and scaling it out more in Phoenix — when you bring it over to San Francisco, in which you have a permit now to be able to actually take people out for rides without exchanging money, is that product going to change at all?
What are you working on, on that end? Or is it going to be fundamentally the same no matter what city you’re in?
Dolgov: I think the product is going to be a core of it.
It’s fundamentally the same. You have an app, you pull it up, you say I want to come over here to there, the car comes picks you up. And of course, San Francisco and different cities have different properties.
There’s some specialization that has to happen. And you asked about cities, are they going to change? I think over time, this deployment of this technology of scale will actually lead to better layouts of cities, right? Right now there’s a lot of space that’s wasted on parking lots of just because people drive their own vehicles that sit around doing nothing 95% of the time.
And I actually think mileage is really powerful. Not just in right hailing, but the way we talk about it is that we’re building a driver. And when you think about that way, you can put that driver into all kinds of vehicles into all kinds of products, from ride-hailing, to trucking to deliveries to connecting to public transit to personally on vehicles. So I think it is going to be massively transformative on a slightly longer time horizon.
Image via Getty Images / petovarga
Korosec: We’re out of time, but I want to squeeze in one very quick question. And it’s a big one, how safe and safe enough, just in terms of industry, do we need for AVs to be prolific?
Dolgov: I don’t have a one-line answer for you. It’s a complex, really, really important question that we’ve been spending a lot of effort on.
And it comes down to a number of different valuation, evaluation and validation methodologies. From empirical data that you see in the real world to the results you’re seeing in simulation to structured testing to various systematic methods like validation and verification and FME and so forth and so on.
We’ve released a little bit of our thinking and we shared some of that previously in our safety report. And we’re going to continue to do that going forward.
Korosec: We’ll look for the safety metrics coming soon, right. Thanks so much.
Dolgov: Thank you.