How Far Is Waymo Ahead?

VW-CEO Diess surprised this week when he said that “Google-subsidiary Waymo is currently one to tow years ahead with self-driving cars.” Without missing a beat he also added at the AI-conference, which was organized by the German daily Welt, that “The game is not yet lost.”

The former is – mildly speaking – wishful thinking. Latter one really needs a huge effort and especially the will to make it come tru.

State of the Art at Waymo

Let’s take a look at the state of the art. Google started in 2009 with tests of autonomous vehicles on public streets, then under the project name Chauffeur, which would become in 2017 Waymo. The Google-founders gave the team led by DARPA winner of 2005, Sebastian Thrun, the goal to drive 100,000 miles, including some of the toughest streets, like the curvy Lombard Street in San Francisco.

When the team reached the goal ahead of time, Google started intensifying their development efforts. While the tests started with several Toyota Prius, they were followed by Lexus SUVs, 60 pieces of the self-designed and built Firefly/Koala-cars without steering wheels and pedals, which were replaced by Fiat-Chrysler Pacifica minivans (currently 600 of them on the roads, with more coming), the Jaguar iPace and trucks as part of the fleet. In the next years Waymo plans to bring 62,000 Pacifica and 20,000 Jaguars on the road..

Six years after the start of development, in summer 2015, Google celebrated its first big milestone. One million miles were driven in autonomous mode on public roads. So far more than 10 millions miles were driven, of which more than 9.5 million in city traffic, the rest on highways and country roads.

Driving in cities is much more challenging, because they are posing more complex driving situations than you find on highways and country roads. Former have besides other cars a lot of other traffic participants. Traffic is moving in multiple directions, street conditions and events are not easily predictable. This Waymo video, published for the 10 million miles celebration, gives a good idea of what situations the cars encounter in city traffic.

Today Waymo-cars drive 25,000 miles per day, adding another million miles every one-and-a-half months. Those miles represent a multiple of lifetimes of human drivers, which reach on average 300,000 miles in their life.

Developing self-driving cars undergoes the following stages:

1. Controlling and directing the vehicle
2. Identifying position and route planning
3. Recognizing and categorizing objects
4. Safety
5. Identifying intent of other participants
6. How humans use self-driving vehicles
7. Comfort

Fleet tests like the ones that Waymo has been doing since at least 2015 focus especially on stages 5., 6. and 7. The first four stages are more or less solved (almost) with the first million mile. Also, because in addition to the real-world driven miles, there are billions of miles driven in simulators. Special driving situations are replayed multiple thousands of times in different variations.

But it’s much more difficult to understand, how humans are using autonomous cars and what they consider ‘comfort’. Comfort is not just simply about comfortable chairs in the car or driving in a way that no one falls sick, but where passengers want to be dropped off or picked up. Turns out that the passengers want to be dropped off in front of the supermarket entrance, but picked up at the shopping cart return.

We only learn about such subtle details, when autonomous cars drive millions of miles under different conditions, with passengers on their daily chores. And this is exactly what Waymo has been doing with the Early Rider-program in Phoenix and with its own employees in Mountain View.

How far are the others?

once we understood that, then we can start calculating and comparing, how far other companies are or could be. And we also need to know that developing autonomous cars is not a linear curve, but an exponential one. The first few scenarios and problems are relatively simple to solve. Even small startups and research teams can reach impressive results in short time. But then it becomes more complicated. The more we try to improve and the more scenarios we encounter and add, the more difficult they become to solve, the more effort has to be put into the development. I analyzed that extensively in another post using the term Short Nose of Innovation.

 

The curve depicts that for a 50% safety rate (however you define safety) 1,700 scenarios have to be mastered, but with a jump to 60% it requires 7,000.

Similar curves are seen in other industries. Enterprise software is such an example. While it’s relatively simple create the technology, understanding different industry requirements for business software requires time and experience. Oracle or Microsoft never really managed to catch up with ERP-behemoth SAP, despite billions of dollars spent in acquisitions and development. Business software has be in use at companies to gather more experience and understanding of their needs, which then allows another iteration of the software.

Such curves don’t allow to just  throw money at it and pray to close the gap, but require long periods of tests and developments and successively increasing resources. Those are cycles that you can’t force and shorten the timeframe. That’s kind of like the old joke, where a project manager thinks to shorten a pregnancy by using 9 women instead.

This fact is displayed with autonomous vehicles in the Disengagement Report, which is published annually by California DMV.

Even though we don’t exactly know how many miles the other companies have driven so far autonomous miles, we can make some assumptions. Just let’s calculate generously that Mercedes and Volkswagen drove one million miles in autonomous mode. Then their state of development represents the one that Waymo achieved in June 2015. That’s 3.5 years from today.

But matter of factly, it’s probably much less than one million miles for Volkswagen, and for Mercedes it’s not completely clear how many miles they drove. Why would I make such a statement? If they’d really be testing massively and piling up miles, then we should see their cars in (German) cities. The forums and blogs should be fill with pictures of videos captured by users. I am following many of those blogs and forums, and believe me: there are almost no pictures and videos. And I follow that closely, not least because my own blog has become the biggest worldwide, with tons of pictures and sightings of autonomous cars.

Even if Mercedes or VW reached the one million miles, then they did that mostly on test tracks under controlled circumstances. No comparison to the chaotic traffic in a real city. Miles driven are not equal to miles driven. The level of difficulty increased from Test track to highway to country road to city. And the ultimate problem: chaotic parking lots with no rules. Add a fleet operation with passengers, including all ages, and you can guess the complexity.

Today, neither Volkswagen nor Mercedes or BMW, and with the exception of GM no other manufacturer has autonomous fleets with passengers in test operation. Mercedes announced one for 2019 in San Jose, and Volkswagen for 2021 in up to five cities. Note: announced! Even if they are doing this with startups like Aurora or suppliers like Bosch, those are also far away from what Waymo has achieved.

As an example, a Mercedes car drives twice(!) a week in Silicon Valley for short time on the same route in Sunnyvale, around the Mercedes Benz Research Center. With that level of ambition, changing the world will not happen. If you watch instead my video compilation of autonomous cars that I encounter on a daily base in Silicon Valley, then you see that what Mercedes is doing is almost meaningless. VW and BMW even do much less: nothing. They may do it somewhere else, but almost nobody has seen them.

Those with the best chances are other Americans. General Motors has shown with its acquisition of and investment in Cruise Automation how seriously they are taking this challenge. The Disengagement Report shows the result. Besides the well-financed startup Zoox GMCruise is the only serious contender for Waymo, though with some distance. And China has just started with it, but should be closely monitored. Today most Chinese self-driving car startups and companies are doing the most serious activities in that field in Silicon Valley.

The big unknown is Tesla, because this company is approaching the problem from a different angle. It’s a race between the larger LiDAR-fraction (‘autonomous cars need LiDARs’) and the smaller non-LiDAR-fraction (‘autonomous cars don’t need LiDARs, but can handle that with cameras’). The race is open, and is one between drops in LiDAR-technology prices and algorithms for cameras.

Meanwhile, Waymo – which is massively betting on LiDARs and even developed their own technology – soft-launched their commercial robotaxi service and is on the way to monopolize the market.

Conclusion

It’s becoming obvious that the statement by VW-CEO Diess to be “one to two years behind Waymo” has absolutely no foundation. It’s at least 5 years that German (and other) manufacturers are behind. There are worlds between them and as it is the property of exponential gaps, it’s almost unbridgeable.

That gap is not just in the space of autonomous vehicles, it’s the same with electric vehicles. Six years after the launch of the Model S they still have no competitive electric vehicle on the market. The first ones are supposed to come in 2019, but if you consider the performance metrics, they are behind the Model S form 2012. The stunning announcement to have electric vehicles in 2020 for a price of 20,000 euros is hitting into the same corner. They sound good (for their own teams), but are totally naive comments from the top manager of a German car maker.

The only chance to stay relevant is to team up with the best. If Mercedes is planning joint developments with Bosch, then it’s more an example of The Walking Dead. As an A-Player you have to team up with A-Players. What many in the German car industry have not yet understood is that the world has changed. You are not setting the pace anymore. Humility is a good starting point to potentially survive.

The game is lost, if you don’t fool yourself or blame others, and if your efforts are only half-heartedly, and show little of humility and will for change.

This article was also published in German.