Did Baidu Just Flip Off California? What CA DMV Must Now Demand From Baidu

In itself, the annual disengagement reports published by the California Department of Transportation are exciting and controversial enough in the community of fans and developers of autonomous cars. But this year the controversy has reached a new level. And it has less to do with individual companies haggling over percentage points on how far their robotic cars can go without the intervention of a human safety driver, but rather on the information provided by a single company.

Baidu, the Chinese Internet giant, began testing autonomous cars in China and California in 2016. The company was granted a test license and, like any other license holder, has since been required to submit an annual disengagement report to the California DMV. This report must contain quite simple information: Number of test vehicles, mileage and number of disengagements. The latter happens when a safety driver has to take control of the car.

The disengagement rate, i.e. how many miles a car can travel without human intervention, is considered an inaccurate figure that has been much criticized by experts and is actually considered irrelevant, but with the data it is currently the only insight into the state of the art. No other region in the world has such a reporting obligation, which is also then published, and certainly no other region has so many robot cars on the roads of so many companies.

The autonomous driving pioneer is Waymo, with the entity founded in 2009, some years after the DARPA Grand Challenge, to develop autonomous cars. To date, Waymo’s 800+ cars have clocked more than 20 million miles in autonomous mode, and even though development is progressing slower than hoped for, Waymo is considered the great technological leader. Spectacular videos of truly driverless Waymos already on the road in Arizona contribute to this.

From 2017 to 2018, Waymo nearly doubled its disengagement rate from 5,596 miles to 11,154 miles, while CRUISE quadrupled the rate from 1,254 miles to 5,204 miles per intervention. Amazing though it may be, it is still within the realm of possibility.

And this is where Baidu comes in.

Curtain up for Baidu

The new figures for the reporting period from December 2018 to November 2019 show for Waymo an increase from last year to 13,219 miles, an improvement of about 19%, and for CRUISE an increase by 135% to 12,221 miles. But Baidu seems to beat everyone.

In 2017, when the first figures for Baidu were available, the company just reached 42 miles per intervention. A year later, the figure was five times that, exactly 205.6 miles. And in 2019, Baidu catapults itself forward with a disengagement rate of all 18,050 miles. An increase of 8,679%(!).

And this is hard to believe and hard to understand. With just four test vehicles in California (the number of vehicles in China is not known) and just 108,300 miles driven (for comparison: Waymo drove 1,454,137 miles last year and CRUISE 831,040), the company is said to have made a technological quantum leap and overtaken everyone else?

Probably not if it was done properly. And this is where it gets confusing. Should and can one trust the data of a company headquartered in China? Not when you look at the country’s economic data. Western economists and analysts have known for years that the fantasy figures on growth and economic dynamism that come from China are invented. Several years later, the same growth figure is simply announced. On closer examination and the offsetting of other economic figures from the provinces and activities of economic units, it becomes immediately clear to a data analyst that not a single figure coming from China can be trusted.

This becomes bad if long-term investments are planned on the basis of these figures. As soon as it turns out that the figures used as a basis for decision-making were wrong, it is often too late and then perhaps there are not enough drugs, apartments, kindergartens, hospital beds or teachers available.

This becomes downright tragic if the figures are intended to simulate the security of a technology that does not exist in this form. A car that only needs my intervention every two years or 18,050 miles and in between drives safely and without accidents? Give them a permit. And then the first people die.

Possible Reasons For Creative Number Reporting

If Baidu – as can be assumed with high probability – in fact did not achieve these numbers, then the question arises why they did this? Is this a signal to the political leadership in Beijing that Chinese companies are now leading the way in the development of autonomous cars, in a key technology as identified by China that it wants to dominate by 2030? Does this mean that there are political benefits that Baidu would get at home from creative number reporting? Was the intention to show the California DMV that they don’t care one iota about their interpretation of disengagement? Or was it a simple calculation or transmission error and absolutely innocuous?

These figures from Baidu may perhaps have been one reason for the late publication of the dis-exposure reports this year, but I’m just speculating without knowing more.

What Should The Californian DMV Do Now?

First of all, the responsible persons in the Department of Motor Vehicles should knock on Baidu’s door again and take a look at the raw data of the vehicles, if necessary under threat of withdrawal of the license.

Secondly, this should be taken as an occasion for a new attempt to define better and more comprehensive indicators, as well as test scenarios for evaluating the development progress of autonomous cars. To this end, the companies and other stakeholders should also provide experts. We are already too close to this technology reaching product maturity for us to wait any longer.

Thirdly, the DMV should also define data standards that allows safety enhancing data and algorithms to be exchanged between manufacturers. This will probably also require the creation of a central and publicly accessible repository of these data and algorithms, so that not only one autonomous system fleet, but all fleets become better and safer for the benefit of society as a whole.

What should Baidu do?

So, Baidu, come clean with yourself and with us. You and I all know your numbers fall pretty much under the creative reporting label. You can’t pull in California the kind of stunt here that you might be able or required to pull back home. You’re losing a lot of trust. You can see the effect of that right now in Huawei. Their loss of trust in the western world has put their backs against the wall. Do you want that? Or don’t you care about business in the West anyway? I don’t think so either.

And don’t tell us that thanks to your Apollo system, which is a great project to create an open source operating system for autonomous driving, you have made this quantum leap. I’m sure that’s wrong.

The faster you come to clean with yourself and with us, the better for innovation, economy , and humanity.

This article was also published in German.