Did Baidu Just Flip Off California? What CA DMV Must Now Demand From Baidu

In itself, the annual disengagement reports published by the California Department of Transportation are exciting and controversial enough in the community of fans and developers of autonomous cars. But this year the controversy has reached a new level. And it has less to do with individual companies haggling over percentage points on how far their robotic cars can go without the intervention of a human safety driver, but rather on the information provided by a single company.

Baidu, the Chinese Internet giant, began testing autonomous cars in China and California in 2016. The company was granted a test license and, like any other license holder, has since been required to submit an annual disengagement report to the California DMV. This report must contain quite simple information: Number of test vehicles, mileage and number of disengagements. The latter happens when a safety driver has to take control of the car.

The disengagement rate, i.e. how many miles a car can travel without human intervention, is considered an inaccurate figure that has been much criticized by experts and is actually considered irrelevant, but with the data it is currently the only insight into the state of the art. No other region in the world has such a reporting obligation, which is also then published, and certainly no other region has so many robot cars on the roads of so many companies.

The autonomous driving pioneer is Waymo, with the entity founded in 2009, some years after the DARPA Grand Challenge, to develop autonomous cars. To date, Waymo’s 800+ cars have clocked more than 20 million miles in autonomous mode, and even though development is progressing slower than hoped for, Waymo is considered the great technological leader. Spectacular videos of truly driverless Waymos already on the road in Arizona contribute to this.

From 2017 to 2018, Waymo nearly doubled its disengagement rate from 5,596 miles to 11,154 miles, while CRUISE quadrupled the rate from 1,254 miles to 5,204 miles per intervention. Amazing though it may be, it is still within the realm of possibility.

And this is where Baidu comes in.

Curtain up for Baidu

The new figures for the reporting period from December 2018 to November 2019 show for Waymo an increase from last year to 13,219 miles, an improvement of about 19%, and for CRUISE an increase by 135% to 12,221 miles. But Baidu seems to beat everyone.

In 2017, when the first figures for Baidu were available, the company just reached 42 miles per intervention. A year later, the figure was five times that, exactly 205.6 miles. And in 2019, Baidu catapults itself forward with a disengagement rate of all 18,050 miles. An increase of 8,679%(!).

And this is hard to believe and hard to understand. With just four test vehicles in California (the number of vehicles in China is not known) and just 108,300 miles driven (for comparison: Waymo drove 1,454,137 miles last year and CRUISE 831,040), the company is said to have made a technological quantum leap and overtaken everyone else?

Probably not if it was done properly. And this is where it gets confusing. Should and can one trust the data of a company headquartered in China? Not when you look at the country’s economic data. Western economists and analysts have known for years that the fantasy figures on growth and economic dynamism that come from China are invented. Several years later, the same growth figure is simply announced. On closer examination and the offsetting of other economic figures from the provinces and activities of economic units, it becomes immediately clear to a data analyst that not a single figure coming from China can be trusted.

This becomes bad if long-term investments are planned on the basis of these figures. As soon as it turns out that the figures used as a basis for decision-making were wrong, it is often too late and then perhaps there are not enough drugs, apartments, kindergartens, hospital beds or teachers available.

This becomes downright tragic if the figures are intended to simulate the security of a technology that does not exist in this form. A car that only needs my intervention every two years or 18,050 miles and in between drives safely and without accidents? Give them a permit. And then the first people die.

Possible Reasons For Creative Number Reporting

If Baidu – as can be assumed with high probability – in fact did not achieve these numbers, then the question arises why they did this? Is this a signal to the political leadership in Beijing that Chinese companies are now leading the way in the development of autonomous cars, in a key technology as identified by China that it wants to dominate by 2030? Does this mean that there are political benefits that Baidu would get at home from creative number reporting? Was the intention to show the California DMV that they don’t care one iota about their interpretation of disengagement? Or was it a simple calculation or transmission error and absolutely innocuous?

These figures from Baidu may perhaps have been one reason for the late publication of the dis-exposure reports this year, but I’m just speculating without knowing more.

What Should The Californian DMV Do Now?

First of all, the responsible persons in the Department of Motor Vehicles should knock on Baidu’s door again and take a look at the raw data of the vehicles, if necessary under threat of withdrawal of the license.

Secondly, this should be taken as an occasion for a new attempt to define better and more comprehensive indicators, as well as test scenarios for evaluating the development progress of autonomous cars. To this end, the companies and other stakeholders should also provide experts. We are already too close to this technology reaching product maturity for us to wait any longer.

Thirdly, the DMV should also define data standards that allows safety enhancing data and algorithms to be exchanged between manufacturers. This will probably also require the creation of a central and publicly accessible repository of these data and algorithms, so that not only one autonomous system fleet, but all fleets become better and safer for the benefit of society as a whole.

What should Baidu do?

So, Baidu, come clean with yourself and with us. You and I all know your numbers fall pretty much under the creative reporting label. You can’t pull in California the kind of stunt here that you might be able or required to pull back home. You’re losing a lot of trust. You can see the effect of that right now in Huawei. Their loss of trust in the western world has put their backs against the wall. Do you want that? Or don’t you care about business in the West anyway? I don’t think so either.

And don’t tell us that thanks to your Apollo system, which is a great project to create an open source operating system for autonomous driving, you have made this quantum leap. I’m sure that’s wrong.

The faster you come to clean with yourself and with us, the better for innovation, economy , and humanity.

This article was also published in German.

6 Comments

  1. Hi Mario,

    I read your blog and articles regularly and I enjoy your views and news a lot. I also like your opinions, although they are sometimes very biased toward autonomous driving and against all the others. I am also a fan of this (even posting your articles on Chinese social media) and would like autonomy to come quick into our daily lives.

    But with this article you have somehow crossed your limit from what you believe, so I do not know what your real goal or wish is for our future.

    I am an EU citizen living in China for 15 years already and I see changes and improvement in China daily, in all aspects of life. And I have seen many amazing stuff and things China has pulled off or invented in this period. Including lately the whole corona virus fight etc. What I do not understand is, why you need to attack (Baidu), if someone posted their numbers, which are too good to be true, or made a mistake in how they collect numbers, in the area you are so much promoting and believe in. Doesn’t this just throw all your beliefs out through the window?
    If the disengagement report was posted with very optimistic numbers, we should be very happy to see this news, but at the same time, with all your expertise, you (we) could question the credibility of reporting from Baidu. Instead of this, you have decided to put a finger to something you believe in. I hope this is not a finger to yourself.

    If you are somehow open minded, you should have read or see by now that most probably full autonomous driving will be firstly implemented in China probably. After all it is maybe the only country which has this in a national development agenda and takes it seriously.

    Lets stick to our main goal – promoting autonomous driving no matter from whom it will come first.
    At the end, we want our lives to become better – soon.

    Cheers

    David

    Like

    1. Thanks for the feedback, and of course I have some preferences, without being blind.
      Let me first start with that I admire China, what they have accomplished in such a short time to lift a large portion of their population out of poverty and become a technological leader. Of course such an effort does not come with its own problems and many of them China has to face and solve. What Europe underwent in a period of 100 years, China did in a decade.

      There is a good reason, why I am harsh with Baidu and call them out. As much as improvement is possible, given the tight networks between the industries and employees moving between different companies developing autonomous vehicle technology, this kind of advancement is impossible. I explain the reasons for my calling them out in the article. And I reflect on the background and the reasons. And we all know that economic stats from China have to be taken with a grain of salt. This specific number on the disengagement report is an ocean of salt. The number is an outright lie. We have to call b.s. where we see it. Such a number flies in the face of everyone’s knowledge. Unless they have invented a super-technology from the future that nobody else has, it’s not possible. As I have seen their cars, they are not different than anyone’s else, so there is no Baidu proprietory super-technology.

      And this is where I call them out for lying to the rest of us. And my question is, why did they do it? Where did this come from? Did they have some beef with CA DMV and thus try to flip them off? They certainly flipped off everyone else with even publishing such numbers. Or do they not care, but only for the signal they are sending back home? Because this is a key technology that China wants to lead, and the CA Disengagement Report being the only public report on autonomous vehicle technology – even with a key metric that is as bad as it is – is the vehicle to show that they are leading?

      Baidu’s number could give people a false impression of autonomous vehicle technology safety. And that’s why it is dangerous. They may get operation licenses (partly) based on such falsely reported numbers and this could kill people. So yes, I care about safety of the technology for humans. But Baidu just trampled the industry’s reputation for no good reason.

      We want to call out a cheater in our middle, otherwise we get hung with them.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s