Much has been discussed about Tesla’s head-start in the development and production of electric vehicles, and how especially the Model 3 is now shaking up the sales numbers, but Tesla today at the Tesla Autonomy Day gave in three hours an insight for investors about their plans on Autopilot, self-driving capabilities, and ultimately their robotaxi fleet.
Readers of of my book (rright now only in German, but coming out in English in summer 2019, pre-order it here) and this very blog are already familiar with many of the elements mentioned (here some articles about the Autopilot Hardware Kit 2.0, Software, Betatests, AI-Chip), but the condensed form with lots of new details made it even more impressive.
Tesla focused in the presentation and discussion on the following elements:
- Hardware: AI-chip
- Vision AI: test data and neural network
- Autopilot Software
- Robotaxi fleet
1. Hardware
About 1.5 years ago Tesla decided to develop its own AI chip, specialized for autonomous driving. The processor was supposed to have higher performance at a lower cost, and with the intent to refit all Teslas with the Hardware Kit 2.x.
The result are two redundantly worikng chips, that take the input from radar, GPS, maps, IMU, ultrasonic, wheel ticks and steering angel, compute and compare them, and control the vehicle. Six billion transistors computer 2.5 gigapixel per second of data at 600 GigaFLOPS. The system also runs with a special encryption that allows only Tesla software. 2,300 frames per second are handled, an increase by a factor of 21. And that with only 72 Watt energy use, 1.25 times the former amount. Atthe same time costs are down to 80%.
Those processors have been used in all Model S and X built since March 2019, and since mid-April in all Model 3s as well.
2. Vision AI
An important part of the presentation was the vision system and the neural network behind it, which is already today supported by half a million Teslas in the machine learning effort.
Tesla is one of the few developers of autonomous companies that approaches self-driving cars without the use of LiDARs. I have written on that topic in the past, but this time Tesla dove into the details to explain why it is not using LiDAR.
As every contemporary neural network today this one trains on a lot of data. The data is provided by the half million customer owned Teslas. On the one hand elements and objects such as lane markers are annotated manually, on the other hand there is also a big effort in annotating automatically. Tesla’s advantage is that their cars are on the road under very different conditions. Different daylights, weather, street and traffic conditions, local specialties, and even some driver induced fringe cases. The larger the dataset, the better the accuracy of the neural network’s output.
What’s special is that Tesla can request from the cars in the fleet data from special driving conditions. For instance, if Tesla wants to improve how a car drives in a tunnel or recognizes bicycles mounted on a car, Tesla’s neural network requests from the fleet specifically data and videos from such cases and with a short period receives hundreds of thousands of datasets from the fleet, which it then can use to train its machine learning system and test them in the simulator.
Tesla owners benefit from that insofar, as the results from the large, varied, and real world datasets are downloaded back to the cars, and each Tesla can then handle driving in tunnels better and safer with the Autopilot. And that in all countries and regions where Teslas are operating.
Before Tesla releases such changes on the fleet, they are uploaded onto the cars and run in a shadow mode. The car simulates in the car the maneuver with the driver’s reaction in a real world scenario. The data comparing the simulation and real maneuver then is used by Tesla to optimize and improve the Autopilot. When the accuracy is high enough, the feature is released for the cars.
Tesla detailed why they are not using LiDARs and where they fail. According to Tesla, LiDARs are creating not enough relevant information, and the creation of a 3D world can also be done by cameras. In fact Tesla showed how they did so from today’s cameras built into the Teslas.
Interestingly, Tesla also forgoes high definition maps (HD maps). Tesla experimented with them, but they appeared so brittle when little changes happen, that according to Musk, HD maps seem not to be the best strategy.
3. Autopilot Software
Tesla also mentioned how objects are recognized and categorized with the sensors on board and how the shadow mode is testing certain driving situations. Not only are all the miles driven in Autopilot mode used for the improvement, but also how human drivers handle such situations. Tesla identifies the best drivers in the fleet and uses their reactions to certain traffic situations as benchmark for the machine learning system.
Tesla also published a video of a fully autonomous drive of a Model 3 with the new Autopilot software and hardware. The vehicle starts from Tesla HQ in Palo Alto on Deer Creek Road, and after passing a traffic signal it turns on Page Mill Road onto northbound highway 280, exiting then on Sandhill Road returning on highway 280 to the Tesla HQ. The car handles multiple traffic lights and stop signs without requiring the safety driver’s interference.
So far Tesla has completed more than 9 million lane changes in Autopilot mode in the fleet, 100,000 every day, all without a single crash.
4. Robotaxi fleet
In the last part of the presentation Elon Musk referred to his master plan. The Masterplan Part Deux was presented several months ago, and we should take it seriously. All parts of the first master plan were completed, even when some were realized later than planned, But the most important part of the Part Deux is the rollout of Tesla’s robotaxi fleet.
Every Tesla owner can provide and operate his/her vehicle in the Tesla network as a robotaxi. Tesla will take a 25-30% cut of the fare. The cars can be provided and ordered via a smartphone app. Musk expects that this feature will go live in 2020 – pending regulatory approval.
How serious Tesla is with this plan can be seen in the just released leasing plan. Model 3s can be leased for three years, but not buy them after the period, as Tesla will integrate those vehicles into their robotaxi fleet.
Tesla also plans to launch mid next year a new battery pack that is designed for operation of one million miles. Today’s battery packs are designed to last between 300,000 and 500,000 miles.
In regards to costs, robotaxis will have dramatically cheaper costs per mile. On average, car owners today pay 62 cents per mile (34 Euro-cents per kilometer), and Uber users pay between two and three dollars per mile (1.11 and 1.66 Euro per kilometer). Tesla’s robotaxi will be costing less than 18 cents per mile (10 Euro-Cent per kilometer).
Per year Tesla expects a gross profit of 30,000 dollars per robotaxi for 90,000 miles (144,000 kilometers). The gross profit per mile will be at 65 cents (36 Euro-cent per kilometer). Musk also stated that from a financial perspective it is foolish to buy a different car than Tesla.
With the projected production rate the number of Teslas that can be used as robotaxis will be at one million vehicles. Musk also forecast that in the future hindsight we will see the use of LiDAR and HD maps as foolish approaches. Also steering wheel and pedals will become a thing of the past, again pending regulatory approval.
Collisions and crashes with robotaxis will be Tesla’s liability, according to Musk.
Conclusion
We already knew many of the parts, but seeing that in such a condensed form is a very strong statement by Tesla to the rest of the automotive industry, and everyone developing autonomous cars.
Elon Musk pointed twice to the fact that in 2019, seven years after the introduction of the Model S, there is not a single electric vehicle that comes close to the performance data of the Model S from 2012.
And now Tesla upped the ante, and we have seen it coming. The bold move to equip all Teslas since October 2016 with the Autopilot Hardware Kit 2.x for a cost of almost a thousand dollars and get a fleet of half a million vehicles on the road, with access to the data created by those cars for the past two years, enabling fleet learning, created an advantage for Tesla – assuming that Musk’s plan is coming true – that other car companies will not be able to catch up with in a decade. And I am calculating that from the experience with the Model S and their failed catch up.
Traditional car makers don’t even have the resources secured or planed, how and what components they want to put into their own cars, not even mentioning a master plan on how to build autonomous cars. Three years of collecting data from the fleet with now half a million cars has created a dataset that others cannot even dream of.
The same carmakers haven’t even understood electric vehicles (yet) or embraced, and they understand autonomous cars even less, not that they wanted anyways. And those signals and trends could be recognized earlier. My book (in German) – which is the namesake of this blog – already talked about that in 2017 and the new English version of my book coming out in Summer 2019 has more information on that as well. And for those who speak German (or know somebody speaking German), my just published book Foresight Mindset describes the method how to discover trends before they are trends.
Even though it is likely that Tesla will miss some of its ambitious timelines, they will fulfill them. They always have. And today they showed it in an impressing way.
Video
Here is the full video from the Tesla Autonomy Day with all the slides.
This article was also published in German.
The Last Driver License Holder... 
One great Overt Tesla Fan Boy Article. Statements from companies the likes of Tesla should always be taken with a grain of salt, their lives depend on duping investors to keep investing.
LikeLike
Right. The world is so simple, according to hugomaxwell.
LikeLike
I worked for one of those companies, and the discrepancy between what is happening and what is proclaimed in the media is just breath taking. Going from lane keeping to FSD (level 4 I suppose?) without a human driver is clearly more than just increasing TOPS or frames per second. Good old snake oil still pays the bills even today. Maybe next year we get 20,000 frames per second, that’ll do the trick for sure.
LikeLike
I think there is a lott that tthey have to do. Just with lane keeping and this stuff, L4 is not solved. Look at the things that Waymo is doing with intent recognition. I posted a video and this was not at all covered in the Tesla Autonomy Day.
What I do appreciate though is their approach and vision. Half a million cars with such equipment (if sufficient in sense of the sensors and their quality: I do doubt that), no LiDAR (bold, but I would project that this may be right, maybe just not now), no HD maps (I think that they are a not the right path, I agree with the brittleness).
LikeLike
I think we are at least 10 years away from having cars drive on the road without a human fallback behind the wheel. It’s not a matter of computing power or amount of data collected but a matter of robustness and intelligence from which we are currently still many orders of magnitude away.
Unless we are talking about a smart city use case where you have a controlled environment. And I suspect that this is the intention, otherwise current estimates and promises make no sense what-so-ever.
LikeLike
I think it comes more in a creeping fashion, and then it’s just there. A region here, a gated community there, then a downtown area with light traffic only for autonomous vehicles; maybe highways earlier. And lo and behold it’s everywhere and in 10 years we look at it and say: “How did that happen?”
LikeLike