Revealed: The AI Systems and Training Models of Tesla, Waymo

Autonomous vehicles have been in the media glare ever since their inception. Today’s autonomous vehicles are technological marvels, from the hardware deployed on the vehicle to the models they use to drive themselves. They are also a hotbed of AI innovation, as autonomous vehicles have to solve real-world problems using machine learning algorithms.

The best-known manufacturer in the autonomous vehicles space is Tesla, and for a good reason. It has innovated extensively for powering capable on-vehicle computers through its collaboration with TSMC. Along with this, it has also trained models in-house for use in its autopilot feature using a supercomputing cluster powered by NVIDIA GPUs.

Google’s autonomous driving company, Waymo, is the only other manufacturer to achieve fully-autonomous driving. Using Google’s silicon chops, Waymo has developed and manufactured a full set of chips and sensors in-house to use with their cars, dubbed Waymo Drivers. The team behind the car has also created a simulation they call ‘Carcraft’ which allows them to train models in a completely virtual environment. 

The hardware powering autonomous vehicles

Tesla has worked closely with ARM and TSMC to create their own chips called Tesla FSD chips. This silicon is custom-built to run inference with low-latency while being extremely power efficient. Modern Teslas are equipped with eight cameras which provide a 360-degree of the car’s surroundings. This information is then fed into the FSD chip to make driving decisions in real-time.

In addition to their supercomputing cluster powered by NVIDIA GPUs, Tesla has also built another supercomputer called the Dojo system. Custom-built from the ground up for machine learning training tasks, this uses Tesla’s datasets to train their full self-driving algorithms.

Download our Mobile App

Waymo, on the other hand, developed a full suite of hardware for use in their self-driving cars. The current 5th generation Waymo Driver starts as a stock Jaguar I-PACE, an all-electric SUV. This car is then modified for use with Waymo’s suite of sensors and compute, consisting of radar, LIDAR, and cameras paired with enterprise-grade CPUs and GPUs. Waymo’s algorithms are trained on Google’s cloud platforms powered by TPUs and the TensorFlow ecosystem. 

The brains of self-driving cars

Both these companies have also invested heavily into creating AI and ML models to be deployed in their vehicles. Tesla was one of the first companies to use neural networks for self-driving applications. Using datasets collected from their FSD beta test fleet, the team behind Tesla Autopilot trained over 48 networks to deploy in their vehicles. A single build of the Autopilot system reportedly takes over 70,000 GPU hours to train. 

Apart from the neural networks that iteratively learn from new scenarios recorded every day from Tesla cars, Tesla also uses a high-fidelity representation of the world around their vehicles to develop autonomy algorithms. These algorithms can reconstruct a complete computer-readable version of the car’s surroundings using data from cameras in the car. This is then used as the ground truth for the algorithms, which perform inference using the car’s onboard FSD chip. 

Waymo, on the other hand, has a 115-acre training facility in California known as Castle. The team behind the self-driving car has created a fully realistic closed-course testing facility that mimics a variety of urban settings. By testing their vehicles in this facility, Waymo is able to train its algorithms to react to emergency situations that humans face every day. 

In addition to testing at Castle, Waymo has also trained its algorithms by driving over 20 billion miles in a simulation. Here, they can accurately identify the most challenging situations a Waymo Driver will encounter, as well as create virtual scenarios to better train their algorithms.  They also work closely with Google Brain to integrate state-of-the-art AI and ML algorithms in their vehicles. 

Behind the scenes: AI in manufacturing

Artificial intelligence creates unique designs that cut down on the amount of material required to create a part while still maintaining structural stability. A prime example of this is Audi’s new AI network, called FelGAN, which uses a generative adversarial network to generate rims for their cars. Trained using self-supervised learning, this model can generate fresh kinds of rims while keeping them lightweight. This is then made into a prototype and tested, with Audi’s engineers praising the model’s capability to ‘think outside the box’. 

While big auto manufacturers are looking to introduce AI as a value add for the manufacturing process, a company has turned to state-of-the-art algorithms to design a car from scratch. Czinger, an American-based automobile manufacturer, has created the 21C, a hypercar designed with AI. 

By using AI, Czinger was able to apply Pareto optimization to every single one of their components, thus making sure that not a single gram of the car has gone to waste. Due to the complex nature of certain components, they had to also invest heavily into additive manufacturing techniques to synthesise them in house. 

AI will soon become another tool in the arsenal of manufacturers that will bring them into a new age of automobile design and creation. Bringing together novel 3D printing techniques, powerful generative AI, and advancements in robotics, the cars of the future will leave the cars of today in the dust.