Revealed: The AI Systems and Training Models of Tesla, Waymo

Tesla and Waymo have been the poster children for the use of AI in the automobile industry, but mainstream manufacturers are now catching up
Listen to this story

Autonomous vehicles have been in the media glare ever since their inception. Today’s autonomous vehicles are technological marvels, from the hardware deployed on the vehicle to the models they use to drive themselves. They are also a hotbed of AI innovation, as autonomous vehicles have to solve real-world problems using machine learning algorithms.

The best-known manufacturer in the autonomous vehicles space is Tesla, and for a good reason. It has innovated extensively for powering capable on-vehicle computers through its collaboration with TSMC. Along with this, it has also trained models in-house for use in its autopilot feature using a supercomputing cluster powered by NVIDIA GPUs.

Google’s autonomous driving company, Waymo, is the only other manufacturer to achieve fully-autonomous driving. Using Google’s silicon chops, Waymo has developed and manufactured a full set of chips and sensors in-house to use with their cars, dubbed Waymo Drivers. The team behind the car has also created a simulation they call ‘Carcraft’ which allows them to train models in a completely virtual environment. 

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

The hardware powering autonomous vehicles

Tesla has worked closely with ARM and TSMC to create their own chips called Tesla FSD chips. This silicon is custom-built to run inference with low-latency while being extremely power efficient. Modern Teslas are equipped with eight cameras which provide a 360-degree of the car’s surroundings. This information is then fed into the FSD chip to make driving decisions in real-time.

In addition to their supercomputing cluster powered by NVIDIA GPUs, Tesla has also built another supercomputer called the Dojo system. Custom-built from the ground up for machine learning training tasks, this uses Tesla’s datasets to train their full self-driving algorithms.

Download our Mobile App

Waymo, on the other hand, developed a full suite of hardware for use in their self-driving cars. The current 5th generation Waymo Driver starts as a stock Jaguar I-PACE, an all-electric SUV. This car is then modified for use with Waymo’s suite of sensors and compute, consisting of radar, LIDAR, and cameras paired with enterprise-grade CPUs and GPUs. Waymo’s algorithms are trained on Google’s cloud platforms powered by TPUs and the TensorFlow ecosystem. 

The brains of self-driving cars

Both these companies have also invested heavily into creating AI and ML models to be deployed in their vehicles. Tesla was one of the first companies to use neural networks for self-driving applications. Using datasets collected from their FSD beta test fleet, the team behind Tesla Autopilot trained over 48 networks to deploy in their vehicles. A single build of the Autopilot system reportedly takes over 70,000 GPU hours to train. 

Apart from the neural networks that iteratively learn from new scenarios recorded every day from Tesla cars, Tesla also uses a high-fidelity representation of the world around their vehicles to develop autonomy algorithms. These algorithms can reconstruct a complete computer-readable version of the car’s surroundings using data from cameras in the car. This is then used as the ground truth for the algorithms, which perform inference using the car’s onboard FSD chip. 

Waymo, on the other hand, has a 115-acre training facility in California known as Castle. The team behind the self-driving car has created a fully realistic closed-course testing facility that mimics a variety of urban settings. By testing their vehicles in this facility, Waymo is able to train its algorithms to react to emergency situations that humans face every day. 

In addition to testing at Castle, Waymo has also trained its algorithms by driving over 20 billion miles in a simulation. Here, they can accurately identify the most challenging situations a Waymo Driver will encounter, as well as create virtual scenarios to better train their algorithms.  They also work closely with Google Brain to integrate state-of-the-art AI and ML algorithms in their vehicles. 

Behind the scenes: AI in manufacturing

Artificial intelligence creates unique designs that cut down on the amount of material required to create a part while still maintaining structural stability. A prime example of this is Audi’s new AI network, called FelGAN, which uses a generative adversarial network to generate rims for their cars. Trained using self-supervised learning, this model can generate fresh kinds of rims while keeping them lightweight. This is then made into a prototype and tested, with Audi’s engineers praising the model’s capability to ‘think outside the box’. 

While big auto manufacturers are looking to introduce AI as a value add for the manufacturing process, a company has turned to state-of-the-art algorithms to design a car from scratch. Czinger, an American-based automobile manufacturer, has created the 21C, a hypercar designed with AI. 

By using AI, Czinger was able to apply Pareto optimization to every single one of their components, thus making sure that not a single gram of the car has gone to waste. Due to the complex nature of certain components, they had to also invest heavily into additive manufacturing techniques to synthesise them in house. 

AI will soon become another tool in the arsenal of manufacturers that will bring them into a new age of automobile design and creation. Bringing together novel 3D printing techniques, powerful generative AI, and advancements in robotics, the cars of the future will leave the cars of today in the dust. 

Sign up for The Deep Learning Podcast

by Vijayalakshmi Anandan

The Deep Learning Curve is a technology-based podcast hosted by Vijayalakshmi Anandan - Video Presenter and Podcaster at Analytics India Magazine. This podcast is the narrator's journey of curiosity and discovery in the world of technology.

Anirudh VK
I am an AI enthusiast and love keeping up with the latest events in the space. I love video games and pizza.

Our Upcoming Events

24th Mar, 2023 | Webinar
Women-in-Tech: Are you ready for the Techade

27-28th Apr, 2023 I Bangalore
Data Engineering Summit (DES) 2023

23 Jun, 2023 | Bangalore
MachineCon India 2023 [AI100 Awards]

21 Jul, 2023 | New York
MachineCon USA 2023 [AI100 Awards]

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Council Post: The Rise of Generative AI and Living Content

In this era of content, the use of technology, such as AI and data analytics, is becoming increasingly important as it can help content creators personalise their content, improve its quality, and reach their target audience with greater efficacy. AI writing has arrived and is here to stay. Once we overcome the initial need to cling to our conventional methods, we can begin to be more receptive to the tremendous opportunities that these technologies present.