Level 5 autonomous vehicles might arrive earlier than expected. Wayve recently published a technical report detailing their generative world model, GAIA-1 (Generative AI for Autonomy), previewed in June 2023. 

GAIA-1 is a generative world model that leverages video, text, and action inputs to generate realistic driving scenarios while offering fine-grained control over ego-vehicle behavior and scene features. Interestingly, Meta’s AI chief is one of the investors of Wavye. 

It is the first generative world model designed specifically for self-driving systems. It was introduced earlier this year and builds on Wayve’s research in future prediction, dreaming about driving, predicting in a bird’s eye view and learning a world model. 

How it Works 

Wayve collects huge amounts of driving videos and uses generative AI to create new datasets of videos to train the cars. Interestingly, the videos created by Wayve are so realistic that it is pretty much impossible to figure out if they are real or created using generative AI. 

GAIA-1 uses special encoders for different inputs like video and text, aligning them in time. Its main part, a world model with 6.5 billion parameters, predicts image tokens based on past data. These predictions are turned back into images using a video model with 2.6 billion parameters.

Trained on 4,700 hours of driving data, GAIA-1 has 9 billion parameters, making it powerful for generating accurate visuals.

Alex Kendall, chief of Wayve shared on X that unlike other models out there, it is not just a simulation but a full world model that understands how the world works. He elaborated on different scenarios demonstrating how autonomous vehicles can be trained using videos, eliminating the necessity of conducting actual road tests.

For instance, below is an example where Wayve prompted a vehicle to swerve right.

Moreover, Wayve’s model also takes into account the different weather conditions. To fully train a self-driving car, giving information about road conditions, lanes, and traffic is not enough. Wayne’s world model can be prompted to generate various driving scenes with the text prompt ‘It is’ followed by either ‘sunny’, ‘rainy’, ‘foggy’, or ‘snowy’.

Kendall said that one of the biggest challenges for self-driving is long-tail robustness: dealing with the enormous magnitude of edge cases we see on the road. An advantage of generative AI is its incredible ability to recombine experiences in new ways.

 For example, one can experience driving in fog, and experience jay-walking pedestrians, and GAIA can learn from these experiences to understand how to generate a fog+jaywalking scenario.

A user from X raised concerns about ensuring that synthetic data’s diversity truly represents real-world scenarios without bias. In response, Kendall stated that with the right approach, it’s possible to build a world model that balances realism and diversity.

The Wayve method of training self-driving cars is different from Tesla. Tesla leverages its cars on the roads to collect the driving data. The company utilises an AI system that collects real-time visual data from eight car cameras.  

This data is processed into a 3D output, identifying obstacles, their movement, lanes, roads, and traffic lights. Tesla gathers additional visual data from its car network, enhancing its training model. The newfound knowledge is integrated into cars through FSD software updates.

Tesla uses simulated data to train its autonomous vehicles. The simulation is created using a combination of real-world data and computer-generated imagery. Also, Tesla was one of the first companies to use neural networks for self-driving applications. 

Ashok Elluswamy, director of autopilot software at Tesla, earlier this year at Investor day said that Tesla will not just stop at computer vision but reach out to techniques in other areas such as language modelling and reinforcement learning to train its fleet of FSD vehicles. 

Wayve Brings Autonomous Cars to UK

It’s common to see self-driving taxis operated by Waymo and Tesla’s FSD cars in San Francisco. However, since both Tesla and Waymo are based in the US, most of their training data is collected from the US driving style.

Wayve interestingly is a UK company where until now self driving vehicles were not allowed on the road. However, earlier this year, Ford’s electric flagship Mustang Mach-E became the first autonomous car model to cruise in the UK, following regulatory approval in April. Mustang Mach-E features Ford’s driving assistance technology “BlueCruise”. 

Currently, Wayve is actively testing its self-driving technology on a daily basis on roads in the UK. Additionally, the company is conducting the largest last-mile autonomous grocery delivery trial in Europe with Asda, the UK’s second-largest online supermarket. 

Moreover, Wayve recently introduced a unique AI model called LINGO-1, which utilizes natural language to provide commentary on driving scenes and clarify its decision-making processes.