Did Elon Musk Take A U-Turn on LiDARs?

On 20th May 2021, a Tesla Model Y with rooftop LiDAR sensors was spotted by Grayson Brulte, a consultant in the autonomous vehicle industry, in Palm Beach, Florida. People found this to be rather interesting given Elon Musk, Tesla’s Chief Executive Officer, and his historically less-than-favourable opinion of the laser sensor technology.

LiDAR, which is short for light detection and ranging, is a laser sensor that most tech and automobile companies deem essential for developing self-driving cars. LiDAR employs lasers to create a three-dimensional image of the surrounding landscape. This image is necessary because it tells us what the autonomous vehicle would perceive of the environment, predicts the behaviour of pedestrians and fellow cars, and helps safely navigate roads. Musk, however, has not been too fond of LiDAR. In 2019, he called LiDAR a ‘fool’s errand’ and said that anyone relying on it ‘is doomed’. Instead, Musk says that Tesla would bank on cameras as its key hardware component to autonomous vehicle development. As per Tesla, camera sensors—especially with their increasing pixel resolution and lower prices—are a far better alternative to LiDAR.

Given this, the picture of one of Musk’s cars equipped with LiDAR technology surprised Brulte. What followed next was his tweet on the same, which launched a slew of rumours that Musk might be making a U-turn from his previous opinion of the laser sensor technology. To stir in additional elements of surprise, it was reported that Tesla has entered a partnership with Luminar to use their LiDAR sensors for ‘testing and developing’ reasons. Neither company has spoken out about what this partnership would mean yet. 

Before we look at Tesla’s plans, let us dissect Elon Musk’s argument: How Necessary is LiDAR in creating autonomous vehicles?

LiDAR has become a standard fixture on multitudes of self-driving cars such as those operated by GM and Alphabet’s Waymo. Furthermore, many people today believe that autonomous driving will not work without LiDAR. Sanjeev Sharma—the founder of Swaayat Robots—pointed out that opting for LiDAR and camera-based perception systems is essentially a tradeoff between accuracy and computational heavy lifting. Sharma also noted that autonomous vehicles required perception, planning and localisation—and most non-LiDAR systems are unable to deliver on all three fronts. 

Blickfeld, a producer of LiDAR, said, in a blog post, that relying only on cameras and radar, but not LiDAR, is not safe. According to Blickfeld, cameras, while being great at capturing colour images (which LiDAR does not do), lack when it comes to recording the third dimension—which is essential to measure distances and avoid objects. Due to this, cameras require image recognition software to interpret the recorded 2D information to estimate distances. The problem with such software is that they use artificial intelligence and machine learning to label and store ‘experienced situations’. This procedure includes thousands of kilometres, both actual and simulated, but can pose problems for a vehicle entering unknown territory. Recording every single situation that is not a part of routine driving on an algorithm is nearly impossible. LiDAR technologies, however, can get away with the arduous step of recording all this data. LiDAR directly captures 3D data and therefore does not need to convert 2D records to 3D. Thus, if the vehicle encounters an obstacle, LiDAR sensors can reliably detect it early and identify its exact dimensions and location from said vehicle.

Problem with LiDAR

A significant complaint people usually have with LiDAR is its cost. The cost of placing a single LiDAR can go up to $10,000, and importing it to India can cost you around one crore rupees. Companies such as Google’s Waymo were able to reduce this due to mass production—which still doesn’t substantially reduce costs and is not feasible for everyone. 

While it is excellent at detecting things on the road, LiDAR cannot tell a potentially harmful roadblock apart from, say, a plastic bag—for which you do not need to stop a vehicle. Issues further pile up when moving objects such as pedestrians, stray dogs, and flying plastic bags appear on the road. LiDAR will not be able to detect how these move and what these objects are.

During their 2019 Autonomy Day event, Tesla assured people that their cars use a system of cameras and radar that will be able to detect what an object is. The vision here will then entail the radar alerting that something potentially harmful may be ahead, after which the camera decides—upon seeing the thing—how the car should react. It is important to note that Tesla has now ditched radar in its cars and will now only use eight cameras in the sensor suite of its best selling Model 3 and Model Y cars. This is reportedly because Musk found radar sensors relatively expensive and said that he believes ‘a vision-only system is ultimately all that is needed for full autonomy.’

Video: https://youtu.be/HM23sjhtk4Q 

It is important to realise, however, that Tesla uses its own vision technology. According to Sanjeev Sharma, LiDAR systems are more accurate than camera-based ones, which require customised hardware such as Tesla’s in-house neural network accelerators. He says that a more effective solution would be a combination of LiDARs and cameras. For example, 1 or 2 LiDARs with eight cameras could address both solutions’ cost and computational aspects. Regardless of this, Sanjeev Sharma himself found a way around working with LiDAR and its enormous cost. His company—Swaayatt Robots—has enabled autonomous vehicles that perceive their environment with the help of only cameras in real-time. According to him, his technology is 40 times faster than the state-of-the-art. He says, however, that the way around using cameras is more a question of mathematical endeavour.

The Musk of it all 

Despite the arguments for and against adopting LiDAR, one thing is certain: Musk hates LiDAR. So, why is Tesla signing a deal with Luminar? This is not clear yet. In an interesting twist, Elon Musk did appear to go softer in his disdain for LiDAR. In a Clubhouse chatroom, he said that he was ‘talking smack’ about LiDAR and that his rocket company, SpaceX, was developing an internal LiDAR to aid the SpaceX Dragon Capsule. He maintains, however, that vision is more important for driving on real-world roads. This being said, it seems highly unlikely that Musk has suddenly decided to change gears and join the LiDAR bandwagon. As per Sam Abuelsamid, an analyst with Guidehouse Insight, it is more probable for Tesla to use Luminar’s LiDARs as a benchmark to test and validate its camera-based systems. This appears to be the most likely explanation for Brulte’s photograph and subsequent tweet and the only one we can hold on to until either Tesla or Luminar explains further.