YOLO – Where Have All the Airplanes Gone?

My 4-year-old pointed to the sky one day and asked, why don’t we see airplanes flying over our house nowadays? (10 Little Airplanes (1) is his favorite counting rhyme). I showed him the below picture and explained that airplanes are grounded due to COVID19. He started counting the aircraft in the picture. Well, what’s your count?

Delta Air Lines jets parked in Victorville, California, on March 28. Source: AirTeamImages

As a Data Science practitioner, a problem statement surfaced – Can we count the number of airplanes parked across various locations globally. With help from my colleague(2), we set out to stitch a quick but efficient deep learning solution that can count airplanes from satellite images. The solution had to perform two operations – Object detection and classification. 

Step # 1 – Getting the Data: We scouted for available satellite imagery that was taken after March 2020. A quick Google image search gave us no satellite images that covered airports or airplane parking lots worldwide. We opted to gather satellite images from 20 airports globally from Maxar Technologies.(3)

Step # 2 – Preparing the Data

To minimize the effort we invest in creating training data for model building, we scouted for labeled data of airplanes. We found two sources with labeled data a) CGI Planes in Satellite Imagery w/BBoxes(4). This contained satellite images of planes flying in the air and b) DOTA V1.5 gallery(5). This included pictures of aircraft parked on the ground. 

Figure: Images of satellite imagery obtained from the Kaggle CGI plane dataset. 

Figure: Images of satellite imagery obtained from the DOTA-v1.5. 

Step # 3 – Building the Model

YOLO v4(6) literature talks about state-of-the-art methods CBN and PAN that make it more efficient and suitable for single GPU training. We wanted to build an airplane detection model that combines the power of YOLO v4 and transfer learning. We set out to accomplish this with minimal hyper-parameter tuning. We took a pre-trained model trained on the ImageNet dataset and used transfer learning to train the satellite image data.

The first iteration of training was done on the CGI dataset with 500 images (400 to 100 split). The trained model was tested on the entire satellite imagery that we collected. 

For the second iteration, we trained the model on the DOTA V1.5 dataset with 200 images. The retrained model was tested on the entire satellite dataset again. 

Training considerations

1. A GPU laptop with Nvidia GTX 1070 graphics card, 8 GB VRAM/ GDDR5
2. It took about 6 hours from start to finish. The speed of execution can be attributed to the transfer learning approach applied to the custom dataset
3. The weights were trained on the ImageNet dataset
4. The height and width of each image was 608×608 pixel
5. The training was done on 3000 epochs
6. We considered the objects as planes for whose confidence score was greater than 0.2.

Step # 4 – Result Time

We considered recall as the evaluation metric as the model’s precision was quite good. When the model was trained on the CGI dataset, the recall was 70%. When the model was retrained on DOTA V1.5 images, the recall jumped to 83.4%. The code repository (7) contains the configuration and weights we used in the model. 

Here’s the sample output of airplanes tagged correctly by model:

Figure: Output of the model on Schiphol Airport Satellite image ©2020 Maxar Technologies.

Figure: Output of the model on Phoenix Airport. Satellite image ©2020 Maxar Technologies.

The model failed to identify airplanes with acceptable confidence in the following situations:

1. The airplane’s shadow was prominent
2. The surface color and the airplane color were  a close match
3. Airplanes that were parked in a row and too close to each other

This short pilot demonstrates the potential of Transfer Learning to expedite AI-based solution building far lesser computation and costs. 

With a gloomy outlook for the global airline industry, the only ones seeing a business boom are the airplane storage service providers, right from California and France to Australia. 

“I heard an airplane passing overhead. I wished I was on it.”

Charles Bukowski

Answer time – No.of planes in the first picture is 74

References

1. 10 little Airplanes – https://www.youtube.com/watch?v=VNRNAloCb5Y
2. Nanda Kishore – https://www.linkedin.com/in/nanda-kishore-mallapragada-91b73864/
3. Maxar Technologies – https://www.maxar.com/
4. CGI Planes in Satellite images – 

https://www.kaggle.com/aceofspades914/cgi-planes-in-satellite-imagery-w-bboxes

5. DOTA-v1.5 – https://captain-whu.github.io/DOAI2019/dataset.html
6. YOLO – https://github.com/AlexeyAB/darknet
7. Code Repository – https://github.com/gramener/planecounting