YOLO – Where Have All the Airplanes Gone?

Advertisement

My 4-year-old pointed to the sky one day and asked, why don’t we see airplanes flying over our house nowadays? (10 Little Airplanes (1) is his favorite counting rhyme). I showed him the below picture and explained that airplanes are grounded due to COVID19. He started counting the aircraft in the picture. Well, what’s your count?

Delta Air Lines jets parked in Victorville, California, on March 28. Source: AirTeamImages

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

As a Data Science practitioner, a problem statement surfaced – Can we count the number of airplanes parked across various locations globally. With help from my colleague(2), we set out to stitch a quick but efficient deep learning solution that can count airplanes from satellite images. The solution had to perform two operations – Object detection and classification. 

Step # 1 – Getting the Data: We scouted for available satellite imagery that was taken after March 2020. A quick Google image search gave us no satellite images that covered airports or airplane parking lots worldwide. We opted to gather satellite images from 20 airports globally from Maxar Technologies.(3)

Step # 2 – Preparing the Data

To minimize the effort we invest in creating training data for model building, we scouted for labeled data of airplanes. We found two sources with labeled data a) CGI Planes in Satellite Imagery w/BBoxes(4). This contained satellite images of planes flying in the air and b) DOTA V1.5 gallery(5). This included pictures of aircraft parked on the ground. 

Figure: Images of satellite imagery obtained from the Kaggle CGI plane dataset. 

Figure: Images of satellite imagery obtained from the DOTA-v1.5. 

Step # 3 – Building the Model

YOLO v4(6) literature talks about state-of-the-art methods CBN and PAN that make it more efficient and suitable for single GPU training. We wanted to build an airplane detection model that combines the power of YOLO v4 and transfer learning. We set out to accomplish this with minimal hyper-parameter tuning. We took a pre-trained model trained on the ImageNet dataset and used transfer learning to train the satellite image data.

The first iteration of training was done on the CGI dataset with 500 images (400 to 100 split). The trained model was tested on the entire satellite imagery that we collected. 

For the second iteration, we trained the model on the DOTA V1.5 dataset with 200 images. The retrained model was tested on the entire satellite dataset again. 

Training considerations

  1. A GPU laptop with Nvidia GTX 1070 graphics card, 8 GB VRAM/ GDDR5
  2. It took about 6 hours from start to finish. The speed of execution can be attributed to the transfer learning approach applied to the custom dataset
  3. The weights were trained on the ImageNet dataset
  4. The height and width of each image was 608×608 pixel
  5. The training was done on 3000 epochs
  6. We considered the objects as planes for whose confidence score was greater than 0.2.

Step # 4 – Result Time

We considered recall as the evaluation metric as the model’s precision was quite good. When the model was trained on the CGI dataset, the recall was 70%. When the model was retrained on DOTA V1.5 images, the recall jumped to 83.4%. The code repository (7) contains the configuration and weights we used in the model. 

Here’s the sample output of airplanes tagged correctly by model:

Figure: Output of the model on Schiphol Airport Satellite image ©2020 Maxar Technologies.

Figure: Output of the model on Phoenix Airport. Satellite image ©2020 Maxar Technologies.

The model failed to identify airplanes with acceptable confidence in the following situations:

  1. The airplane’s shadow was prominent
  2. The surface color and the airplane color were  a close match
  3. Airplanes that were parked in a row and too close to each other
Figure: Output of the model on Abu Dhabi, Victorville, and Hongkong Airport. Satellite image ©2020 Maxar Technologies.

This short pilot demonstrates the potential of Transfer Learning to expedite AI-based solution building far lesser computation and costs. 

With a gloomy outlook for the global airline industry, the only ones seeing a business boom are the airplane storage service providers, right from California and France to Australia. 

“I heard an airplane passing overhead. I wished I was on it.”

Charles Bukowski

Answer time – No.of planes in the first picture is 74


References

  1. 10 little Airplanes – https://www.youtube.com/watch?v=VNRNAloCb5Y
  2. Nanda Kishore – https://www.linkedin.com/in/nanda-kishore-mallapragada-91b73864/
  3. Maxar Technologies – https://www.maxar.com/
  4. CGI Planes in Satellite images – 

https://www.kaggle.com/aceofspades914/cgi-planes-in-satellite-imagery-w-bboxes

  1. DOTA-v1.5 – https://captain-whu.github.io/DOAI2019/dataset.html
  2. YOLO – https://github.com/AlexeyAB/darknet
  3. Code Repository – https://github.com/gramener/planecounting

More Great AIM Stories

Sundeep Reddy Mallu
Sundeep heads the Analytics & Hiring teams at Gramener, a data science company that solves business problems by conveying insights as data stories. Sundeep advises enterprise executives on data science and helps them accelerate digital transformation through Advanced Analytics techniques. He is passionate about the convergence of data-driven business leadership, AI/ML adoption across industries, AI ethics, and data privacy. As TA head he drives the hunt for great data science team players across industries and campuses and attracts them to join team Gramener.

Our Upcoming Events

Conference, in-person (Bangalore)
MachineCon 2022
24th Jun

Conference, Virtual
Deep Learning DevCon 2022
30th Jul

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
MORE FROM AIM
Sreejani Bhattacharyya
Why is edtech falling first?

With the lockdown being imposed due to the COVID-19 pandemic and schools being shut down, the edtech startups witnessed some of their best times during 2020 and 2021.