Now an AI Scarecrow to shoo away pigeons

Pigeons are called flying rats for a reason. Not to forget the dirty mess these birds leave behind. The control strategies developed over the years to chase them away have been either ineffective or harmful.

To this end, researchers at EPFL in Switzerland have developed and deployed an autonomous system that can spot roof-invading pigeons and send a drone to scare the birds off.

Pigeons are the most persistent and invasive species among birds and are estimated to cause an annual damage of USD 1.1 billion in the United States alone.

Fabrizio Schiano, Dominik Natter, Davide Zambrano, and Dario Floreano from the Laboratory of Intelligent Systems, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland are behind the system.

System explained

The paper proposes a system capable of autonomously detecting and deterring pigeons on building roofs using a drone. The presence and position of pigeons were detected in real-time by a neural network using images taken by a video camera located on the roof. A drone was used to deter the animals.

The system consisted of three hardware modules: a camera, ground station, and drone. The ground station commands the camera to scan the environment and receive images. A neural object detector was trained using pigeon images to identify the bounding box in an image in which pigeons are present. Then, the position of the bounding box in the two-dimensional image space is translated into the three-dimensional global navigation satellite systems coordinates. The drone flies over the identified coordinates (i.e., detected pigeons) before returning to its home base. 

The team conducted field experiments in a real-world urban setting to assess the proposed system. They compared the number of animals and their stay durations for over five days against the 21-day-trial experiment without the drone. During the five days of experiments, the drone was automatically deployed 55 times and was significantly effective in reducing the number of birds and their stay durations without causing any harm to them.

Though drones are already being used for bird control, it’s with an active human pilot using a drone to scare flocks of birds at specific places and times: in many countries, it is illegal to fly autonomous drones.

Switzerland also has strict drone regulations. However, the research involved a human supervisor on standby, ready to jump in and take if the fully autonomous system suffered any glitch. The study has proven the effectiveness of this system in deterring birds.

Steps

The Camera: The system relies on a PTZ camera, which is weather-resistant and can oversee its full surroundings using a 360° pan, mounted at a fixed position in the environment. The combination of a 12× optical zoom and 4 MP resolution enables detailed representation of the environment.

Pigeon Detection: In this study, the researchers have combined two ideas. ‘Application of deep-learning methods to bird detection using unmanned aerial vehicle imagery’ as it compared different object detectors used for detecting birds with an aerial view, and ‘Faster R-CNN: Towards real-time object detection with region proposal networks’ as this faster region-based convolutional neural network was the most accurate approach. The study proposed a detector model with pre-trained weights, which the team used to fine-tuned specifically collected images to counteract overfitting.

The Faster R-CNN is a two-stage object detector where the first stage (CNN) extracts features from the image and proposes image regions where objects are supposedly located. The second stage involves a neural network that predicts the bounding boxes – object class and coordinates – in the proposed regions.

The resulting model turned out to be one of the best performing object detectors on the MSCOCO dataset (mean Average Precision, mAP, is 38.7) in the Tensorflow Model Zoo (for the Tensorflow Object Detection API).

The model was pre-trained on the MS-COCO dataset and fine-tuned with specifically collected images of pigeons in an urban environment. The camera was mounted on the roof of a building and collected over 30 hours of video footage following a predefined routine of PTZ commands. The amount of data was reduced through random sampling and subsequent inspection of the images. The model was allowed to distinguish between two different classes. Then, the team balanced the number of images per class by performing image augmentation (e.g., random affine transforms, colour-channel swaps, and noise addition) on the other class, which resulted in 2,539 images that were equally divided for final training and testing. Then, 10% of the training set was isolated for validation.

Position estimation: The team converted the bounding box generated by the detector into GPS coordinates to send the drone closer to the pigeons.

The team adopted an approach that requires leveraging only the dimensions and position of the bounding boxes and assumed the pigeon height is known and constant. Given the bounding box height as a percentage of the image height and a pinhole camera model, the distance between the camera and pigeon was then calculated.

The drone: This is the final step. The GPS coordinates from the position estimation serve as GPS targets for the drone. The team used the Parrot Anafi drone that enables autonomous flights based on GPS targets. Since Anafi has a low sound level and is weightless, it was suitable for urban environment applications. Moreover, the development of algorithms to control the drone was sped up using the simulation software based on Gazebo. The team built a one-to-one simulation environment that enabled them to test the entire pipeline in the simulation.

Wrapping up

The overall system was quite successful. While on average, a pigeon flock could spend up to 2.5 hours on a roof and create a mess, the maximum bird loitering time by the drone was cut down to just a couple of minutes. This includes the minutes it took to do the detection and actually launch the drone.