With the increased attention towards democratisation of machine learning resources, there has been a spike in the number of open source datasets that have been released over the past couple of years.
However, the efficacy of these datasets are rarely reported for many reasons. One, these datasets are primarily aimed at promoting research and not necessarily to implement in real world scenarios and two, data labeling can be a tricky task sometimes.
Addressing these issues, Brad Dwyer of Roboflow has brought to light how the widely popular dataset by Udacity has critical discrepancies such as unlabeled pedestrians, duplicated bounding boxes amongst others.
This raises the question of how much monitoring do datasets need to be subjected to be deployed for critical cases like autonomous driving and medical diagnosis.
In this article, we list a few top annotation tools that can help in reliable dataset generation:
Google’s Vision API
Google cloud’s Vision API offers label detection feature that predicts the most appropriate labels that describe an image. This tool gives the user most accurate labels after robust feature identification in the background. The features are identified over a broad range of object sets across thousands of different object categories before returning a label annotation for each detected label in an image.
With AutoML Vision, you provide labeled datasets in order to train models that perform custom label detection with your labels.
The procedure is as follows:
- A user uploads an image.
- The new image file details are configured for push delivery to the App Engine endpoint.
- App Engine calls the Vision API on the uploaded image to process and add labels to it. These labels are also added to the search index.
- App Engine calls AI Platform to classify images into user-defined categories using the detected labels.
Intel’s CVAT
Intel’s Computer Vision Annotation Tool (CVAT) is an open source tool for annotating images and videos. This tool is versatile and provides the users with convenient annotation instruments.
CVAT is a browser-based application for both individuals and teams that supports different work scenarios.
Supervisely’s AI Assisted Labeling
Supervisely’s tool allows one to label 20x faster with their SmartTool powered by AI. Neural Network inside our tool can be adapted to any industry or act as a general solution. Not to forget that you can do all this without the need to install any software — your labelers can start annotation right from the browser!
IBM’s Cloud Annotation Tool
IBM’s latest Cloud Annotations offers one of the easiest platforms for labeling images makes labeling images and training machine learning models. This annotations platform is built on top of IBM Cloud Object Storage that uses a cloud object storage offering provides a reliable place to store training data. This service by IBM also offers real time data annotation.
Google’s Fluid Annotation
Google’s Fluid Annotation allows image segmentation and annotation in the most fluidic way as the name suggests. The user just has to click over the image and the output is as shown above. This tool still has humans in the loop so that one can modify through machine-assisted edit operations using a natural user interface.
Scale’s Video Annotation
Using Scale video annotation tool
Built by machine learning engineers for machine learning engineers, Scale’s API is aimed at large scale democratisation of data driven solutions. The users just have to send the video through Scale’s simple API and can get flawless ground truth data.
According to Google AI, manual labeling of tools that require an annotator would require one to carefully click on the boundaries of the image to outline each object in the image. Labeling a single image in the COCO+Stuff dataset takes at least 19 minutes, whereas, a whole dataset would take over 53k hours!
Given how crucial and laborious labeling is, the above tools come in handy while dealing with large datasets.
