Now Reading
5 Object Detection Evaluation Metrics That Data Scientists Should Know

5 Object Detection Evaluation Metrics That Data Scientists Should Know

Prudhvi varma

Download our Mobile App


In computer vision, object detection is one of the powerful algorithms, which helps in the classification and localization of the object. Object detection is more challenging because it needs to draw a bounding box around each object in the image. While going through research papers you may find these terms AP, IOU, mAP, these are nothing but Object detection metrics that help in finding good models. In this article, we will demonstrate how to get insights and a clear understanding of each from these metrics.

Basic Object detection metrics covered in this article

  • Introduction to the precision and recall
  • Intersection over Union(IOU)
  • Average Precision(AP)
  • Mean Average Precision(mAP)
  • Variations among mAP

Introduction to Precision and Recall

Precision – It is used to measure the correct predictions.



Recall – it is used to calculate the true predictions from all correctly predicted data.

precision and recall formulas

Intersection Over Union(IOU)

IOU is a metric that finds the difference between ground truth annotations and predicted bounding boxes. This metric is used in most state of art object detection algorithms. In object detection, the model predicts multiple bounding boxes for each object, and based on the confidence scores of each bounding box it removes unnecessary boxes based on its threshold value. We need to declare the threshold value based on our requirements.

              IOU =   Area of union /  area of intersection

def IOU(box1, box2):

    x1, y1, w1, h1 = box1


Stay Connected

Get the latest updates and relevant offers by sharing your email.

    x2, y2, w2, h2 = box2

    w_intersection = min(x1 + w1, x2 + w2) - max(x1, x2)

    h_intersection = min(y1 + h1, y2 + h2) - max(y1, y2)

    if w_intersection <= 0 or h_intersection <= 0: 

        return 0

    I = w_intersection * h_intersection

    U = w1 * h1 + w2 * h2 - I 

    return I / U

iou = [IOU(y_test[i], y_pred[i]) for i in range(len(x_test))]

Ground truth image

By using IOU

The model predicted image without using IoU looks like

bounding boxes generated by model without using IOU

After setting some threshold value it removes the unnecessary boxes based on the confidence scores.

Average Precision(AP) 

To evaluate the detection commonly we use precision-recall curve but average precision gives the numerical values it is easy to compare the performance with other models. Based on the precision-recall curve AP it summarises the weighted mean of precisions for each threshold with the increase in recall. Average precision is calculated for each object.

Average precision formula

From the above formula, P refers to precision and R refers to Recall suffix n denotes the different threshold values.

import numpy as np

from sklearn.metrics import average_precision_score

ground_truth = np.array([0, 0, 1, 1])

model_predicted_confidences = np.array([0.1, 0.4, 0.35, 0.8])

average_precision_score(ground_truth, model_predicted_confidences)

See Also

Avarage Precision result

In the above output, we achieved 0.83333 average precision based on the confidence scores.

Mean Average Precision(mAP) 

Mean average precision is an extension of Average precision. In Average precision, we only calculate individual objects but in mAP, it gives the precision for the entire model. To find the percentage correct predictions in the model we are using mAP.

Here N denoted the number of objects.

mAP= [0.83,0.66,0.99,0.78,0.60]

a=len(mAP)

b=sum(mAP)

c=a/b

print(c)

mAP result

Each object has its individual average precision values, we are adding all these values to find Mean Average precision.

Variations Among  mAP

In most of the research papers, these metrics will have extensions like mAP iou = 0.5, mAP iou = 0.75, mAP small, medium, large. In this, we will clearly demonstrate what it actually means.

  • mAP iou=0.5 represents the model has used 0.5 threshold value to remove unnecessary bounding boxes, it is the standard threshold value for most of the models.
  • mAP iou=0.75 represents the model has used 0.75 threshold value, By using this we can get accurate results by removing bounding boxes with less than 25% of the intersection with ground truth image.
  • mAP small represents the model has given mAP score based on smaller objects in the data.
  • mAP large represents the model has given mAP score based on larger objects in the data.

Conclusion 

In the above article, we demonstrated what are the basic object detection metrics by comparing the predicted output and ground truth values and their variations,  These metrics are used by most of the state of art models to find the prediction in the model.

What Do You Think?

If you loved this story, do join our Telegram Community.


Also, you can write for us and be one of the 500+ experts who have contributed stories at AIM. Share your nominations here.
What's Your Reaction?
Excited
0
Happy
2
In Love
0
Not Sure
1
Silly
0

Copyright Analytics India Magazine Pvt Ltd

Scroll To Top