How I Created The Credit Card Reader Using OpenCV?

The purpose of this article is to build a credit card reader with OpenCV and machine learning techniques to identify the card number and the card type.

Optical character recognition has seen many applications in machine learning and deep learning. One such practical application of OCR is for identification of credit card type and number on the card. This could be of great help of banks and other financial institutions for digitally recognising the card numbers and type of card. It can also be useful for UPI services where the user can just scan the card instead of typing in the details. OCR is usually handled by a library called Tesseract. But, here we will use OpenCV to read the credit card and identify the type and number on the card. 

The purpose of this article is to build a credit card reader with OpenCV and machine learning techniques to identify the card number and the card type. Let us get started. 

Getting a Font Reference

On a typical credit card, you can see that the numbers are in a slightly different font. For these numbers to be read by the machine, we need to show the machine and save the font of these numbers. The font on the cards is called MICR which stands for magnetic ink character recognition code. So, let us now write the code in OpenCV and perform a template matching so that this font is understood for recognition later. 

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Import the Required Libraries

import cv2
import numpy as np
from matplotlib import pyplot as plt

Get the reference image

The image used for this is shown below. 

This is the type of font that is used in the cards. You can download this image here. Once this is done, let us read the image and perform some basic image processing on this. We will convert the image to greyscale and apply a binary inversion threshold on it.

Download our Mobile App

template = cv2.imread('numbers.png')
template = cv2.cvtColor(template, cv2.COLOR_BGR2GRAY)
template = cv2.threshold(template, 10, 255, cv2.THRESH_BINARY_INV)

The output of this is as shown below. 

Getting the contours for the image

Next, we need to find contours in the above image and after locating each digit we will sort them and put them in a dictionary for further reference. To contour the image we will do the following steps. 

get_contour = cv2.findContours(template.copy(), cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
get_contour = imutils.grab_contours(get_contour)
get_contour = contours.sort_contours(get_contour, method="left-to-right")
numbers = {}

Next, we will find a bounding box for these digits by looping over them and then finding the region of interest and then appending them to the dictionary created above. We will also define two kennels one rectangle and one square to get the structure of the boxes.

for (key,val) in enumerate(get_contour):
    (s1, s2, s3, s4) = cv2.boundingRect(c)
    region= template[s2:s2 + h, s1:s1 + s3]
    region= cv2.resize(region, (57, 88))
    numbers[key] = region
kernel_1 = cv2.getStructuringElement(cv2.MORPH_RECT, (10, 5))
kernel_2 = cv2.getStructuringElement(cv2.MORPH_RECT, (4, 4))

The dictionary contains keys as the numbers for 0 to 9 and the values are arrays which represent these numbers.

Now that we have the template matching ready for the card, we can go ahead and select a credit card image and read that. I have selected an image from google. You can download the image by clicking on this link.

Image processing in OpenCV

Let us now read this image and perform some basic pre-processing on the image. 

card = cv2.imread('master.png')
card = imutils.resize(card, width=300)
card_grey = cv2.cvtColor(card, cv2.COLOR_BGR2GRAY)

The image in greyscale looks like this

credit card

Now we need to separate the light-coloured numbers from the dark background so that they can be detected. To do this, we will use a morphological operation called top-hat. Then, we will use a gradient called Sobel gradient and scale the image. 

Morphology and scaling

morph = cv2.morphologyEx(card_grey, cv2.MORPH_TOPHAT, kernel_1)
sobel = cv2.Sobel(morph, ddepth=cv2.CV_32F, dx=1, dy=0, ksize=-1)
sobel = np.absolute(sobel)
(minimum, maximum) = (np.min(sobel), np.max(sobel))
sobel = (255 * ((sobel - minimum) / (maximum - minimum)))

The morphed image looks like this:

credit card

The scaled image is as follows


As you can see above, there are gaps that exist between the numbers and it can become a hurdle for the machine as they consider even the gaps as characters. To close them we need to use a technique called Otsu threshold. 

otsu = cv2.morphologyEx(sobel, cv2.MORPH_CLOSE, kernel_1)
out = cv2.threshold(otsu, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
out = cv2.morphologyEx(out, cv2.MORPH_CLOSE, kernel_2)

The next step is to group the digits and find their locations and apply the bounding boxes around them. To do this we will have to loop through the contours and apply the boxes. 

Bounding boxes

contours = cv2.findContours(out.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = imutils.grab_contours(contours)
locations = []
for (key, vals) in enumerate(contours):
    (s1, s2, s3, s4) = cv2.boundingRect(vals)
    arr = s3 / float(s4)
    if arr > 2.5 and arr < 4.0:
        if (s3 > 40 and s3 < 55) and (s4 > 10 and s4 < 20):
            locations.append((s1, s2, s3, s4))
locations = sorted(locations, key=lambda s1:s1[0])
final_locs = []

Now that we got the locations and applied the boxes around the numbers, we now need to perform the final part which is the OCR on these 4 groups of numbers. 

Extracting the numbers

To do this phase we will go over the groups of digits and collect the region of interests from the bounding boxes. Then we will apply contours on them and sort the numbers from left to right. Next, we will extract the numbers in the bounding boxes and then compare them with our template to identify the digits. Finally, we will apply a correlation-based template matching and store the outputs. 

for (idx, (group1, group2, group3, group4)) in enumerate(locations):
    detection = []
get_group = gray[group2 - 5:group2 + group4 + 5, group1 - 5:group1 + group3 + 5]
    get_group = cv2.threshold(get_group, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
    nums = cv2.findContours(get_group.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    nums = imutils.grab_contours(nums)
    nums = contours.sort_contours(nums, method="left-to-right")[0]
    for val in nums:
        (s1, s2, s3, s4) = cv2.boundingRect(val)
        regions = get_group[s2:s2 + s4, s1:s1 + s3]
        regions = cv2.resize(regions, (57, 88))
        final_vals = []
        for (num, numROI) in numbers.items():
            res = cv2.matchTemplate(regions, numROI, cv2.TM_CCOEFF)
            (_, mark, _, _) = cv2.minMaxLoc(res)
    cv2.rectangle(image, (group1 - 5, group2 - 5),
        (group1 + group3 + 5, group2 + group4 + 5), (0, 0, 255), 2)
    cv2.putText(image, "".join(detection), (group1, group2 - 15),
        cv2.FONT_HERSHEY_SIMPLEX, 0.65, (0, 0, 255), 2)

Lastly, for the detection of the type of card, we need a dictionary that contains the first numbers as key and name of the card as values. Usually, in Indian cards, visa cards start with 4 and MasterCard start with 5.

starting_digit = {
    "4": "Visa",
    "5": "MasterCard"

Now we can print the output

print("Credit Card Type: {}".format(starting_digit[final_locs[0]]))
print("Credit Card digits: {}".format("".join(final_locs)))
plt.imshow(cv2.cvtColor(card, cv2.COLOR_BGR2RGB))

I will show the output for a visa card as well. I have just replaced the image and ran the program again.

optical character recognition


In this article, we saw how to build a credit card reader using OpenCV and OCR. We also identified the type of card with the help of the first number. With OpenCV, building an optical character recognition is simple and is accurate as well. Similar to cards this can be applied on cheques as well.

Sign up for The Deep Learning Podcast

by Vijayalakshmi Anandan

The Deep Learning Curve is a technology-based podcast hosted by Vijayalakshmi Anandan - Video Presenter and Podcaster at Analytics India Magazine. This podcast is the narrator's journey of curiosity and discovery in the world of technology.

Bhoomika Madhukar
I am an aspiring data scientist with a passion for teaching. I am a computer science graduate from Dayananda Sagar Institute. I have experience in building models in deep learning and reinforcement learning. My goal is to use AI in the field of education to make learning meaningful for everyone.

Our Upcoming Events

24th Mar, 2023 | Webinar
Women-in-Tech: Are you ready for the Techade

27-28th Apr, 2023 I Bangalore
Data Engineering Summit (DES) 2023

23 Jun, 2023 | Bangalore
MachineCon India 2023 [AI100 Awards]

21 Jul, 2023 | New York
MachineCon USA 2023 [AI100 Awards]

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Council Post: The Rise of Generative AI and Living Content

In this era of content, the use of technology, such as AI and data analytics, is becoming increasingly important as it can help content creators personalise their content, improve its quality, and reach their target audience with greater efficacy. AI writing has arrived and is here to stay. Once we overcome the initial need to cling to our conventional methods, we can begin to be more receptive to the tremendous opportunities that these technologies present.