How I Created The Credit Card Reader Using OpenCV?

The purpose of this article is to build a credit card reader with OpenCV and machine learning techniques to identify the card number and the card type.

Optical character recognition has seen many applications in machine learning and deep learning. One such practical application of OCR is for identification of credit card type and number on the card. This could be of great help of banks and other financial institutions for digitally recognising the card numbers and type of card. It can also be useful for UPI services where the user can just scan the card instead of typing in the details. OCR is usually handled by a library called Tesseract. But, here we will use OpenCV to read the credit card and identify the type and number on the card. 

The purpose of this article is to build a credit card reader with OpenCV and machine learning techniques to identify the card number and the card type. Let us get started. 

Getting a Font Reference

On a typical credit card, you can see that the numbers are in a slightly different font. For these numbers to be read by the machine, we need to show the machine and save the font of these numbers. The font on the cards is called MICR which stands for magnetic ink character recognition code. So, let us now write the code in OpenCV and perform a template matching so that this font is understood for recognition later. 


Sign up for your weekly dose of what's up in emerging technology.

Import the Required Libraries

import cv2
import numpy as np
from matplotlib import pyplot as plt

Get the reference image

The image used for this is shown below. 

This is the type of font that is used in the cards. You can download this image here. Once this is done, let us read the image and perform some basic image processing on this. We will convert the image to greyscale and apply a binary inversion threshold on it.

Download our Mobile App

template = cv2.imread('numbers.png')
template = cv2.cvtColor(template, cv2.COLOR_BGR2GRAY)
template = cv2.threshold(template, 10, 255, cv2.THRESH_BINARY_INV)

The output of this is as shown below. 

Getting the contours for the image

Next, we need to find contours in the above image and after locating each digit we will sort them and put them in a dictionary for further reference. To contour the image we will do the following steps. 

get_contour = cv2.findContours(template.copy(), cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
get_contour = imutils.grab_contours(get_contour)
get_contour = contours.sort_contours(get_contour, method="left-to-right")
numbers = {}

Next, we will find a bounding box for these digits by looping over them and then finding the region of interest and then appending them to the dictionary created above. We will also define two kennels one rectangle and one square to get the structure of the boxes.

for (key,val) in enumerate(get_contour):
    (s1, s2, s3, s4) = cv2.boundingRect(c)
    region= template[s2:s2 + h, s1:s1 + s3]
    region= cv2.resize(region, (57, 88))
    numbers[key] = region
kernel_1 = cv2.getStructuringElement(cv2.MORPH_RECT, (10, 5))
kernel_2 = cv2.getStructuringElement(cv2.MORPH_RECT, (4, 4))

The dictionary contains keys as the numbers for 0 to 9 and the values are arrays which represent these numbers.

Now that we have the template matching ready for the card, we can go ahead and select a credit card image and read that. I have selected an image from google. You can download the image by clicking on this link.

Image processing in OpenCV

Let us now read this image and perform some basic pre-processing on the image. 

card = cv2.imread('master.png')
card = imutils.resize(card, width=300)
card_grey = cv2.cvtColor(card, cv2.COLOR_BGR2GRAY)

The image in greyscale looks like this

credit card

Now we need to separate the light-coloured numbers from the dark background so that they can be detected. To do this, we will use a morphological operation called top-hat. Then, we will use a gradient called Sobel gradient and scale the image. 

Morphology and scaling

morph = cv2.morphologyEx(card_grey, cv2.MORPH_TOPHAT, kernel_1)
sobel = cv2.Sobel(morph, ddepth=cv2.CV_32F, dx=1, dy=0, ksize=-1)
sobel = np.absolute(sobel)
(minimum, maximum) = (np.min(sobel), np.max(sobel))
sobel = (255 * ((sobel - minimum) / (maximum - minimum)))

The morphed image looks like this:

credit card

The scaled image is as follows


As you can see above, there are gaps that exist between the numbers and it can become a hurdle for the machine as they consider even the gaps as characters. To close them we need to use a technique called Otsu threshold. 

otsu = cv2.morphologyEx(sobel, cv2.MORPH_CLOSE, kernel_1)
out = cv2.threshold(otsu, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
out = cv2.morphologyEx(out, cv2.MORPH_CLOSE, kernel_2)

The next step is to group the digits and find their locations and apply the bounding boxes around them. To do this we will have to loop through the contours and apply the boxes. 

Bounding boxes

contours = cv2.findContours(out.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = imutils.grab_contours(contours)
locations = []
for (key, vals) in enumerate(contours):
    (s1, s2, s3, s4) = cv2.boundingRect(vals)
    arr = s3 / float(s4)
    if arr > 2.5 and arr < 4.0:
        if (s3 > 40 and s3 < 55) and (s4 > 10 and s4 < 20):
            locations.append((s1, s2, s3, s4))
locations = sorted(locations, key=lambda s1:s1[0])
final_locs = []

Now that we got the locations and applied the boxes around the numbers, we now need to perform the final part which is the OCR on these 4 groups of numbers. 

Extracting the numbers

To do this phase we will go over the groups of digits and collect the region of interests from the bounding boxes. Then we will apply contours on them and sort the numbers from left to right. Next, we will extract the numbers in the bounding boxes and then compare them with our template to identify the digits. Finally, we will apply a correlation-based template matching and store the outputs. 

for (idx, (group1, group2, group3, group4)) in enumerate(locations):
    detection = []
get_group = gray[group2 - 5:group2 + group4 + 5, group1 - 5:group1 + group3 + 5]
    get_group = cv2.threshold(get_group, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
    nums = cv2.findContours(get_group.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    nums = imutils.grab_contours(nums)
    nums = contours.sort_contours(nums, method="left-to-right")[0]
    for val in nums:
        (s1, s2, s3, s4) = cv2.boundingRect(val)
        regions = get_group[s2:s2 + s4, s1:s1 + s3]
        regions = cv2.resize(regions, (57, 88))
        final_vals = []
        for (num, numROI) in numbers.items():
            res = cv2.matchTemplate(regions, numROI, cv2.TM_CCOEFF)
            (_, mark, _, _) = cv2.minMaxLoc(res)
    cv2.rectangle(image, (group1 - 5, group2 - 5),
        (group1 + group3 + 5, group2 + group4 + 5), (0, 0, 255), 2)
    cv2.putText(image, "".join(detection), (group1, group2 - 15),
        cv2.FONT_HERSHEY_SIMPLEX, 0.65, (0, 0, 255), 2)

Lastly, for the detection of the type of card, we need a dictionary that contains the first numbers as key and name of the card as values. Usually, in Indian cards, visa cards start with 4 and MasterCard start with 5.

starting_digit = {
    "4": "Visa",
    "5": "MasterCard"

Now we can print the output

print("Credit Card Type: {}".format(starting_digit[final_locs[0]]))
print("Credit Card digits: {}".format("".join(final_locs)))
plt.imshow(cv2.cvtColor(card, cv2.COLOR_BGR2RGB))

I will show the output for a visa card as well. I have just replaced the image and ran the program again.

optical character recognition


In this article, we saw how to build a credit card reader using OpenCV and OCR. We also identified the type of card with the help of the first number. With OpenCV, building an optical character recognition is simple and is accurate as well. Similar to cards this can be applied on cheques as well.

More Great AIM Stories

Bhoomika Madhukar
I am an aspiring data scientist with a passion for teaching. I am a computer science graduate from Dayananda Sagar Institute. I have experience in building models in deep learning and reinforcement learning. My goal is to use AI in the field of education to make learning meaningful for everyone.

AIM Upcoming Events

Early Bird Passes expire on 3rd Feb

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Do machines feel pain?

Scientists worldwide have been finding ways to bring a sense of awareness to robots, including feeling pain, reacting to it, and withstanding harsh operating conditions.

IT professionals and DevOps say no to low-code

The obsession with low-code is led by its drag-and-drop interface, which saves a lot of time. In low-code, every single process is shown visually with the help of a graphical interface that makes everything easier to understand.

Neuralink elon musk

What could go wrong with Neuralink?

While the broad aim of developing such a BCI is to allow humans to be competitive with AI, Musk wants Neuralink to solve immediate problems like the treatment of Parkinson’s disease and brain ailments.

Understanding cybersecurity from machine learning POV 

Today, companies depend more on digitalisation and Internet-of-Things (IoT) after various security issues like unauthorised access, malware attack, zero-day attack, data breach, denial of service (DoS), social engineering or phishing surfaced at a significant rate.