Today, we are surrounded by a variety of products which makes it difficult for one to identify the best or the combination of the best products. Businesses are flooded with millions of transactions every day and it becomes ever more important to understand the choices and preferences of customers’ buying behaviour. People use a variety of strategies to make choices about what to buy and how much to spend. Recommender Systems automate some of these strategies with the goal of providing affordable, personal and high-quality recommendations. Given below are the approaches to develop a state-of-the-art Recommender System.

WHAT IS A RECOMMENDER SYSTEM?

Recommender System or algorithms start by finding a similarity score over the customers’ purchased and rated items which overlaps the other users’ purchased and rated items. The algorithm aggregates items from these similar customers, removes items the user has already purchased or rated, and recommends the remaining items to the user. Two popular versions of these algorithms are Collaborative Filtering and Cluster Models. Other algorithms like Market Basket Analysis and Association Rule Mining are also used by the industry that focuses on finding similar items, and not similar customers.

WHAT WE NEED TO RECOMMEND?

So, if a customer has not yet bought item 2,3,5 and 7, the Recommender System will calculate the similarity of the customer purchased items with his non-purchased items and will recommend the best items which are similar to his purchased items.

IMPLEMENTING A RECOMMENDER SYSTEM

Recommendation algorithms provide an effective form of targeted marketing by creating a personalised shopping experience for each customer. For large online retailers, a good recommendation algorithm should be scalable over very large customer bases and product catalogue and should generate compelling recommendations for all users regardless of the number of purchases and ratings.

Item-to-Item Collaborative Filtering which we are showcasing below in this report is able to meet these challenges.

We move a step further and try to present cases where retail industry can broadly apply recommendation algorithms for targeted marketing, both online and offline. With the techniques to measure conversion rate, click rate, and open rate it has become effective for offline retailers to use Recommendation System in postal mailings, coupons, POS driven promotions and other forms of customer communication.

There are three common approaches to solving the recommendation problem: Cluster Models, Collaborative Filtering and Association Rule Mining.

CLUSTER MODELS

To find customers who are similar to the user, Cluster Models divide the customer base into many segments and treat the task as a classification problem. The algorithm’s goal is to assign the user to the segment containing the most similar customers. It then uses the purchases and ratings of the customers in the segment to generate recommendations. The segments typically are created using a clustering or other machine learning algorithm, although some applications use manually determined segments.

ITEM-TO-ITEM COLLABORATIVE FILTERING

Rather than matching the user to similar customers, Item-to-Item Collaborative Filtering matches each of the users’ purchased and rated items to similar items, then combines those similar items into a recommendation list. To determine the most-similar match for a given item, the algorithm builds a similar-items table by finding items that customers tend to purchase together. We could build a product-to-product matrix by iterating through all item pairs and computing a similarity metric for each pair.

ASSOCIATION RULES

It’s a rule-based machine learning method for finding frequent patterns and analysing item sets in customers’ basket or transactions and identifying frequently occurring item sets, which can be the basis for recommendations also known as Market Basket Analysis in Retail

Not going much into the statistical part, we can clearly see that in the previous baskets insect killer and deo combination was present in 3 out of the 4 baskets. This shows that whenever insect killer was bought, deo was bought along with it (one of the arguments in favour of this correlation can be attributed to men buying insect killer and deo combination as a top up when they visit super market). This is an intro to how Market Basket Analysis works.

ILLUSTRATION 1

Below are the few illustrations of Recommendation Engine live in action that we have compiled:

Recommendations for online grocery purchases based on historical transactions. Such recommendations can easily be deployed live on online ecommerce portals.

We have used Item-to-Item Collaborative Filtering approach which uses similarity scores between two items and recommends the top items using the similarity score.

This method also solves the problem of recommending new items as the algorithm compares the new item similarity with the existing items and then will recommend the new items alongside the existing items bought by the user.

Let’s say, a new beer-based shampoo is introduced in the catalogue. The algorithm will calculate the similarity of this product with all the existing products in the catalogue. The products with the highest similarity scores with the beer-based shampoo will be recommended to the user who will buy those products or who have already bought those products. On an intuitive level, it is not hard to see that beer-based shampoo will be recommended to beer buyers, shampoo and other toiletries buyers.

In the table given below, Rec1 to Rec8 are the top eight recommendations of the items list. So, after a customer has added citrus fruit in the cart, the Recommender System will recommend bread and yogurt as the top recommendations.

ILLUSTRATION 2

Recommendations for Tours and Travels brands and portals based on the historical trips made by the customers

Below is the recommendation tracker for few of the destinations that Tours and Travels portal has under his umbrella based on travel destinations opted by the customers previously.

Now the merchant knows that a customer who has visited Goa has higher affinity to visit Alibagh, Coorg, Mahabaleshwar, Shirdi and Ganpatipule. These combinations of Goa vs the top 5 recommendations can be suggested to customers under different packages.

This will inform customers about what their fellow travellers did and will garner additional business for the merchant.

In below table, Rec1 to Rec5 are the top five recommendations of the items list. So, after a customer has travelled to Goa, the Recommender System will recommend Alibagh, Coorg as the top recommendations for his/her next visit.

Retail industry can broadly apply recommendation algorithms for targeted marketing, both online and offline. With the techniques to measure conversion rate, click rate and open rate, it has become effective for offline retailers to use the Recommendation System in postal mailings, coupons, POS driven/Real time in-store promotions and other forms of customer communication.

[divider size=”1″]

Authors

Nishit Mittal

Nishit Mittal is working with the Consulting-Econometrics and analytics research team at Hansa Cequity and is based out of Mumbai office. Nishit is a data science enthusiast and is trained in data science and business analytics from IIM Bangalore and LSE. He has worked as an economist with RBS and UBS and as a consultant with NCAER before joining Hansa Cequity. He can be reached at nishit.mittal@cequitysolutions.com

Ankit Patel

Ankit Patel is an alumnus from DA-IICT(Dhirubhai Ambani Institute for ICT) and currently working with the analytics team at Hansa Cequity. He leverages his skills to explore more meaningful avenue for analytics with the clients at Hansa Cequity. He can be reached at ankit.patel@cequitysolutions.com