Mining Customer Loyalty Data for Insights on Purchasing Behaviour

We hope to answer the following key questions in this case study

  • What does Loyalty card do for a retailer which they cannot observe from a basket or invoice level details?
  • How to measure the success of a loyalty program?
  • What can the loyalty data be mined for?
  • What are various ways the data can be mined?


A typical Customer goes through 5 stages in their Lifecycle as shown in the diagram below

d1 At every stage, analytics can be used as an aid to take key decisions for


Sign up for your weekly dose of what's up in emerging technology.
  1. Right Prospect / Customer – based on estimates of Lifetime Profitability
  2. Right Channel – based on estimates of  response rates & ROI
  3. Right Message – based on resonance with target customer group
  4. Right Offer /Promo – based on estimated response rates  & ROI

This whole process is generally described as Customer Driven Marketing. At its essence is using available data about customer’s transactions, demographics & other characteristics, to segment target and personalize the organization’s marketing efforts.

Customer loyalty systems – a goldmine of customer information

The data, to be used for customer driven marketing, is usually taken from the retailers Loyalty Management System. Most retailers today have adopted such a system, which is extensively used for reward point accumulation & redemption. Shopper’s had led the way with their First Citizen initiative and now most large departmental stores have a well established customer loyalty program, notable being The Landmark Group’s The Inner Circle and Trent’s Clubwest programs.

The initial intention of loyalty programs was to act as an additional incentive to the customer, to increase her purchase levels. Over time as organizations kept accumulating data on purchase patterns at customer level over time, they realised the potential of the underlying data to unlock value. The purchase patterns inherent in the data would reveal the true preferences of a customer, which has enormous advantages over data collected through a survey.

  1. Customers may hide their true preferences in a survey
  2. A survey can be done only on a very small section of the customer population
  3. The survey needs to be pre-designed and hence the retailer needs to have some pre conceived hypotheses, that they validate/invalidate
  4. The cost of fielding/collating results from a survey is high

Transaction data suffers from none of the above drawbacks, plus since the system is already in place for operational reasons, leveraging it for data analysis is a marginal increase in cost.

  1. Retailer can learn demographic based preferences of its customer base
  2. A retailer can predict the next best items to stock based on previous purchase behaviour
  3. Ability to upsell and cross sell at POS becomes lot more effective
  4. Ability to target prospect base with a focused approach rather than one size fits all

Customer Lifecycle Management typically follows a 4 step closed loop approach, as shown below, and the basis for it all is Customer Segmentation, which acts as the foundation for the entire structure

d2 TESCO, was the worldwide trendsetter in utilizing customer loyalty data to increasing ROI of marketing efforts. Starting in the mid 90’s, with help from the retail consultants Dunnhumby, they have used their Clubcard data to refine a range of marketing activity

  1. Personalising quarterly point redemption offers, based on purchase preferences of the individual customer
  2. Deciding on product placement within stores, based on complementary products
  3. Segmentation of customer base, based on loyalty (RFM) and purchase patterns
  4. Customising assortment in stores based on customer segment make up in the catchment areas etc

Indian retailers have also woken up to the value of the data at their disposal and are looking for ways and means to monetize it. The subsequent pages shall talk about a case of using customer loyalty data to refine direct marketing strategy.

Business brief

Our client, a leading apparel & lifestyle good retailer, which is part of a large Indian conglomerate. The retail business has been in operation for over 5 years, and the client has over 30 stores all across India.

The client introduced a customer loyalty scheme in the mid 2000’s and at the time of consideration, had ~ 350,000 loyalty club members. There were 2 major streams of direct marketing that were undertaken directed exclusively at the loyalty club members.

  • A bi-annual mailing of a product brochure, with heavy emphasis on seasonal apparel
  • Multiple single focus marketing campaigns carried out via multiple channels, throughout the year

The client was adopting a one size fits all strategy, for all direct marketing communication, which they recognized was not optimal. To help them customize their marketing communication, they wanted to use the power of data mining, and approached IRIS TEG.

Broad approach

Looking at the issue at hand, a four step process was suggested, as depicted in the figure below


Owing to space constraints & confidentiality issues, the rest of the article shall primarily focus on Step 1 i.e. customer segmentation and briefly touch upon steps 2-4.

Customer Segmentation Primer

Segmentation, refers to the practice of identifying homogeneous groups which are distinctively different from each other while they share similarities within. In statistical terms we call it as groups have least distance within the group and maximum distance across groups. . Once the distinct customer segments are identified, you then have the ability to market to those groups with the message/offer/tone that best resonates with the niche group.

Various analytical techniques are used for customer segmentation, with the most popular being Cluster Analysis. This technique identifies naturally occurring segments within your customer population and identifies the key variables that are driving these segments.

Advanced techniques like Latent Class Analysis, Decision Trees etc are also used in situations where a Cluster Analysis algorithm does not yield the required results

Basis for segmentation

Customers can be segmented on a variety of attributes, demographic, value based, attitudinal etc. The exact basis for segmentation would depend on the ultimate business objective. The table below illustrates how the business objective dictates the segmentation scheme


In the case in point, our prime objective was to refine the marketing strategy and move it to a stage of intensive customization. Hence, our recommendation to use behavioural i.e customer purchase patterns as the bases for the segmentation

Data – the fuel of the analytical engine!

The primary source of the data was the customer loyalty system, of the client, which gave us data around three major streams

Customer Transaction details

  • Basket level
  • Item level within a basket
  • Price of items bought
  • Amount of discount availed on each purchase
  • Mode of settlement

Customer Personal Details

  • Demographics – Age, Occupation, Gender, Marital Status etc
  • Tenure with Loyalty Program

Item Master Details

  • Item code & description
  • Item hierarchy across levels

Store Master Details

  • Store Size, Age, Competitive Intensity in catchment area

Once the data is received, we kick off the modelling process, which has 6 distinct phases as shown below.


The first two stages, having been covered above we shall concentrate on the process data preparation onwards

Data Preparation

There are three major activities that were carried out in this stage prior to creating the model proper

  1. The data was transformed from its ‘raw’ state to the customer level, by aggregating key variables
  2. Some derived variables as per the requirement of the segmentation scheme were created, from the base variables
  3. The customer base was split into a modelling sample to create the model & a holdout sample to validate our results

At the customer level, the segmentation would be run on key transaction variables, like the ones mentioned below

  1. Recency, Frequency & Monetary value variables (RFM)
  2. No. of visits per year
  3. No. of items purchased per visit
  4. Amount spent per category per year
  5. Discount chasing behaviour (proportion of spend, on discounted items)
  6. Etc

Customer profile variables were also created from their basic information, e.g. Lifestyle preferences from their purchase patterns across formal, casual, Indian and western wear and Life-stage variables using age, marital status & no. of children

Analytical Model

Post creation of the basic data, a Non Hierarchical Cluster Algorithm, was used to determine the segment membership. The modelling process looked at various cluster combinations and decided on a 5 cluster solution as the ideal one. The decision on the exact no. of segments was based on

Screen Shot 2013-05-10 at 12.10.02 PM

Segmentation Results

The clusters obtained from the modelling process were profiled on the customer demographics to create a holistic description, based on which they were called

Loyal Families

  1. 16% of population
  2. Most Valuable & Loyal Segment
  3. Purchase for Rs. 15,000 annually
  4. Make more than 7 trips per annum
  5. Contribute 37% of annual sales
  6. Buy Premium priced work wear & Fashionable products
  7. No promo hunting behaviour
  8. Married with children, buy across
  9. Men’s, Women’s & Kids departments

Married Loyal Value Seekers

  1.  18% of population
  2. Loyal segment seeking value for money (day to day) products
  3. Make an average of 7 trips per year
  4. Buy more than Rs. 10,000 in a year
  5. Contribute to 25% of annual sales
  6. Do not buy stylish, adventurous & party wear
  7. Very little purchases during sales
  8. Married without children

Married Occasional Shoppers

  1.  35% of population
  2. Largest segment in terms of size, but low in value contribution
  3. Make one trip in 6 months
  4.  Buy about Rs. 3,000 in a year
  5.  Account for 18% of annual sales
  6. Make just 2 trips a year, but are not promo hunters
  7. Married without children
  8. Buy VFM products and avoid Casual & Fashionable products

Single Female Promo Hunter

  1.  19 % of population
  2. Visits stores mainly during major discount sales
  3. Make one trip in 6 months
  4. Buy about Rs. 3,000 in a year
  5. Account for just 8% of annual sales
  6. Least valuable segment in terms of contribution to sales
  7. Seek Fashionable & Casual products.
  8. Do not buy VFM & Work wear

Single Male Value Seeker

  1.  12% of Population
  2. Smallest segment in terms of size
  3. Make about 3 trips a year
  4. Buy about Rs. 6,000 in a year
  5. Contribute to 12% of annual sales
  6. About half their purchases are during sales
  7. Buy VFM & Travel products, Avoids Fashionable & Work wear

Based on our study of the segmentation characteristics a detailed marketing strategy was worked out for each of the segments

Fine-tuning marketing strategy based on segmentation results

Screen Shot 2013-05-10 at 12.03.47 PM

Once the broad marketing strategy was decided, the next phase was to set up a series of test & learn experiments, in the form of pilot marketing campaigns, where the strategy was fine tuned in terms of ideal creative, channel of communication etc

The bi-annual marketing brochures were also customised for the segments. The customization was around

  1. Look & Feel – The valuable segments got a more detailed and well designed brochure.
  2. Product focus – Each segment had a target category, where they had already shown a past predilection to purchase and a secondary category where we wanted them to start purchasing

Key take-aways & learnings

  • Life – stage of a customer is the single greatest determinant of purchase patterns
    • Both how much the customer buys and what items she buys.
  • Married couples with children are the most valuable customers
    • Engaged across Male, Female & Kids departments
  • Whereas single customers, both male & female, are primarily deal seekers
    • Mostly purchase during major discount sales
  • About 2/3rds of loyalty members are occasional shoppers, who visit the store only 2 -3 times a year
  • The top 10% members by value, generate almost 40% of the annual sales
  • There are more than 55,000 loyalty customers who are listed in the records as Male, but their purchases are overwhelmingly of female products (> 90% of all items bought)
    • This indicates a situation where the card carrying member is a passive customer and his spouse make most of the purchase decisions

More Great AIM Stories

Dipayan Chakraborty
Dipayan leads the global delivery operations for TEG, servicing clients in US, Latin America, and India. Dipayan has over nine years of analytics experience across various domains. He has operated in analytics KPO captive units for a major MNC Financial institution as well as been part of the core team of the first successful offshore based analytics consulting firm out of India. Dipayan holds a MBA from the Indian Institute of Management, Ahmedabad and a Bachelor’s degree in Economics from St.Xavier’s College, Calcutta.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

Conference, in-person (Bangalore)
Cypher 2023
20-22nd Sep, 2023

3 Ways to Join our Community

Whatsapp group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our newsletter

Get the latest updates from AIM