A hands-on tutorial on Customer Lifetime Value (CLTV) prediction

Customer Lifetime Value (CLTV) measures how valuable a customer is to your company, not only within the scope of a single transaction but across the entire relationship. As part of a customer experience program, it is important to track customer lifetime value (CLTV) in order to achieve great retention of customers. In general, CLTV represents the profit associated with a particular customer relationship. In this article, we would be learning how to calculate the CLTV in python and predict the probabilities of a customer. The major points to be covered in this article are listed below.

Table of contents

1. What is CLTV?
2. Dataset description
3. What is RFM? 
4. Predict the CLTV
5. Calculate the probability of customer retention
6. Effect of time on prediction of transactions

Let’s first understand what CLTV is.

What is Customer Lifetime Value (CLTV)?

Customer Lifetime Value (CLTV) is all the potential profits a particular customer can bring to the organization. For instance, you have a clothing store and a new customer bought a sneaker from your store. In future, they may buy another pair of sneakers or a t-shirt or a shirt. All these potential purchases at CLTV. In other words, if you estimate one customer's CLTV at ₹50k, you would not spend more than that to maintain the relationship. The cost is simply not worth it. Ultimately CLTV should guide how much you should invest to maintain the relationship.

Are you looking for for a complete repository of Python libraries used in data science, check out here.

How to calculate?

CLTV of a customer can be calculated with the help of the below-mentioned formulas,

Average Purchase Value =  Total revenueNumber of orders

Average Purchase Frequency Rate = Number of purchasesNumber of customers 

Customer value = Average Purchase Value * Average Purchase Frequency Rate

Customer Lifetime Value (CLTV) = Customer Value *  Average Customer lifespan

Description of Dataset

Data used in the article is related to online retail customer segmentation based on their purchases. This dataset has been taken from a kaggle repository which is mentioned in references. It contains a total of 8 columns which contain different information about the customer purchases. The description of columns are mentioned below:

• Invoice no: A unique 6 digit number assigned to the invoice of the purchase order.
• Stock code: 5 digit code assigned to the distinct products.
• Description: Brief description about the products.
• Quantity: Number of products purchased by the customer.
• Invoice date: Date of the purchase.
• Unit price: Price of the product per unit purchased.
• Customer id: Unique identification number of the customer.
• Country: Location of the customer.

Sample data

There are all the required columns but there is no column related to the total sales per order. We can calculate the sales per order by multiplying the quantity and unit price.

What is RFM? 

In this section, RFM methodology will be used on the data to assign a value to the customers based on their purchases which would be further used to predict the CLTV.

R stands for  Recency of purchase, how recently a customer has made a purchase. This could be found out by checking the recent purchase date of the last purchase of the customer

F stands for Frequency of purchase, how often a customer makes a purchase. This could be calculated by dividing the number of purchases by the number of days.

M stands for Monetary value of purchase, how much money a customer spends on purchases. Calculate the total revenue generated from the customer.

Implementing RFM on data

Import libraries

import numpy as np
import pandas as pd
import lifetimes
from lifetimes.utils import calibration_and_holdout_data

Read the dataset

df=pd.read_csv('/content/customer_segmentation.csv',encoding='cp1252')
df.head()

Feature engineering

df['InvoiceDate']=pd.to_datetime(df['InvoiceDate'])
df['orderdate']=df['InvoiceDate'].map(lambda date: date.date())

Converting ‘InvoiceDate’ column to the DateTime datatype for further calculations and ease and storing only the dates in the new column ‘orderdate’.

Apply RFM

calibration_period_ends = '2011-09-05'

calib_holdout = calibration_and_holdout_data(df, 
                          customer_id_col='CustomerID', 
                          datetime_col = 'orderdate', 
                          freq = 'D', #days
                          calibration_period_end=calibration_period_ends,
                          observation_period_end='2011-12-9' )
calib_holdout.head()

lifetimes.utils.calibration_and_holdout_data () summarize each customer based on calibration and holdout period. Calibration is the training and holdout is the testing. Parameters provided to the functions are:

• transactions: The data needs to be given. 
• customer_id_col: Customer id column must be described. 
• datetime_col: Order date column should be provided. 
• freq: Days
• calibration_period_end: Mention the date on which the training should be performed.
• observation_period_end: Mention the date on which the testing should be performed.

In the above sample of customers, their RFM values have been calculated for both calibration and holdout.

Predict the CLTV

As in the above section RFM values have been calculated for all the customers, now let us proceed to the prediction part. For predicting the CLTV of future customers, we need to train a machine learning model.

Training - MBG/NBD model

This model has some assumptions which are listed below:

1. An active customer's transaction rate (λ) follows a Poisson process.
2. Heterogeneity across transition rate (λ) is governed by a Gamma distribution with shape parameters (Г) and scale parameter (𝛼)
3. Customers become inactive at time zero and after each purchase with a constant probability (p)
4. Probability (p) is heterogeneous across customers based on the Gamma distribution with parameters a and b
5. The drop out probability (p) and transaction rate (λ) are not dependent on each other and vary independently across customers. 

Importing library:

from lifetimes import ModifiedBetaGeoFitter

Building the model and training data based on RFM:

mbg_nbd = ModifiedBetaGeoFitter(penalizer_coef=0.01)
mbg_nbd.fit(calib_holdout['frequency_cal'], 
        calib_holdout['recency_cal'], 
        calib_holdout['T_cal'],
       verbose=True)

Modified Beta Geo fitter (MBG) is given a parameter penalizer_coef which is applied on the L2 norm or Euclidean norm. 

Euclidean norm is the shortest distance to go from one point to another.  

The model was trained on 3388 different customer data.

Estimating CLTV on the second model  Gamma-Gamma

Gamma-Gamma assumes that there is no relationship between the monetary value and the purchase frequency. For that, we need to check it by using the Pearsons’ correlation test.

raw_summary=lifetimes.utils.summary_data_from_transaction_data(df,'CustomerID','orderdate','Sales')
raw_summary.head()
raw_summary=raw_summary[raw_summary['monetary_value']>0]
gg=lifetimes.GammaGammaFitter(penalizer_coef=0.01)
gg.fit(raw_summary['frequency'], 
       raw_summary['monetary_value'],time=1)
gg.summary

No days in the future are given by the time parameter, so we are taking for 1 month or 30 days. It depends upon you to initial the value according to your analysis. Trained the model on the raw summary data formed by the lifetimes.utils.summary_data_from_transaction_data() function.

We can see the output summary of the performance of the gamma-gamma fitter model on the data. Using the gamma-gamma fitter model we will now predict the CLTV for customers.

raw_summary['predicted_CLTV']=gg.customer_lifetime_value(mbg_nbd,
                              raw_summary['frequency'], 
                              raw_summary['monetary_value'],
                              raw_summary['recency'],
                              raw_summary['T'])
                              raw_summary.head() 

A new column has been added to the summary data frame with all the values predicted by the gamma-gamma fitter model.

Calculate the probability of customer retention

Let’s see the history of a particular customer and calculate the probability of retention of the customer in future.

Importing libraries

from lifetimes.plotting import plot_history_alive
from datetime import date
from lifetimes.utils import calculate_alive_path

Create a graph

cust = calib_holdout.iloc[3265]
cust_id = cust.name
total = 365*90
today = date.today()
ago = today.replace(year=today.year - 12)
now = today.replace(year=today.year + 1)

Describing the particulars of the customers and initializing the total number of time units, defining the future date and past date for plotting the history.

tran = df.loc[df['CustomerID'] == cust_id]
total = (today - tran.orderdate.min()).days
p_alive_today = pd.DataFrame(calculate_alive_path(mbg_nbd, tran, 'orderdate', t, freq='D'))[0].tail(1).values
p_alive_today = np.round(p_alive_today[0], 2)
print('The Probability of that customer is alive today is', p_alive_today)

Output: 

The probability of that  customer is alive today is [0.57]

Calculating the probability of the retention of the customer as a future buyer.

total = (now - tran.orderdate.min()).days
ax = plot_history_alive(mbg_nbd, total, tran, 'orderdate', start_date=ago) 
ax.vlines(x=today, ymin=0, ymax=1.05 )
ax.hlines(y=0.8, xmin=ago, xmax=now )
 
ax.set_xlim(ago, now) 
ax.set_ylim(0, 1.05)
 
plt.xticks(rotation=-90)
text(0.75, 0.1, p_alive_today, ha='center', va='center', transform=ax.transAxes)
 
sns.despine()

Plotting the above information obtained.

Effect of time on prediction of transactions

Import libraries

from lifetimes.utils import expected_cumulative_transactions

In this, only the customer id and the order date is required so we create a new data frame with only these columns.

df1=df[['CustomerID','orderdate']]

Predict the transactions

time_unit = (df1.orderdate.max() - df1.orderdate.min()).days
df2 = expected_cumulative_transactions(mbg_nbd, df1, 'orderdate', 'CustomerID', time_unit)
df2.head()

In the above sample, we can observe that there is variation in predictions. Let’s plot and have a more detailed view of the variation due to time.

cal = datetime.strptime('2011-12-05', '%Y-%m-%d')
 
from lifetimes.plotting import plot_cumulative_transactions
time_unit = (df1.orderdate.max() - df1.orderdate.min()).days
time_unit_cal = (cal.date() - df1.orderdate.min()).days
plot_cumulative_transactions(mbg_nbd, df1, 'orderdate', 'CustomerID', time_unit, time_unit_cal, freq='D')
sns.despine()

Conclusion

In this article, we got to learn about how segmented customer demographic data can be utilized to predict Customer Lifetime Value (CLTV), the customer retention probability and the effect of time on the transaction prediction.

References 

1. Link for above code
2. Lifetimes utils documentation
3. Dataset