using-markov-modelThe present business scenario is observing all the major brands competing in the social media space to influence their customers through targeted digital marketing. Customers today are making 70% of their buying decisions before they interact with the sales representatives. Many researches over the last decade have indicated that 63 percent of the social media users consider the consumer rating as a top influencer of their buying decisions.[1] Social networks influence nearly 50% of all IT decision makers[2]. Thus digital marketing as emerged as an undisputed critical marketing channel determining the business success at a fraction of the cost.

Digital marketing is poised to take off in leaps and bounds with an expected growth rate of 30%[3]. It has the potential to track a customer’s journey in real time through robust micro data that can be sourced from the multiple online platforms.

Channel attribution problem:

As digital marketing and digital content has become the critical determinant of business success, brand marketers are carefully contemplating the content that needs to be designed and posted for influencing the customer decisions in their favor. Inefficiency in content production results in an estimated $958 million excessive spend each year for mid-to-large B2B companies[4]. These inefficiencies arise from the inability to identify effective online formats or channels. A typical consumer comes in contact with advertisements in multiple formats and channels. Early encounters need to be especially effective to influence the customer positively failing which brands may run a risk of permanently losing the attention of the customer. It is in the interest of the marketers to identify and quantify the influence of the multiple formats and channels individually. For instance a consumer could be initialized down the conversion path by print ads but incorrect attribution could allot the channel zero contribution. This could result in lower advertisement investments to the channel resulting in detrimental impact on marketer profitability. The identification of contribution accurately can help the marketers optimally allocate resources across channels.

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Funnel approach using Markov process:

The funnel approach is a powerful tool to understand the strength of marketing activities at different states. It allows us to systematically identify and rectify the obstruction in the marketing process. The funnel approach is ideal as it is agnostic of industries. The challenge still persists in accurately predicting the customer digital behavior and estimating the customer buy probability of a product. The behavior of a consumer is a function of his expectation and prior experiences which makes attribution accounting complex. The absence of disaggregate individual level of data makes attribution accounting difficult while dealing with traditional marketing channels such as television print etc. However this is no longer the case as marketing moves online. Granular level data can enable the construction of rich models to capture consumer response to marketing channels as shown in the figure below


Download our Mobile App

Figure 1 Latent States of Consumer’s Decision Process

Individual consumer behavior can be modelled using the conversion approach. It captures the incremental effect that marketers have on the consumer’s deliberation process. This can be divided into four stages disengaged, active, engaged and converted, on the basis of his interaction with advertisements. In the disengaged state there is low to no interaction with the advertisers. He moves into active state on either exposure to multiple ads or on gathering more information on the product of interest. The state of engagement is one where he actively looks for product related information. This formulation is not linearly progressive, a consumer is allowed the flexibility to move between states and in any direction. Each state is mapped to certain pattern of observed and latent research behavior exhibited by the consumer. The possibility of conversion increase as he moves forward along the path. This approach takes into account the heterogeneity within and across Designated Market Areas (DMA). Heterogeneity within the DMA captures the underlying idiosyncrasy in consumer behavior. Heterogeneity across DMAs allow marketers to launch campaigns keeping in mind the social and cultural tastes of the customers. The assimilation of information from the funnel approach can help marketers target advertisements more efficiently and  optimally allocate resources. .

Solutions framework:

A study[5] conducted using monthly data to ascertain the optimal policy for maximizing the value generated by the customer over a year The procedure was iterated over all states for each month, table 1 displays the optimal policy at each state.

State t ϵ [1,3] t ϵ [4,6] t ϵ [7,12]
S1 Special offer Do nothing Do nothing
S2 Club membership Club membership Do nothing
S3 Do nothing Do nothing Do nothing
Table 1: Optimal policy at each state

From the Table 1 we observe that the optimal policy is to not send any offer to any customer for the t-time investment horizon t ϵ [7, 12]. Sending a club membership offer to customer in state S2 pays off only after at least 6 months, while sending special offer to customer in state S1 pays off only after at least 9 months.


Fig 2: Value of marketing in optimal policy versus static state

Fig 2. Estimates the difference in performance between the optimal policy and blanket policy. Value per state is used to measure the performance of the optimal approach. For time steps up to 7 the optimal policy returns higher average value per state post which an equalization of returns can be observed. Clearly displaying better performance by the optimal policy.

Marketing strategy formulation:

The Markov decision model can be used to help the firm manage its marketing strategy. The firm’s challenge is to find the optimal marketing policy. Marketing Strategy using Markov chain model for customers will ideally have 4 states. Each state has a probability that is calculated using the customers’ recency, frequency and monetary value. Useful extensions to this model would be to consider net contributions that vary with frequency and marketing expenditures that vary with both recency and frequency. These extensions are easy to implement because they involve changes to the reward vector only. The expected present value of the customer propensity to purchase is calculated for each state. This in turn tells us that effect of marketing policies on the customers’ propensity.

Initially the expected impact of an arbitrary policy is evaluated. The next step is to revisit the firm’s decision at each and every state and replace the current decision with a best decision based on the calculated values. This ends the first iteration in the policy improvement algorithm. To evaluate the policy Markov decision model is modified by changing the purchase probabilities to zero for any state outside the firm’s contact policy. In addition, the reward for transitioning is set to any of these states at zero. In effect, if the customer transitions to a state outside the firm’s marketing policy she or he will transition with probability 1 and with reward zero to state. Improvements in the expected value of customer’s propensity to purchase for this strategy is examined and repeated for different states.

This evaluation helps us predict the customer digital behavior with reasonable accuracyand judicious allocation of digital resources.





[5] Allocation of Marketing Resources to Optimize Customer Equity by Giuliano Rocco Tirenni


untitledDivya James Athoopallil: Analytics Consultant at Bridgei2i Analytics Solution

Graduate from Madras School of Economics, Chennai (2014) worked in analytics and policy space.

Mobile: +919940239762 || email:



untitledAditya Joseph James: Manager at HSBC Electronic Data Processing

Graduate from FMS, Delhi (2013), worked across product management and analytics space

Mobile: +919535838658 || email:

Leave a Reply