Council Post: How to build robust forecasting models amid chaos

The pandemic was a reality check for companies across the world. No matter how prepared you think you are, black swan events like Covid-19 can throw your company into disarray. However, sitting on their hands is not an option for modern businesses. To bring antifragility to their preparedness, companies leverage AI and ML to develop robust forecast models. The quality of such models rely on the data fed into them. But what if such events put a strain on the data collection pipeline to start with? 

We have sounded out the thought leaders in the industry to figure out ways to tackle such situations.

Take a collaborative approach

The collaboration across industry, government, agencies, and academia can help significantly increase the breadth of the data for the application of X-analytics techniques that can amalgamate structured and unstructured streams such as text, image, audio, and video to address the contextual need more effectively. This will also be enabled by traction with the creation and utilisation of synthetic data to overcome the privacy, and confidentiality aspects of the data owned by enterprises, agencies, governments, and the like.

Satyamoy Chatterjee, Executive Vice President at Analyttica Datalab

Break it down

A good way that I have seen work is to break it down by what kind of refinements are needed on the input (data), process (the models and algorithms), and output (adjustments to final prediction) and take it, step by step.

Ruble Joseph, Lead Strategist (VP) - Global Data Science and Analytics Practice at eClerx

Robust techniques

To keep forecasts meaningful, we adopted a couple of strategies – we added new data points (incorporated manual forecasts by demand planners, incorporated supply constraints based on qualitative inputs), changed our model selection process based on recent sales data vs looking at year on year changes, changed the operational process for forecasting, and incorporated learning-based model techniques like deep learning in addition to casual models like Bayesian structural time series, ARIMAX, etc.  

Ajoy Singh, Chief Operating Officer at Fractal

Address the drift 

Identifying the type of drift – i.e., concept drift, data/covariate shift, and the appropriate mitigation strategy to combat the effects of the drift on the quality of the predictive model, holds the key to building models that can stand the test of time. This can be achieved by comparing pre-drift forecasting to post-drift forecasting, detecting data distribution changes or I/O relationships, developing novel difference transform approaches, whereas adding domain-specific and pandemic related external regressors to pre and post-event training data can also help improve the quality of models.

 Ashish Kumar, VP - Data Science at Salesken

Take the long view

For all forecasting/prediction, we need to consider this time period; one can keep a separate indicator (or something similar) to flag it out. Moreover, macroeconomic factors need to be considered to make predictions with relatively higher accuracy. To complicate things, customer behaviour towards certain product categories has drastically changed with more reliance on digital services. Overall, while these factors mentioned above should be considered to adjust the forecasting/ predictive models, from an algorithm standpoint, we should rely more on machine learning models that give relatively higher weightage to certain data points (Sequence to Sequence, Encoder-Decoder with LSTM, etc.)

Anirban Nandi, Head of Analytics (Vice President) at Rakuten India

Be versatile

When generating predictive models depends primarily on the specific use case and the industry under consideration; some generic approaches include:

• While creating the models, we can consider data over a longer period of time; or validate the existing model across different time periods and scenarios. If model performance is good, the model is good to go; else, we may need to recalibrate the model by removing outliers or providing weights across different time periods. 
• We may consider macroeconomic parameters or alternate data, which may give more real-time information for assigning weights to data about time periods more specifically affected by the pandemic. 
• We can leverage alternate data such as auto sales to help predict the spending propensity across different customer segments. In specific instances, e.g., for industries like travel, where the impact of lockdown has been significant - we may create models removing the data corresponding to the specific time period. 

Swati Jain, Vice President Analytics at EXL

This article is a collation of quotes by members of the AIM Leaders Council. AIM Leaders Council is an invitation-only forum of senior executives in the Data Science and Analytics industry. To check if you are eligible for a membership, please fill out the form here.