Article complete

Critical Success Factors for Analytical Models: Some Recent Research Insights

Critical Success Factors for Analytical Models: Some Recent Research Insights

Key takeaways

  • Successful analytical models must address specific business problems, ensuring relevance to the intended application.
  • Statistical performance and validity are crucial, as models need to demonstrate significant and reliable predictive capabilities.
  • Interpretability is essential, as decision-makers must easily understand the model's outcomes and implications.

Takeaways drafted with AI from the article text.

Big data & Analytics: terms that frequently pop up in newspapers, magazines, airports or even during pub chats to pimp a conversation. These days, everybody talks about it, but only few are actually doing it successfully! One of the reasons is that firms often lack a clear insight into the critical success factors for building actionable analytical models. Hence, in this column, we provide some recent research insights based upon partnerships we initiated with firms world-wide, and which are further elaborated on in the new E-learning course Advanced Analytics in a Big Data World. In order to be successful, an analytical model needs to satisfy various requirements. A first key requirement is business relevance. The analytical model should solve the business problem that it was developed for! It makes no sense to have a high-performing analytical model that was sidetracked from the original business problem. In other words, if the business problem is detecting insurance fraud, then the analytical model must be sure to detect insurance fraud. Obviously, this requires a thorough business knowledge and understanding of the problem to be addressed before any analysis can start. Some example kick-off questions are: how do we define, measure and manage fraud? Another important success factor is statistical performance and validity. In other words, the analytical model should make sense statistically. It should be significant and provide good predictive or descriptive performance. Depending on the type of analytics, various performance metrics can be used. In customer segmentation, statistical evaluation measures will contrast intra-cluster similarity with inter-cluster dissimilarity. Analytical churn prediction models will be evaluated in terms of their ability to assign high churn scores to the most likely churners. Interpretability refers to the fact that the analytical model should be comprehensible or understandable to the decision maker (e.g. marketer, fraud analyst, credit expert). Justifiability indicates that the model is in accordance with the expectations and business knowledge of the expert. Both interpretability and justifiability are subjective and depend on the knowledge and experience of the decision maker. Both often need to be balanced against statistical performance, which implies that complex, non-interpretable models (e.g. neural networks, random forests, …) are often better performing in a statistical sense. In settings like credit risk modeling, interpretability and justifiability are very important because of the societal impact of these models. However, in settings like fraud detection and marketing response modeling, they are typically less of an issue. Operational efficiency relates to the effort that is needed to evaluate, monitor, backtest or rebuild the model. From this perspective, it is quite obvious that a neural network or random forest is less efficient that e.g. a plain vanilla regression model or decision tree. In settings like credit card fraud detection, operational efficiency is very important because a decision should be made within a few seconds after the credit card transaction was initiated. Economical cost refers to the cost that is needed to gather the model inputs, run the model and process its outcome(s). Also the cost of external data and/or models should be taken into account here. This will enable you to calculate the economic return on the analytical model, which is typically not a straightforward exercise. Finally, regulatory compliance is becoming more and more important. This refers to the extent to which the model is compliant with regulation and legislation. In a credit risk modeling setting, it is important that the models are compliant with the Basel II and III regulations. In an analytical insurance setting, the Solvency II accord must be respected. To conclude, in this blog article we briefly zoomed into the critical success factors for building analytical models. As already mentioned, the importance of each of them depends on the application field in which you are working. For more information, we are happy to refer you to the new E-learning course Advanced Analytics in a Big Data World. Self-Paced E-learning course: Advanced Analytics in a Big Data World See: https://support.sas.com/edu/schedules.html?ctry=us&id=2169 The E-learning course starts by refreshing the basic concepts of the analytics process model: data preprocessing, analytics and post processing. We then discuss decision trees and ensemble methods (bagging, boosting, random forests), neural networks, support vector machines (SVMs), Bayesian networks, survival analysis, social networks, monitoring and backtesting analytical models. Throughout the course, we extensively refer to our industry and research experience. Various business examples (e.g. credit scoring, churn prediction, fraud detection, customer segmentation, etc.) and small case studies are also included for further clarification. The E-learning course consists of more than 20 hours of movies, each 5 minutes on average. Quizzes are included to facilitate the understanding of the material. Upon registration, you will get an access code which gives you unlimited access to all course material (movies, quizzes, scripts, ...) during 1 year. The E-learning course focusses on the concepts and modeling methodologies and not on the SAS software. To access the course material, you only need a laptop, iPad, iPhone with a web browser. No SAS software is needed. See https://support.sas.com/edu/schedules.html?ctry=us&id=2169 for more details.

Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

churn prediction model · 5 stories
  1. You are here: Critical Success Factors for Analytical Models: Some Recent Research Insights
  2. How Hollywood Is Using AI To Churn Out The Perfect Formula For A Cinematic Blockbuster
  3. How Max Life Insurance Deployed AI Models To Optimise Processes
  4. Inside GrokNet, Facebook’s Product Recognition System Revolutionising Online Shopping
  5. Salesforce Open-Sources WarpDrive Deep RL Framework
About the author

Contributor

Professor Bart Baesens is a professor at KU Leuven (Belgium), and a lecturer at the University of Southampton (United Kingdom). He has done extensive research on big data & analytics, customer relationship management, web analytics, fraud detection, and credit risk management. His findings have been published in well-known international journals and presented at international top conferences. He is also author of the books Analytics in a Big Data World (goo.gl/k3kBrB), and Fraud Analytics using Descriptive, Predictive and Social Network Techniques (http://goo.gl/nlCjUr). His research is summarized at www.dataminingapps.com. He is also teaching the E-learning course, Advanced Analytics in a Big Data World, see http://goo.gl/WibNPF. He also regularly tutors, advises and provides consulting support to international firms with respect to their analytics and credit risk management strategy.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.