In 2017, Meta open-sourced Prophet, a tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth. It’s based on an additive model where nonlinear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. Prophet works best with time series that have strong seasonal effects and several seasons of historical data and is robust to missing data and shifts in the trend, and handles outliers well.
Prophet’s star rose quickly, and has been downloaded 32,247,931 times at the time of writing. However, lately the tool’s popularity has dipped. Dr Valery Manokhin, MBA, CQF, pointed out that recent papers on time series have not used Facebook Prophet as baselines as it does not perform well across any general forecasting task.
“Every model should be evaluated against simple benchmarks to determine if a model adds business value. A simple benchmark could be using current sales value to predict value for tomorrow. Or it could be a value from last year. Forecasting is a game of relatives, not absolutes, and hence to determine if the Prophet (or any other model) adds value, one should establish a proper metric and naive forecast as a benchmark first,” he said.
Criticism
Prophet was developed to make it easier for experts and non-experts to deliver high-quality forecasts. According to a study published last year, the Prophet performed poorly compared to AutoRegressive Integrated Moving Average (ARIMA). Prophet’s forecasts were subpar even when it used more data than other models. The study concluded that the Prophet failed to deliver the goods in multiple use cases. That said, Prophet could be useful in certain contexts where quick and preliminary forecasts are needed.
Prophet has been facing criticism for its underlying assumptions being weak or too simplistic. Further, the tool was designed to deal with holidays known in advance and tends to miss observations and outliers. Also, Prophet does not directly consider the recent data points which affects the performance in cases where prior assumptions do not fit.
Is Prophet still relevant?
In a recent paper ‘ PROFHIT: Probabilistic Robust Forecasting for Hierarchical Time-series’, the researchers have not used Prophet as a baseline model.
A baseline model acts as a reference in a machine learning project and helps contextualise the results of trained models.
“The reason that Prophet does not work well is that it is a variant of rigid Generalised Linear Model. A curve-fitter that is not responsive to local patterns because Prophet doesn’t include auto regressive terms. There are papers and independent studies that demonstrated that even in situations for which it was expressly designed for – namely time series patterns with trends and volatility – Prophet is often outperformed by other model classes such as ARIMA and many more.
“By now, it has been established by many academic studies and papers that the Prophet is not a very good model. In fact, after data scientists like myself and others commented on this issue, Prophet development team removed some of the original performance claims from their website,” Manokhin said.
Despite all the shortcomings, Prophet does have one advantage. Since it can automatically find seasonal trends and offers easy-to-understand parameters, Prophet does not require much prior knowledge of forecasting time series data.
