Now Reading
Guide to Pytorch Time-Series Forecasting

Guide to Pytorch Time-Series Forecasting


Pytorch is a scientific library operated by Facebook, It was first launched in 2016, and it is a python package that uses the power of GPU’s(graphic processing unit), It is one of the most popular deep learning frameworks used by machine learning and data scientists on a daily basis. It is a very flexible and fast deep learning framework. It provides Tensors supports for GPU acceleration and construction of neural nets in tape-based auto-grading systems.

This library has the potential to alter deep learning and AI performance. This library is completely python-based, so it’s easy to create a neural network.

Now PyTorch is capable of handling a full pipeline in deep learning and AI projects, but some of the things can be pretty messy like using PyTorch for Forecasting, so a third party is introduced by Jan Beitner Pytorch Forecasting”

Forecasting time series is important, and with the help of this third-party framework made on top of PyTorch, we can do time series forecasting just like Tensorflow very easily. Forecasting is in the industry for a very long time, and it is used by many businesses for making an extra profit by just predicting the future outcome and keeping them on the safe side. Now with the help of deep learning, traditional forecasting methods have surpassed the last three years.



Image for post

Pytorch Forecasting is a framework made on top of PyTorch Light used to ease time series forecasting with the help of neural networks for real-world use-cases. It is having state of the art time series forecasting architectures that can be easily trained with input data points. This library has many benefits as they composed all the different requirements into specific classes like:

  • TimeSeriesDataSet is used to prepare the dataset for training with PyTorch, this class takes care of the variable transformation, random sampling, and missing value filling, etc. 
  • BaseModel class is used to provide data visualization such as showing predicting values vs real values.

Some of the other features we get with Pytorch forecasting are:

  • Faster model training as it is built on PyTorch lightning which allows you to train the model on CPU as well as multiple GPU.
  • Temporal fusion Transformer: An architecture developed by Oxford University and Google for Interpretable Multi-horizon Time Series forecasting that beat Amazon’s DeepAR with 39-69% in benchmarks.
  • N-BEATS model
  • DeepAR model: Most popular baseline model for time-series forecasting.
  • Ranger optimizer for faster model training.
  • Hyperparameter tuning using optuna

Installation

First, install Pytorch as the forecasting library is inherited from Pytorch, install PyTorch with this command:

pip install torch -f https://download.pytorch.org/whl/torch_stable.html

Install PyTorch forecasting using pip:

pip install pytorch-forecasting

Or you can install it with conda forge command:

conda install -c conda-forge pytorch-forecasting

Getting Started with Pytorch Forecasting

We are going to use the Stallion Dataset from Kaggle, this dataset contains the following files:

  • pricesalespromotion.csv: Holds the price, sales & promotion in dollars.
  • historicalvolume.csv: contains sales data.
  •  weather.csv: the average maximum temperature at Agency monthly/ 
  • Industrysodasales.csv: Holds industry level soda sales 
  • eventcalendar.csv: Holds event details (sports, carnivals, etc.)
  • industry_volume.csv: industry actual beer volume
  • demographics.csv: demographic details

This data set contains the sales of various beverages. Our goal is to predict six months of sold volume by stock-keeping units(SKU).

See Also
Tech leaders on COVID-19

This dataset is already included in pytorch forecasting library so import it using below commands:

from pytorch_forecasting.data.examples import get_stallion_data
data = get_stallion_data()  # load data into pandas dataframe

Data Cleaning

Below commands are compulsory data cleaning code for things like one-hot encoding and modifying columns.

 data["date"].dt.monthdata["time_idx"] -= data["time_idx"].min()
 data["time_idx"] = data["date"].dt.year * 12 + data["date"].dt.monthdata["time_idx"] 
 # adding features
 # categories must be string
 data["month"] = data.date.dt.month.astype(str).astype("category")
 data["log_volume"] = np.log(data.volume + 1e-8)
 data["avg_volume_by_sku"] = (
     data
     .groupby(["time_idx", "sku"], observed=True)
     .volume.transform("mean")
 )
 data["avg_volume_by_agency"] = (
     data
     .groupby(["time_idx", "agency"], observed=True)
     .volume.transform("mean")
 )
 # we will encode special days into one variable and apply Reverse one-hot encoding
 special_days = [
     "easter_day", "good_friday", "new_year", "christmas",
     "labor_day", "independence_day", "revolution_day_memorial",
     "regional_games", "fifa_u_17_world_cup", "football_gold_cup",
     "beer_capital", "music_fest"
 ]
 data[special_days] = (
     data[special_days]
     .apply(lambda x: x.map({0: "-", 1: x.name}))
     .astype("category")
 )
 # sample data
 data.sample(10, random_state=521) 
Image for post

Let’s convert the dataset into Pytorch forecasting format.

 from pytorch_forecasting.data import (
     TimeSeriesDataSet,
     GroupNormalizer
 )
 max_prediction_length = 6  # forecast of 6 months
 max_encoder_length = 24  # using history of 24 months
 training_cutoff = data["time_idx"].max() - max_prediction_length
 training = TimeSeriesDataSet(
     data[lambda x: x.time_idx <= training_cutoff],
     time_idx="time_idx",
     target="volume",
     group_ids=["agency", "sku"],
     min_encoder_length=0,  # allowing predictions without history
 max_encoder_length=max_encoder_length,
     min_prediction_length=1,
     max_prediction_length=max_prediction_length,
     static_categoricals=["agency", "sku"],
     static_reals=[
         "avg_population_2017",
         "avg_yearly_household_income_2017"
     ],
 time_varying_known_categoricals=["special_days", "month"],
     # group of categorical variables can be treated as one variable
     variable_groups={"special_days": special_days},
     time_varying_known_reals=[
         "time_idx",
         "price_regular",
         "discount_in_percent"
     ],
     time_varying_unknown_categoricals=[],
     time_varying_unknown_reals=[
         "volume",
         "log_volume",
         "industry_volume",
         "soda_volume",
         "avg_max_temp",
         "avg_volume_by_agency",
         "avg_volume_by_sku",
     ],
     target_normalizer=GroupNormalizer(
         groups=["agency", "sku"], coerce_positive=1.0
     ),  # use softplus with beta=1.0 and normalize by group
     add_relative_time_idx=True,  # add as feature
     add_target_scales=True,  # add as feature
     add_encoder_length=True,  # add as feature
 )
 # creating validation set (predict=True) which means to predict the
 # last max_prediction_length points in time for each series
 validation = TimeSeriesDataSet.from_dataset(
     training, data, predict=True, stop_randomization=True
 )
 # create dataloaders for model
 batch_size = 128
 train_dataloader = training.to_dataloader(
     train=True, batch_size=batch_size, num_workers=0
 )
 val_dataloader = validation.to_dataloader(
     train=False, batch_size=batch_size * 10, num_workers=0
 ) 

Train the Temporal Fusion Transformer

 import pytorch_lightning as pl
 from pytorch_lightning.callbacks import (
     EarlyStopping,
     LearningRateLogger
 )
 from pytorch_lightning.loggers import TensorBoardLogger
 from pytorch_forecasting.metrics import QuantileLoss
 from pytorch_forecasting.models import TemporalFusionTransformer
 # training will stop, when validation loss does not improve 
 early_stop_callback = EarlyStopping(
     monitor="val_loss",
     min_delta=1e-4,
     patience=10,
     verbose=False,
     mode="min"
 )
 lr_logger = LearningRateLogger() 
 logger = TensorBoardLogger("lightning_logs") 
 # creating trainer
 trainer = pl.Trainer(
     max_epochs=30,
     gpus=0,  
     gradient_clip_val=0.1,
     early_stop_callback=early_stop_callback,
     limit_train_batches=30,  # running validation for every 30 batches
     callbacks=[lr_logger],
     logger=logger,
 )
 # initialise the model
 tft = TemporalFusionTransformer.from_dataset(
     training,
     learning_rate=0.03,
     hidden_size=16,  # biggest influence network size
     attention_head_size=1,
     dropout=0.2,
     hidden_continuous_size=8,
     output_size=7,  # by default QuantileLoss has 7 quantiles 
     loss=QuantileLoss(),
     log_interval=10,  # log example for every 10 batches
     reduce_on_plateau_patience=4,  # reduce learning automatically
 )
 tft.size() # 29.6k parameters in model
 # fit network
 trainer.fit(
     tft,
     train_dataloader=train_dataloader,
     val_dataloaders=val_dataloader
 ) 
Image for post
Source- https://towardsdatascience.com/introducing-pytorch-forecasting-64de99b9ef46/

During training, you can also view the tensorboard for prediction visualization using tensorboard –logdir=lightning_logs.

Image for post

Code of this tutorial is available here.

Conclusion

As you have seen how easy it is to train and analyze the time series data using the Pytorch forecasting framework, you can also evaluate the trained model using matrices. MAE, another feature of this framework is an interpretation of trained models. You can also see the variable importances by the design of neural networks, read more about PyTorch forecasting here 

Pytorch forecasting is open sourced at GitHub here if you want to contribute or submit an issue the community support for this library is very overwhelming. And for learning more about go there, models go to these links:


Subscribe to our Newsletter

Get the latest updates and relevant offers by sharing your email.
What's Your Reaction?
Excited
1
Happy
2
In Love
1
Not Sure
0
Silly
2

Copyright Analytics India Magazine Pvt Ltd

Scroll To Top