# How To Do Multivariate Time Series Forecasting Using LSTM

This is the 21st century, and it has been revolutionary for the development of machines so far and enabled us... This is the 21st century, and it has been revolutionary for the development of machines so far and enabled us to perform supposedly impossible tasks; predicting the future was one of them. But now, with the help of advanced computational power and a tremendous boost in the field of artificial intelligence, machine learning, the process of predicting the future has become quite simple and fast. Some of the major applications of this field are Image recognition, Speech recognition, Traffic prediction, Self-driving car, Virtual Personal assistance, and the list continues.

Time series forecasting is also an important area in machine learning. However, it is neglected due to its complexity, and this complexity is due to the time components like trend, seasonality, base level of series, Noise. Time series forecasting involves fitting models on historical data and using the fitment to predict the future data the same as the other ML technique. The only major difference between the simple prediction based model and forecasting model is that here the forecasting is completely unavailable and must be only estimated with the help of what has already happened.

#### THE BELAMY

##### Sign up for your weekly dose of what's up in emerging technology.

This article will discuss deep learning techniques used to address forecasting using multiple dependent variables and one target variable. This technique is taken from the Book called ‘Hands on Time series analysis using Python’.  The author used a Bidirectional LSTM based network with customized data preparation, and the result is supposed to follow the trend.

Let’s check the result practically by leveraging python.

## Code implementation Multivariate Time Series Forecasting Using LSTM

##### Import all dependencies:
``` import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import plotly.express as px # to plot the time series plot
from sklearn import metrics # for the evaluation
from sklearn.preprocessing import LabelEncoder,MinMaxScaler
import tensorflow as tf ```

Dataset is about the Metro interstate traffic status comprising nine variables and the target variable, and the samples are taken for six years from 2012 to 2018. First, let’s have a look at the data frame.

``` data = pd.read_csv('metro data.csv')
data ```

Check out the trend using Plotly w.r.to target variable and date; here target variable is nothing but the traffic_volume for one year.

Some of the variables are categorical. So we have to use LabelEncoder to convert it into numbers and use MinMaxScaler to scale down the values. The neural network converges sooner when it exposes the same scaled features and gives better accuracy.

``` for i in data.select_dtypes('object').columns:
le = LabelEncoder().fit(data[i])
data[i] = le.transform(data[i]) ```
``` X_scaler = MinMaxScaler()
Y_scaler = MinMaxScaler()
X_data = X_scaler.fit_transform(data[['holiday', 'temp', 'rain_1h', 'snow_1h', 'clouds_all', 'weather_main',
'weather_description','traffic_volume']])
Y_data = Y_scaler.fit_transform(data[['traffic_volume']]) ```

Below is the user-defined function which preprocesses the data suitable for forecasting.

``` def custom_ts_multi_data_prep(dataset, target, start, end, window, horizon):
X = []
y = []
start = start + window
if end is None:
end = len(dataset) - horizon
for i in range(start, end):
indices = range(i-window, i)
X.append(dataset[indices])
indicey = range(i+1, i+1+horizon)
y.append(target[indicey])
return np.array(X), np.array(y) ```

As we are doing multiple-step forecasting, let’s allow the model to see past 48 hours of data and forecast the 10 hrs after data; for that, we set the horizon to 10.

``` hist_window = 48
horizon = 10
TRAIN_SPLIT = 30000
x_train, y_train = custom_ts_multi_data_prep(X_data, Y_data, 0, TRAIN_SPLIT, hist_window, horizon)
x_vali, y_vali = custom_ts_multi_data_prep(X_data, Y_data, TRAIN_SPLIT, None, hist_window, horizon) ```

The train window should contain eight variables and one target variable for about ten observations.

``` print ('Multiple window of past history\n')
print(x_train)
print ('\n Target horizon\n')
print (y_train) ```

Output:

``` Multiple window of past history
[[0.63636364 0.92972555 0.         0.         0.4        0.1
0.7        0.76167582]
[0.63636364 0.93320863 0.         0.         0.75       0.1
0.06666667 0.62032967]
[0.63636364 0.93391815 0.         0.         0.9        0.1
0.56666667 0.65480769]
[0.63636364 0.93569194 0.         0.         0.9        0.1
0.56666667 0.69038462]
[0.63636364 0.93894927 0.         0.         0.75       0.1
0.06666667 0.67554945]
[0.63636364 0.94081981 0.         0.         0.01       0.
0.73333333 0.71167582]
[0.63636364 0.94549618 0.         0.         0.01       0.
0.73333333 0.76703297]
[0.63636364 0.94772148 0.         0.         0.01       0.
0.73333333 0.82623626]
[0.63636364 0.9486245  0.         0.         0.2        0.1
0.13333333 0.79546703]
[0.63636364 0.94527042 0.         0.         0.2        0.1
0.13333333 0.65521978]
[0.63636364 0.93840101 0.         0.         0.2        0.1
0.13333333 0.48612637]
[0.63636364 0.93327313 0.         0.         0.01       0.
0.73333333 0.38241758]
[0.63636364 0.93078982 0.         0.         0.01       0.
0.73333333 0.32431319]
[0.63636364 0.92611346 0.         0.         0.01       0.
0.73333333 0.21002747]]
Target horizon
[[0.75563187]
[0.78475275]
[0.86428571]
[0.83200549]
[0.67403846]
[0.48118132]
[0.41717033]
[0.38763736]
[0.27362637]
[0.16016484]] ```

Prepare the training data and validation data using the TensorFlow data function, which faster and efficient way to feed data for training.

``` batch_size = 256
buffer_size = 150
train_data = tf.data.Dataset.from_tensor_slices((x_train, y_train))
train_data = train_data.cache().shuffle(buffer_size).batch(batch_size).repeat()
val_data = tf.data.Dataset.from_tensor_slices((x_vali, y_vali))
val_data = val_data.batch(batch_size).repeat() ```

Build and compile the model

``` lstm_model = tf.keras.models.Sequential([
tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(200, return_sequences=True),
input_shape=x_train.shape[-2:]),
tf.keras.layers.Dense(20, activation='tanh'),
tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(150)),
tf.keras.layers.Dense(20, activation='tanh'),
tf.keras.layers.Dense(20, activation='tanh'),
tf.keras.layers.Dropout(0.25),
tf.keras.layers.Dense(units=horizon),
])
lstm_model.summary() ```

Configure the model and start training with early stopping and checkpoint. Early stopping stops training when monitored loss starts increasing above the patience, and checkpoint saves the model weight as it reaches the minimum loss.

``` model_path = 'Bidirectional_LSTM_Multivariate.h5'
early_stopings = tf.keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0, patience=10, verbose=1, mode='min')
checkpoint =  tf.keras.callbacks.ModelCheckpoint(model_path, monitor='val_loss', save_best_only=True, mode='min', verbose=0)
callbacks=[early_stopings,checkpoint] ```

`history = lstm_model.fit(train_data,epochs=150,steps_per_epoch=100,validation_data=val_data,validation_steps=50,verbose=1,callbacks=callbacks)`

Early stopping has done its job; out of 150 epochs model stopped training at 32 epochs.

Check the loss curve for training and validation.

``` plt.figure(figsize=(16,9))
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train loss', 'validation loss'])
plt.show() ```

Prepare the testing data for the last 48 hrs and check the prediction against it by visualizing the actual and predicted values. Finally, evaluate the result with standard performance metrics.

```   data_val = X_scaler.fit_transform(data[['holiday', 'temp', 'rain_1h', 'snow_1h', 'clouds_all', 'weather_main','weather_description', 'traffic_volume']].tail(48))
val_rescaled = data_val.reshape(1, data_val.shape, data_val.shape)
pred = lstm_model.predict(val_rescaled)
pred_Inverse = Y_scaler.inverse_transform(pred)
pred_Inverse ```
``` def timeseries_evaluation_metrics_func(y_true, y_pred):
def mean_absolute_percentage_error(y_true, y_pred):
y_true, y_pred = np.array(y_true), np.array(y_pred)
return np.mean(np.abs((y_true - y_pred) / y_true)) * 100
print('Evaluation metric results:-')
print(f'MSE is : {metrics.mean_squared_error(y_true, y_pred)}')
print(f'MAE is : {metrics.mean_absolute_error(y_true, y_pred)}')
print(f'RMSE is : {np.sqrt(metrics.mean_squared_error(y_true, y_pred))}')
print(f'MAPE is : {mean_absolute_percentage_error(y_true, y_pred)}')
print(f'R2 is : {metrics.r2_score(y_true, y_pred)}',end='\n\n') ```

`timeseries_evaluation_metrics_func(validate['traffic_volume'],pred_Inverse)`

``` plt.figure(figsize=(16,9))
plt.plot( list(validate['traffic_volume']))
plt.plot( list(pred_Inverse))
plt.title("Actual vs Predicted")
plt.ylabel("Traffic volume")
plt.legend(('Actual','predicted'))
plt.show() ```

## Conclusion:

We have seen how the time series forecasting differs from any other prediction technique and the component like a trend; seasonality affects the analysis. We have discussed only one technique from the book, where the author has covered many more techniques for single-step multi-step analysis. The discussed method is quite impressive and can be adept in real-time situations.

## More Great AIM Stories

### How to use genetic algorithm for hyperparameter tuning of ML models? Vijaysinh is an enthusiast in machine learning and deep learning. He is skilled in ML algorithms, data manipulation, handling and visualization, model building.

## Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

### Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

### Telegram Channel

Discover special offers, top stories, upcoming events, and more.