Active Hackathon

Introduction to LSTM Autoencoder Using Keras

LSTM autoencoder is an encoder that makes use of LSTM encoder-decoder architecture to compress data using an encoder and decode it to retain original structure using a decoder.
LSTM Autoencoder

Simple Neural Network is feed-forward wherein info information ventures just in one direction.i.e. the information passes from input layers to hidden layers finally to the output layers. Recurrent Neural Network is the advanced type to the traditional Neural Network. It makes use of sequential information. Unlike conventional networks, the output and input layers are dependent on each other. RNNs are called recurrent because they play out a similar undertaking for each component of an arrangement, with the yield being relied upon the past calculations.LSTM or Long Short Term Memory are a type of RNNs that is useful in learning order dependence in sequence prediction problems.

In this article, we will cover a simple Long Short Term Memory autoencoder with the help of Keras and python.

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

What is an LSTM autoencoder?

LSTM autoencoder is an encoder that makes use of LSTM encoder-decoder architecture to compress data using an encoder and decode it to retain original structure using a decoder.

About the dataset

The dataset can be downloaded from the following link. It gives the daily closing price of the S&P index.

Code Implementation With Keras

Import libraries required for this project

import numpy as np
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt

Read the data

df = pd.read_csv('spx.csv', parse_dates=['date'], index_col='date')

Split the data

train_size = int(len(df) * 0.9)
test_size = len(df) - train_size
train, test = df.iloc[0:train_size], df.iloc[train_size:len(df)]
train.shape

Pre-Processing of Data

We need to pre-process the training and test data using the standardscaler library imported from sklearn.

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler = scaler.fit(train[['close']])
train['close'] = scaler.transform(train[['close']])
test['close'] = scaler.transform(test[['close']])

Create a sequence with historical data

Now we will split the time series data into subsequences and create a sequence of 30 days of historical data.

def create_dataset(X, y, time_steps=1):
    X1, y1 = [], []
    for i in range(len(X) - time_steps):
        t = X.iloc[i:(i + time_steps)].values
        X1.append(t)
        y1.append(y.iloc[i + time_steps])
    return np.array(X1), np.array(y1)
TIME_STEPS = 30
X_train, y_train = create_dataset(
  train[['close']],
  train.close,
  TIME_STEPS
)
X_test, y_test = create_dataset(
  test[['close']],
  test.close,
  TIME_STEPS
)
print(X_train.shape)

Creating an LSTM Autoencoder Network

The architecture will produce the same sequence as given as input. It will take the sequence data. The dropout removes inputs to a layer to reduce overfitting. Adding RepeatVector to the layer means it repeats the input n number of times. The TimeDistibuted layer takes the information from the previous layer and creates a vector with a length of the output layers.

import keras
model = keras.Sequential()
model.add(keras.layers.LSTM(
    units=64,
    input_shape=(X_train.shape[1], X_train.shape[2])
))
model.add(keras.layers.Dropout(rate=0.2))
model.add(keras.layers.RepeatVector(n=X_train.shape[1]))
model.add(keras.layers.LSTM(units=64, return_sequences=True))
model.add(keras.layers.Dropout(rate=0.2))
model.add(
  keras.layers.TimeDistributed(
    keras.layers.Dense(units=X_train.shape[2])
  )
)
model.compile(loss='mae', optimizer='adam')
model.summary()

Fitting the Model

Here, we train the model with epoch:20 and batch size 32.

history = model.fit(
    X_train, y_train,
    epochs=20,
    batch_size=32,
    validation_split=0.1,
    shuffle=False
)

Evaluation

plt.plot(history.history['loss'], label='train')
plt.plot(history.history['val_loss'], label='test')
plt.legend();

From the above plot we can see the training and test error is decreasing. For better result, we can train the model with more epochs.

Actual Value of Test Data

y_test

Prediction on Test Data

pred = model.predict(X_test, verbose=0)

Conclusion

In this article, we have covered the basics of Long-short Term Memory autoencoder by using Keras library. Comparing the prediction result and the actual value we can tell our model performs decently. Further, we can tune this model by increasing the epochs to get better results.The complete code of the above implementation is available at the AIM’s GitHub repository. Please visit this link to find the notebook of this code.

More Great AIM Stories

Ankit Das
A data analyst with expertise in statistical analysis, data visualization ready to serve the industry using various analytical platforms. I look forward to having in-depth knowledge of machine learning and data science. Outside work, you can find me as a fun-loving person with hobbies such as sports and music.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM