Now Reading
Anomaly Detection in Temperature Sensor Data using LSTM RNN Model

Anomaly Detection in Temperature Sensor Data using LSTM RNN Model

anomaly detection using LSTM

Anomaly detection has been used in various data mining applications to find the anomalous activities present in the available data. With the advancement of machine learning techniques and developments in the field of deep learning, anomaly detection is in high demand nowadays. This is because of implementing machine learning algorithms with heavy datasets and generating more accurate results. The anomaly detection is not limited now to detecting the fraudulent activities of customers, but it is also being applied in industrial applications in a full swing.

In manufacturing industries, where heavy machinery is used,  the anomaly detection technique is applied to predict the abnormal activities of machines based on the data read from sensors. For example, based on the temperature data read through the sensors, the possible failure of the system can be predicted. In this article, we will discuss how to detect anomalies present in the temperature data that is available in the time-series format. This data is captured from the sensors of an internal component of a large industrial machine.

What is Anomaly?

In machine learning and data mining, anomaly detection is the task of identifying the rare items, events or observations which are suspicious and seem different from the majority of the data. These anomalies can indicate some kind of problems such as bank fraud, medical problems, failure of industrial equipment, etc. The anomaly detection has two major categories, the unsupervised anomaly detection where anomalies are detected in an unlabeled data and the supervised anomaly detection where anomalies are detected in the labelled data. There are various techniques used for anomaly detection such as density-based techniques including K-NN, one-class support vector machines, Autoencoders, Hidden Markov Models, etc. 



The Data set

In this experiment, we have used the Numenta Anomaly Benchmark (NAB) data set that is publicly available on Kaggle. It is a novel benchmark for evaluating machine learning algorithms in anomaly detection in streaming, online applications. It consists of more than 50 labelled real-world and artificial time-series data files plus a novel scoring mechanism designed for real-time applications. This dataset also comprises a time-series data named ‘machine_temperature_system_failure’ in the CSV format. It comprises temperature sensor data of an internal component of a large industrial machine. This dataset contains anomalies including the shutdown of the machine, catastrophic failure of the machine etc. 

Implementation of Anomaly Detection

To implement our work, first, we need to import the required libraries. The Pandas library is required for data frame operations and the NumPy library is required for array operations. For plotting, we are importing matplotlib and seaborn libraries and for the preprocessing of the data, we are importing the preprocessing library. For the LSTM Recurrent Neural Network, the required Keras libraries are imported. The time library is used in viewing the compile time of our LSTM RNN model.

#Importing Required Libraries
import pandas as pd
import numpy as np
import matplotlib
import seaborn
import matplotlib.dates as md
from matplotlib import pyplot as plt
from sklearn import preprocessing
from keras.layers.core import Dense, Activation, Dropout
from keras.layers.recurrent import LSTM
from keras.models import Sequential
import time

As this implementation was done in Google Colab, we use the below code snippet to read the data from the local computer system.

#To upload file from local system
from google.colab import files
uploaded = files.upload()
upload file in Google Colab

Once the upload is shown as 100%, we will read the CSV data file that is the temperature sensor data in the time-series format.

#Reading file from local system
df = pd.read_csv("ambient_temperature_system_failure.csv")

To check whether the data is read successfully, we see the header of the file and then visualize the data through plotting. 

df.head()
temperature sensor data
#Visualize the data
figsize=(10,5)
df.plot(x='timestamp', y='value', figsize=figsize, title='Temperature (Degree Farenhite)');
plt.grid();
plt.show();
temperature sensor data plot

As we can see that the dates do not appear clearly on the X-axis, we need to change the type of timestamp column. As the temperature in our data set is given in the degree Fahrenheit, we will convert it into degree Celcius. 

#Changing the type of timestamp column for visualization
df['timestamp'] = pd.to_datetime(df['timestamp'])
df['value'] = (df['value'] - 32) * 5/9
df.plot(x='timestamp', y='value', figsize=figsize);
plt.title('Temperature (Degree Celcius)', fontsize=16);
plt.grid();
plt.show();
temperature sensor data


Now we can see the data is visualized perfectly. To check the stability of temperature during days and nights of weekdays and weekends we are going to preprocess our data accordingly. First, we will specify hours, then days, then weekdays and nights. Finally, we will visualize the temperature during these time-periods using a histogram. 

#Formating the data into required format
df['hours'] = df['timestamp'].dt.hour
df['daylight'] = ((df['hours'] >= 7) & (df['hours'] <= 22)).astype(int)
df['DayOfTheWeek'] = df['timestamp'].dt.dayofweek
df['WeekDay'] = (df['DayOfTheWeek'] < 5).astype(int)

# Anomaly estimated population
outliers_fraction = 0.01

df['time_epoch'] = (df['timestamp'].astype(np.int64)/100000000000).astype(np.int64)

df['categories'] = df['WeekDay']*2 + df['daylight']
a = df.loc[df['categories'] == 0, 'value']
b = df.loc[df['categories'] == 1, 'value']
c = df.loc[df['categories'] == 2, 'value']
d = df.loc[df['categories'] == 3, 'value']

#Visualizing the formatted data
figsize=(10,5)
fig, ax = plt.subplots(figsize=figsize)
a_heights, a_bins = np.histogram(a)
b_heights, b_bins = np.histogram(b, bins=a_bins)
c_heights, c_bins = np.histogram(c, bins=a_bins)
d_heights, d_bins = np.histogram(d, bins=a_bins)
width = (a_bins[1] - a_bins[0])/6
ax.bar(a_bins[:-1], a_heights*100/a.count(), width=width, facecolor='blue', label='Weekend Night')
ax.bar(b_bins[:-1]+width, (b_heights*100/b.count()), width=width, facecolor='green', label ='Weekend Light')
ax.bar(c_bins[:-1]+width*2, (c_heights*100/c.count()), width=width, facecolor='red', label ='Weekday Night')
ax.bar(d_bins[:-1]+width*3, (d_heights*100/d.count()), width=width, facecolor='black', label ='Weekday Light')
plt.legend()
plt.show()
temperature distribution; anomaly detection using LSTM

The above histogram shows that the temperature is comparatively more stable during Week Days in the daylights. Now, we will preprocess our dataset for training the LSTM recurrent neural network. For this purpose, first, we will take the required columns from the dataset and scale it using Standard Scaler. 

#Preparing the data for LSTM model
data_n = df[['value', 'hours', 'daylight', 'DayOfTheWeek', 'WeekDay']]
min_max_scaler = preprocessing.StandardScaler()
np_scaled = min_max_scaler.fit_transform(data_n)
data_n = pd.DataFrame(np_scaled)

In the next step, we will define and initialize the required parameters and define the training and the test data set. We are going to learn from 50 previous values and we predict through the LSTM model just the one next value.

#Important parameters and training/Test size
prediction_time = 1 
testdatasize = 1000
unroll_length = 50
testdatacut = testdatasize + unroll_length  + 1

#Training data
x_train = data_n[0:-prediction_time-testdatacut].values
y_train = data_n[prediction_time:-testdatacut  ][0].values

#Test data
x_test = data_n[0-testdatacut:-prediction_time].values
y_test = data_n[prediction_time-testdatacut:  ][0].values

As we know that the architecture of a Recurrent Neural Network has a hidden state. The hidden state at time t is a function of the hidden state at time t−1 and the input at time t. This hidden state at the time 0 is typically initialized to 0. The fundamental reason why RNNs are unrolled is that all previous inputs and hidden states are used in order to compute the gradients with respect to the final output of the RNN. The below unrolling function will create a sequence of 50 previous data points for each of the training and test data point.

def unroll(data,sequence_length=24):
    result = []
    for index in range(len(data) - sequence_length):
        result.append(data[index: index + sequence_length])
    return np.asarray(result)

#Adapt the datasets for the sequence data shape
x_train = unroll(x_train,unroll_length)
x_test  = unroll(x_test,unroll_length)
y_train = y_train[-x_train.shape[0]:]
y_test  = y_test[-x_test.shape[0]:]

Now, we will see the final shape of our training and test data. 

#Shape of the data
print("x_train", x_train.shape)
print("y_train", y_train.shape)
print("x_test", x_test.shape)
print("y_test", y_test.shape)

shape of the dataset 

In the next step, we will define and add layers to the LSTM Recurrent Neural Network one-by-one. For basic details about the LSTM RNN model, you may refer to the article ‘How to Code Your First LSTM Network in Keras’. 

See Also

#Building the model
model = Sequential()

model.add(LSTM(input_dim=x_train.shape[-1], output_dim=50, return_sequences=True))
model.add(Dropout(0.2))

model.add(LSTM(100, return_sequences=False))
model.add(Dropout(0.2))

model.add(Dense(units=1))
model.add(Activation('linear'))

start = time.time()
model.compile(loss='mse', optimizer='rmsprop')
print('compilation time : {}'.format(time.time() - start))

LSTM compile time

Once, the LSTM RNN model is defined and compiled successfully, we will train our model. The below hyperparameters can be tuned to check the better performance. 

model.fit(x_train, y_train, batch_size=3028, nb_epoch=50, validation_split=0.1)
LSTM Training

After the successful training of the model, we will visualize the training performance.

#Visualizing training and validaton loss
plt.figure(figsize = (10, 5))
plt.plot(model.history.history['loss'], label = 'Loss')
plt.plot(model.history.history['val_loss'], Label = 'Val_Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.grid()
plt.legend()

LSTM Training and validation loss

Now, we will execute the below code snippet to better understand the difference between the original data and the predicted data through the visualization.

#creating the list of difference between prediction and test data
loaded_model = model
diff=[]
ratio=[]
p = loaded_model.predict(x_test)
for u in range(len(y_test)):
    pr = p[u][0]
    ratio.append((y_test[u]/pr)-1)
    diff.append(abs(y_test[u]- pr))

#Plotting the prediction and the reality (for the test data)
plt.figure(figsize = (10, 5))
plt.plot(p,color='red', label='Prediction')
plt.plot(y_test,color='blue', label='Test Data')
plt.legend(loc='upper left')
plt.grid()
plt.legend()

prediction by LSTM model ; anomaly detection using LSTM

Now, in the next step, we are going to find the anomalies. The most distant predicted values are considered as anomalies. Using the below code snippet, we will find the anomalies in the data.

#Pick the most distant prediction/reality data points as anomalies
diff = pd.Series(diff)
number_of_outliers = int(outliers_fraction*len(diff))
threshold = diff.nlargest(number_of_outliers).min()
#Data with anomaly label
test = (diff >= threshold).astype(int)
complement = pd.Series(0, index=np.arange(len(data_n)-testdatasize))
df['anomaly27'] = complement.append(test, ignore_index='True')
print(df['anomaly27'].value_counts())

Finally, we will visualize the anomalies using the below code for plotting. 

#Visualizing anomalies (Red Dots)
plt.figure(figsize=(15,10))
a = df.loc[df['anomaly27'] == 1, ['time_epoch', 'value']] #anomaly
plt.plot(df['time_epoch'], df['value'], color='blue')
plt.scatter(a['time_epoch'],a['value'], color='red', label = 'Anomaly')
plt.axis([1.370*1e7, 1.405*1e7, 15,30])
plt.grid()
plt.legend()

anomaly detection using LSTM

As we can see in the above picture, the anomalies are visualized as red points. All the above steps can be repeated multiple times to visualize the anomalies by tuning the hyperparameters. If we find the same visualization at the end, then we can end-up with these anomalies. 

References:

  1. A Zimek, E Schubert, “Outlier Detection”, Encyclopedia of Database Systems, Springer New York.
  2. V. J. Hodge, J Austin, “A Survey of Outlier Detection Methodologies”, Artificial Intelligence Review.
  3. BoltzmannBrain, “Numenta Anomaly Benchmark: Dataset and scoring for detecting anomalies in streaming data”, Kaggle.
  4. Victor Ambonati, “Unsupervised Anomaly Detection”, Kaggle.

Provide your comments below

comments


If you loved this story, do join our Telegram Community.


Also, you can write for us and be one of the 500+ experts who have contributed stories at AIM. Share your nominations here.

Copyright Analytics India Magazine Pvt Ltd

Scroll To Top