Guide to SelfTime: Self-supervised Time Series Representation Learning Framework with Python code

SelfTime is the state-of-the-art time series framework by finding inter-sample and intra-temporal relations

Time-series forecasting is one of the most widely dealt with machine learning problems ever. Time series forecasting finds crucial applications in various fields including signal communication, climate, space science, healthcare, financial and marketing industries. Deep learning models outshine in time series analysis nowadays with great performance in various public datasets. 

The key idea of deep learning models is to learn the inter-sample relationships to predict the future. However, intra-temporal relationships among different features within a sample are hardly dealt with. Extracting intra-temporal relationships yields better pattern understanding in the time series representation learning even with fewer data samples. Without an intra-temporal relationship extraction system, a deep learning model becomes data-hungry and requires a lot of manually-annotated tabular data for supervised training.


Sign up for your weekly dose of what's up in emerging technology.

Present self-supervised representation learning approaches let models train with unannotated as well as fewer data. Self-supervised learning approaches yield huge success in different computer vision tasks including image inpainting, video inpainting, object tracking, rotation prediction and pace prediction. However, these approaches fail to capture the complete temporal-relationships in the case of high-dimensional time series structured data. 

Haoyi Fan and Fengbin Zhang of Harbin University of Science and Technology, China and Yue Gao of Tsinghua University, China introduced SelfTime, the Self-supervised Time Series Representation Framework. This framework explores inter-sample relationships among subsequent samples and intra-temporal relationships within each sample to capture the underlying spatial and temporal patterns in an unannotated structured time series dataset. 

How does SelfTime work?

A sample or a piece of a sample taken for exploring the pattern is called an anchor sample or an anchor piece respectively. In the case of inter-sample relation reasoning, a sample is taken as an anchor and is transformed using the standard augmentation method to develop a new transformed-time-series sample that is denoted the positive sample. Another real sample is taken from some nearer time scale and is denoted the negative sample. Inter-sample relation reasoning is captured between these positive and negative samples. 

Concept of inter-sample relation reasoning and Intra-temporal relation reasoning (Source)

On the other hand, a sample of interest is broken into pieces to learn intra-temporal relationships. One of the broken pieces is taken as an anchor and the rest are called references. Intra-temporal relation reasoning is performed between the anchor and each of the references individually. The number of references for each anchor can be varied based on the problem. If three references are chosen for analysis, it is termed 3-scale temporal relations and the references are called short-term, middle-term and long-term relations based on the temporal distance between the anchor and references.

A shared representation learning backbone is developed with two relation-reasoning-heads based on the inter-sample relations and the intra-sample relations extracted separately. Finally, the time series representations are extracted from unlabeled data by supervising the two different relation-reasoning-heads. Since SelfTime explores in-depth relationships among different data samples and different time pieces, it exhibits extraordinary performance in representation learning of time series data, especially high-dimensional data.

Architecture of SelfTime framework (Source)

Python implementation of SelfTime

The desired requirements for SelfTime architecture building are Python 3.6 or 3.7, Pytorch 1.4.0 and CUDA GPU runtime. The following command downloads the source code to the local machine. 

!git clone


Verify proper download by exploring the SelfTime directory using the command,

!ls SelfTime


Change directory to run the containing files using the following line-magic command

%cd SelfTime/

The following command pre-trains the inter-sample relation head-on in-built CricketX dataset

!python --dataset_name CricketX --model_name InterSample

The following command pre-trains the intra-temporal relation head-on in-built CricketX dataset

!python --dataset_name CricketX --model_name IntraTemporal

The following command pre-trains SelfTime model on in-built CricketX dataset

!python --dataset_name CricketX --model_name SelfTime


Once trained the framework parts separately, the three parts of the framework can be evaluated on the test data. The following commands evaluate the three networks on the in-built CricketX data’s test set.

 !python --dataset_name CricketX --model_name InterSample
 !python --dataset_name CricketX --model_name IntraTemporal
 !python --dataset_name CricketX --model_name SelfTime 

Finally, supervised training of the whole framework based on the pre-trained individual networks can be performed using the following command. This training is governed by early stopping based on on-time evaluation performance. 

!python --dataset_name CricketX --model_name SupCE

A portion of the output:


The results of the final supervised training are stored in a new directory called ‘results’. Model performance can be explored by visualizing the losses and other metrics over iterations.

Performance of SelfTime

SelfTime has been evaluated and compared with other well-acclaimed models using public time-series datasets such as CricketX, UWaveGestureLibraryAll (UGLA), DodgerLoopDay (DLD), and InsectWingbeatSound (IWS) from the UCR Time Series Archive, and the bearing datasets XJTU2 and MFPT

Transformed data samples are generated by augmenting the magnitude domain and the time domain such as jittering, scaling, cutout, magnitude warping, time warping, window warping and window slicing. These transformed samples are used as positive samples to extract inter-sample relationships. All the recent state-of-the-art models have undergone identical augmentations to justify comparisons.

Different transformations performed on real data (blue in colour) to generate positive samples (red in colour).

SelfTime massively outperforms recent state-of-the-arts in time-series domain namely, Triplet Loss model (2019), Deep InfoMax (2019), Forecast (2020), Transformation (2020), SimCLR (2020) and Relation (2020).

Qualitative comparison of state-of-the-art models with SelfTime on UGLA classification dataset. Classes are differentiated using different colours. SelfTime classifies the classes clearly.

Further reading

More Great AIM Stories

Rajkumar Lakshmanamoorthy
A geek in Machine Learning with a Master's degree in Engineering and a passion for writing and exploring new things. Loves reading novels, cooking, practicing martial arts, and occasionally writing novels and poems.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM