How Transformers Are Making Headway In Reinforcement Learning

In 2017, Google researchers announced Transformers to the world in a NIPS paper titled “Attention is All You Need”. The self-attention mechanism-based novel neural network architecture has since been used in several NLP applications, GPT-3 being a good case in point. Transformers--thanks to their ability to integrate information over long time horizons and scale to a large amount of data--have achieved success in domains such as language modelling and machine translation.  Reinforcement learning using Transformers Reinforcement learning is one of the three machine learning paradigms, along with supervised and unsupervised learning. Reinforcement learning is a goal-oriented algorithm that works towards attaining a complex goal where every correct step is rewarded. In other