Google Transformer Model Reformer Works On A Single GPU & Is Memory Efficient

Since its inception in 2017, transformer models have become popular among researchers and academia in the machine and deep learning sector. This deep machine learning model is used for various natural language processing (NLP) tasks such as language understanding, machine translation, among others. In one of our articles, we had discussed why transformers play such a crucial role in NLP development. Massive transformer models have the capability to achieve state-of-the-art results on a number of tasks. However, training these models especially on long sequences can be a costly affair and can have massive computations. These large models when trained with model parallelism fails to be fine-tuned on a single GPU.  To overcome the issue of cost and massive computations, recently, t