Google’s NLP-Powered Pretraining Method ALBERT Is Leaner & Meaner

Natural Language Processing (NLP) is one of the most diversified domains in emerging tech. Last year, search engine giant Google open-sourced a technique known as Bi-directional Encoder Representations from Transformers (BERT) for NLP pre-training. This model helped the researchers to train a number of state-of-the-art models in about 30 minutes on a single Cloud TPU, or in a few hours using a single GPU.      Now, the researchers at Google designed A Lite BERT (ALBERT) which is a modified version of the traditional BERT model. The latest model incorporates two-parameter reduction techniques which are factorised embedding parameterisation and cross-layer parameter sharing in order for lifting the major obstacles in scaling pre-trained models in NLP. ALBERT has outperformed the benchm