The bigger the better? Google AI’s new 540 billion parameter language model PaLM 

Large language transformer models are able to constantly benefit from bigger architectures and increasing amounts of data. Since 2018, larger language models like BERT and its variants GPT-2 and GPT-3 have shown that a wide array of tasks can be performed using few-shot learning. Models like Microsoft and NVIDIA’s Megatron-Turing Natural Language Generation, which had 530 billion parameters, Generalist Language Model’s (GLaM) full version, which contained 1.2 trillion parameters, LaMDA or Language Models for Dialogue Applications which had 137 billion parameters; and Gopher which had 280 billion parameters, have marked the past few years just because of their sheer size. Has the desire to build bigger and bigger models become a mindless race?  A new paper released by Google