Google Trains A Trillion Parameter Model, Largest Of Its Kind

Switch transformer

Google has developed and benchmarked Switch Transformers, a technique to train language models, with over a trillion parameters. The research team said the 1.6 trillion parameter model is the largest of its kind and has better speeds than T5-XXL, the Google model that previously held the title. Switch Transformer According to the researchers, the Mixture […]