Google DeepMind Just Made Small Models Irrelevant with RRTs

While there’s no shortage of efforts to build powerful LLMs that solve all sorts of challenges, Google DeepMind’s approach to reducing the cost, computing, and resources required for an LLM to function, is a ray of hope for various environmental and sustainability concerns.  Google DeepMind, in collaboration with KAIST AI, suggests a method called Relaxed Recursive Transformers, or RRTs.  It allows LLMs to be programmed to behave like small language models yet outperform many of the standard SLMs present in the ecosystem today.  Less is More Google DeepMind proposes a technique called Layer Tying. Instead of processing the input through a large number of layers, it can be made to pass through a small number of layers recursively, so that it creates an equivalent