|
Listen to this story
|
The year 2022 saw a lot of groundbreaking breakthroughs in the field of AI/ML. Big tech companies like Google, Meta, and Microsoft made major advancements along with completely new innovations starting from LLMs to quantum computing, to generative AI.
For instance, some of the biggest breakthroughs include Meta’s HyperTreeProofSearch (HTPS) that solved International Math Olympiad problems; DeepMind’s Alpha Fold and Meta AI’s ESMFold for protein fold prediction; Google’s DeepNull that models the relationship linking covariate effects among phenotypes and improves Genome-Wide Association Studies (GWAS). The list goes on.
Following this, let’s look at some of the predictions for 2023.
Read: 10 Biggest Algorithmic Breakthroughs of 2022
Refining of LLMs and SLMs
ChatGPT took the internet by storm for its excellent conversational abilities. It was built on OpenAI’s GPT-3 which has 176 billion parameters, which lies on the larger size of models. While there are other LLMs that have double, triple, or even ten times more parameters than GPT-3, there are models by DeepMind or Meta with half the number of parameters (aka small language models (SLMs)) that have been outperforming GPT-3 on several tasks like logical reasoning and prediction.
Apart from reducing the size of models, a larger model like GPT-4, with around 100 trillion parameters is expected. The jump would be massive as the current largest model is Google’s Switch Transformer model with 1.6 trillion parameters.
However, for greater latency and predictability, fine-tuning of existing models to serve specific purposes can be seen in the coming years. Recently, OpenAI fine–tuned GPT-3 with the DaVinci update.
Generative AI calls for Explainable AI
Text-to-image generation was the trend that broke the charts in 2022. Models like DALL-E, Stable Diffusion, and Midjourney were on top of the list for enthusiasts who wanted to experiment with AI to generate art. The conversation moved from text-to-image to text-to-video to text-to-anything pretty quickly and multiple models were created that could generate 3D models as well.
The text-to-anything trend is expected to rise even higher with language models expanding along with improvement in diffusion models. The publicly available datasets are making the generative AI models even more expandable.
These datasets bring in the part about explainable AI, where the attribution of each image that these generative models are trained on becomes essential.
Read: Stable Diffusion vs Midjourney vs DALL.E2
FastSaaS Race Begins
Companies catching up with the trend of generative AI have started offering it as a service on the cloud. With LLMs and generative models like GPT-3 and DALL-E being openly available, it is getting easier for companies to offer them as services. This gave birth to FastSaaS.
In recent times, Shutterstock plans to integrate DALL-E 2 to its platform, Microsoft VS Code added Copilot, as an extension, TikTok announced the rollout of an in-app text-to-image AI generator, and Canva came out with an AI-generating feature on its platform.
Reliance on Supercomputer
This is where the trend of building supercomputers to rely for generative tasks, along with offering services comes in for companies. With these increasing datasets and generative models, the need for supercomputers is rising and is expected to rise even further. And with the race to FastSaaS, the need for better and high performing computing is the next thing.
NVIDIA and Microsoft recently partnered to create Quantum-2, a cloud native supercomputing platform. In October, Tesla announced its Dojo supercomputer built entirely from scratch using chips developed by Tesla. Soon, it looks to provide access to enterprise customers. Also, Cerebras unveiled Andromeda, a 13.5-million core AI supercomputer delivering more than 1 exaflop of AI compute. Recently, Jasper partnered with Cerebras to enable better performance.
Looking Beyond 3nm Chips
As Moore’s law predicts, processing power increases with decreasing chip size. So for a supercomputer to run a large model, smaller chips are required, and we are already seeing chips getting smaller and smaller.
In recent years, there has been a push towards miniaturisation in the chip industry, with manufacturers constantly looking for ways to make chips smaller and more compact. For instance, for M2 chip and A16, Apple is using 5 nm and 4 nm chips respectively, with expectations of a 3 nm chip developed by TSMC in 2023. This will increase the efficiency and performance for development of AI/ML algorithms.
Amalgamation of Quantum and Traditional Computing
With companies like NVIDIA, Google, and Microsoft offering their hardware services to the cloud, more innovations in the quantum computing front are bound to happen. This will allow small tech companies to train, test, and build AI/ML models without needing heavy hardware.
The rise of quantum computing in the coming years should definitely be incorporated by developers as its use will increase in a number of other domains like healthcare, financial services, among several others.
In a recent announcement, a quantum computer was connected to Europe’s fastest supercomputer, for combining traditional and quantum computers for quicker solution of problems. Similarly, Nvidia has also released QODA – in short Quantum-Optimised Device Architecture, which is a first-of-its-kind platform for hybrid quantum-classical computers.
IBM recently announced during its annual Quantum Summit 2022 their quantum hardware and software outlining the pioneering vision for quantum-centric supercomputing with a 433-quantum-bit (qubit) processor. At the Global AI Summit, IBM announced that next year they will demonstrate a 1000-qubit system and this will be a disruptor for further innovations in every field.
