- Published on January 13, 2023
- In AI Features
Beyond the Surface: The Role of Diffusion in Language Models
Diffusion models became the paradigm for generative models in 2022. Unlike autoregressive models that require restricted connectivity patterns for ensuring causality, diffusion models are unconstrained and thus allow more creative freedom. But given the efficiency of current autoregressive models (example ChatGPT), is researching diffusion models in the language field even worth it?
- By Mohit Pandey
Diffusion Models were introduced in the field of AI in 2015. Jascha Sohl-Dickstein, finding inspiration from the physics principles of non-equilibrium thermodynamics, developed the technique for generating images that outperformed GANs in terms of quality and speed. This gave rise to the successful text-to-image models like DALL.E2 and Stable Diffusion.
Sander Dieleman, Research Scientist at DeepMind, recently published a blog exploring if diffusion models can also be used for language tasks. He perfectly iterates the part about how we proceed with language in real life—starting with a basic concept in mind about a topic and then writing words, phrasing, and structuring comes later. This concept, interestingly, looks similar to how diffusion models work and the approach can definitely
📣 Want to advertise in AIM? Book here
Mohit Pandey
Mohit writes about AI in simple, explainable, and often funny words. He's especially passionate about chatting with those building AI for Bharat, with the occasional detour into AGI.
Related Posts
Building an AI Economy That Includes Everyone
“What we’re seeing is that accessibility-driven design often solves a broader problem. It’s not charity. It’s engineering.”
Top 10 Companies That Crowned Hyderabad as India’s Greenfield GCC Leader in 2025
Telangana has attracted over 75 greenfield GCCs in 2025, compared with 40-plus in Karnataka.
The AI Coding Gold Rush Ends Where Harness Begins
“Only 30% of software engineering happens on the laptop. The real 70% starts after you commit the code,” says Jyoti
How Gradient-Boosting is Quietly Powering India’s Research Push
From groundwater and slopes to carbon sinks, tools like CatBoost are enabling Indian scientists to extract insights and drive sustainability.
India’s Data Centre Boom Is Running Into a Talent Wall
With capacity expected to more than double this decade, the industry is investing in training as graduates struggle to meet
This Firm Wants to be the ‘Next Big Disruptor’ in Networking
Arrcus positions itself as a horizontal software layer that can run across different types of networking hardware.
Will 2026 be the year of AI IPOs?
With CoreWeave’s listing and Fractal Analytics going for an IPO, an array of AI companies are now looking to raise
Fighting Deepfakes May Not Be a Technology Problem
Defenders must be active at all times, while attackers need only one opportunity.
Download the easiest way to
stay informed
stay informed
