Meet The Sketchy Cousin of Stable Diffusion, ‘Scribble Diffusion’

This open-source app can convert your hand drawn art, along with a text-prompt, into an art piece.
Listen to this story

What more than you need if AI can convert your rough scribbling into a refined image, and make it look like a masterpiece? An engineer from Replicate, who goes by the GitHub name zeke, created just that. Scribble Diffusion, is an open-source app that can convert your hand drawn art, along with a text-prompt, into an art piece.

The model is built on ControlNet, a neural network structure developed by Lvmin Zhang and Maneesh Agrawala for controlling large diffusion models with additional input conditions, which is human scribbling in this case. The model conditions Stable Diffusion with other inputs like edge detection, depth maps, and pose estimation.

Check out the app here.

Subscribe to our Newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

ControlNet was released only three weeks ago and the developers are already building apps with it. Another developer built an app that generates DreamBooth images by matching the pose of the input stick figure. 

Click here to check out the GitHub repository and build your own app.

Stable Diffusion Cousins

Since the release of open-source Stable Diffusion, developers and ML enthusiasts have been experimenting with it to create different types of models. 

Firstly, it does not take much time for a new technology to be released and users start trying to make NSFW content with it. That is what happened with Unstable Diffusion. Stability AI has been under a lot of scrutiny since the release of this model.

Other models include, Riffusion, for generating real-time AI music by converting images of audio into audio. The approach involves fine-tuning Stable Diffusion for creating spectrograms, which are visualisations of audio. Another model, Amazing AI, was built on top of Stable Diffusion 1.5 that runs natively on macOS, without the need of the internet.

Mohit Pandey
Mohit dives deep into the AI world to bring out information in simple, explainable, and sometimes funny words. He also holds a keen interest in photography, filmmaking, and the gaming industry.

Download our Mobile App

MachineHack | AI Hackathons, Coding & Learning

Host Hackathons & Recruit Great Data Talent!

AIMResearch Pioneering advanced AI market research

With a decade of experience under our belt, we are transforming how businesses use AI & data-driven insights to succeed.

The Gold Standard for Recognizing Excellence in Data Science and Tech Workplaces

With Best Firm Certification, you can effortlessly delve into the minds of your employees, unveil invaluable perspectives, and gain distinguished acclaim for fostering an exceptional company culture.

AIM Leaders Council

World’s Biggest Community Exclusively For Senior Executives In Data Science And Analytics.

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox