Stability AI’s Stable Diffusion, the AI image generator is one of the most widely hailed models in the field. When the team decided to open-source it, communities from the field of AI and beyond were excited to make new innovations with it. But, just like any other open-source system, it did not take long for the internet to build something that generated adult-only content. 

Enter scene: ‘Unstable Diffusion’, a server that is dedicated to the production of AI-generated NSFW content. It has been trending in a small art generation community. Started in August 2022, the system generates realistic and anime-style images—mostly consisting of women—and other nude characters.

Click here to try it out.

There have been several communities and subreddits that have been banned in the past for creating sexual content on the platform, but Unstable Diffusion is among the largest yet—with their Patreon account still running and generating funds up to $2,500 a month to keep the development running. 

Arman Chaudhry, member of the admin team of Unstable Diffusion, told TechCrunch, “In just two months, our team expanded to over 13 people as well as many consultants and volunteer community moderators.” Chaudhry is also the founder of ‘Equilibrium AI’ and claims to have generated more than 4,375,000 images till date.

Only 2.9% of Stable Diffusion’s dataset contains NSFW material. Admins therefore recruit volunteers to create porn datasets in order to fine-tune Stable Diffusion. The company is also planning to seek venture funding in their kickstarter campaign.

Read: The Societal Dangers of DALL-E 2

Though Chaudhry’s words seem optimistic in the light of the expansion of the community, AI art generators have been in one too many controversies since their launch—such as copyright issues and ethical concerns—for allowing them into art competitions. Currently, there is no government policy in place to stop the moderation of content on these image generators, rendering them more or less unfiltered. 

While Midjourney, DALL-E, and Stable Diffusion too, have filtered out NSFW content, how do we restrict models such as these from popping up?