Google AI has announced the launch of an application that turns sketches into fantastical and imaginary creatures.
The tech giant, in its recent blog post, described this new prototype application — Chimera Painter, which is an ML model that has been trained “automatically creates a fully fleshed out rendering from a user-supplied creature outline.” The application allows users to add features and textures to a sketch of a creature with body part labels, such as “teeth”, “wings” or “claws” and with the transform button on the application, it creates a very accurate looking monster.
To facilitate this process of creating high-quality images of creatures, the company decided to experiment with generative adversarial networks (GANs) which made the creatures appropriate for the fantasy card game prototype.
Explaining better, the blog stated that the GANs paired two convolutional neural networks against each other — a generator network to create new images and a discriminator network that would determine if these images are samples from the training dataset or not. For this, the company deployed a conditional GAN, where the generator network picks up a separate input to guide the image generation process.
To train the GANs, the company created a training dataset of full-colour images with single-species creature outlines adapted from 3D creature models, stated by the blog post. The outline characterised the shape and size of each creature and provided a segmentation map that identified each of the body parts. This enabled the model to generate new creatures based on the drawings of different body parts.
Once the model is being trained, the model is asked to generate multi-species chimeras, based on sketch outlines provided by artists. The model is then incorporated into Chimera Painter, based on its performance.
One can try the tool with their own sketches on Chimera Painter website. One can also load pre-designed images and sketches or can generate new ones in the application.
“It is our hope that these GAN models and the Chimera Painter demonstration tool might inspire others to think differently about their art pipeline,” stated in the blog post by the creators. “What can one create when using machine learning as a paintbrush?”
The application has been proposed and designed with an aim to help artists with an application that “acted less like a tool and more like an assistant.
