NVIDIA recently announced the latest version of NVIDIA Research’s AI painting demo, GauGAN2. The new model is powered by deep learning and consists of a text-to-image feature. The original version could only turn a rough sketch into a detailed image. GauGAN2 can generate images from phrases like ‘sunset at a beach,’ which can then be further modified with adjectives like ‘rocky beach,’ or by changing ‘sunset’ to a different time of day or even modifying weather conditions.
Powered by generative adversarial networks (GAN), NVIDIA spokesperson said “With the press of a button, users can generate a segmentation map, a high-level outline that shows the location of objects in the scene. From there, they can switch to drawing, tweaking the scene with rough sketches using labels like sky, tree, rock and river, allowing the smart paintbrush to incorporate these doodles into stunning images.”
By adding text-to-image capabilities, the new version of GauGAN is more customisable and can be tuned much quicker. Even a quick sketch is not nearly as fast and simple as typing a phrase. The latest version is also one of the first AI models to incorporate multiple modalities, text, semantic segmentation, sketch, and style, within a single GAN network.
The Text-based starting point, such as a ‘snow-capped mountain range,’ can be further customised with sketching. Users can add trees, change the height and size of objects, add clouds to the sky and much more. And then GauGAN2 generates a new, modified image.
GauGAN2 may prove useful for concept artists, as they can now create worlds with two suns, like Tatooine in Star Wars. ‘It’s an iterative process, where every word the user types into the text box adds more to the AI-created image,’ NVIDIA spokesperson added further.
Earlier this year, NVIDIA released a tool built upon GauGAN, NVIDIA Canvas, which can be used on any NVIDIA RTX GPU. In addition, GauGAN2 has been trained on 10 million landscape images using the NVIDIA Selene supercomputer, which is among the world’s ten most powerful supercomputers. If you’d like to try your hand at creating simulated photos of places that never existed, head over to NVIDIA’s AI Playground and click on the “Launch Interactive Demo” button for GauGAN2.
