Nvidia GauGAN 2 AI creates landscape images from simple text

Nvidia has just released the newest version of GauGAN 2, an AI painter that can translate simple text inputs into beautifully painted lanscape images.

It can be used to generate beautiful artworks from a wide range of texts. The project's underlying algorithms use a process called GAN (Generative Adversarial Networks) which can produce images that have a high level of realism and detail.

The system was trained on collections of works by famous artists, before being unleashed on any text provided as input data.

For example, just write a sentence like a 'sunset at a beach,' and AI will create the scene in real-time.

The new GauGAN2 text-to-image functionality is now available here.

So, how does GauGAN 2 work?

GauGAN 2 is a strong tool for creating photorealistic art using a combination of words and drawings since it integrates segmentation mapping, inpainting, and message production in one model.

The demo is one of the first to include numerous modalities into a single GAN framework, including text, semantic segmentation, drawing, and style. This allows turning an artist's vision into a high-quality AI-generated image faster and easier.

Artists may utilize the sample to create fantastic landscapes as well as real photographs. Users may use a brief word to rapidly produce the major characteristics and theme of an image, such as a snow-capped mountain range, rather than having to sketch out every detail of an imagined landscape.

This starting point may then be tweaked using drawings to make a certain mountain taller or to put a couple of trees in the foreground or clouds in the sky.

It's an iterative process in which each word entered into the text field adds to the AI-generated image. GauGAN 2's AI model was trained on 10 million high-quality landscape photographs on the NVIDIA Selene supercomputer, an NVIDIA DGX SuperPOD system that is among the top ten most powerful supercomputers in the world.

The researchers employed a neural network that learns the relationship between words and the sights they represent, such as "winter," "foggy," or "rainbow."

Key points to take away:

• After being trained on over 1 million public Flickr pictures, Nvidia's first GauGAN system, named after post-Impressionist painter Paul Gauguin, was launched in 2019. It was used to develop Nvidia's quick painting Canvas app, which is now in beta.
• GauGAN2 was trained on a dataset of 10 million pictures. It "integrates segmentation mapping, inpainting, and text-to-image generation into a single model."
• When compared to other cutting-edge models, GauGAN2's neural network generates "a wider diversity and higher-quality of pictures."
• Users enter a short word, such as "snow-capped mountain range," and the image is generated. The user may then draw a rough sketch of the scene and use it to construct a segmentation map defining the position of items.