A.I. turns doodles into landscapes

Nvidia G.A.N. creates a new kind of art

The new Nvidia prototype software can turn doodles into amazing landscapes. It’s still far from being perfect but it’s really astonishing how it can paint beautiful and dreamy landscapes with just a few simple rough brushstrokes made by us.

Using a type of AI model known as a Generative Adversarial Network (GAN), the software gives users what Nvidia is calling a “smart paintbrush”. So, someone can make a very basic outline of a scene before filling in their rough sketch with natural textures like grass, clouds, forests, sea, or rocks.

This software isn’t properly new because researchers have shown off similar tools in the past, including one from Google that turns doodles into clipart but this is the most polished demonstration of this concept. The software generates AI landscapes instantly, and it’s surprisingly intuitive. You just have to click what you’re going to paint (sky, sea, clouds, grass, etc…), sketch some strokes, and they will turn into a more realistic picture.

In the demo, you can use 3 tools: a brush (to draw), a paint bucket (to fill an area), and an eyedropper (to sample a color). You can also choose a starting image from the ones available or you can upload yours. Moreover, you can upload a segmentation map (labeled sketches that depict the layout of a scene) or a custom style filter image (to suggest a style for the AI image).

However, this technology doesn’t always give perfect results and can’t just paint in any texture you can think of but the images created often have an evocative atmosphere.

Generating fake grass and water is relatively easy for GANs because the visual patterns involved are unstructured. Generating pictures of buildings and furniture, by comparison, is much trickier, and the results are much less realistic. That’s because these objects have a logic and structure that humans are sensitive to. GANs can overcome this sort of challenge but it takes a lot of extra effort.

The images generated by the GAN are said to be multimodal, namely, the same sketch gives different results each time.

To train the AI, they have used 1 million photos from Flicker.

Nvidia didn’t say if it has any plans to turn the software into an actual product, but it suggests that tools like this could help “everyone from architects and urban planners to landscape designers and game developers” in the future.

Here you can try to paint your own AI image with the GAN, available in the Nvidia Playground.

We have been seeing how many things an AI can do; art will be another field that Artificial Intelligence will explore. Maybe it won’t be as easy to replace as for other works, because this deals with creativity but probably we’ll have a new kind of art and artists. I don’t think classic art will be replaced because it’s not just an automatable job and maybe people will always look for creativity made by humans.

Source theverge