NVIDIA’s AI system can create photorealistic images from your drawings

NVIDIA's AI system can create photorealistic images from your drawings

Credit: NVIDIA

For many, the difference between AI and the human brain lies in creativity. It’s hard to teach a machine to express itself, though that may be about to change.

NVIDIA has created an AI system that turns your scribbles into photorealistic landscape images. GauGAN Beta gives users a canvas resembling Microsoft Paint to draw onto. As if by artificial intelligence-powered magic, the machine creates an image on the right of your screen based on what you’ve cobbled together with the painting and filling tools.

The results are somewhat rough so far. You can, however, switch up the filter of your picture to create different effects. The tool demonstrates how far “machine drawing” has come in the last few years.

Creativity is a difficult thing for science and technology to analyse. Expression often comes from specific influence and human experience; it’s hard to quantify a particular magic about an artistic piece and the greatest pieces of artwork, whether musical, painted or otherwise, are often iconic, at least partly, thanks to a backstory. For years, there has been a debate about whether machines and computers – especially AI – can truly produce art to rival a human being.

NVIDIA’s latest attempt to create an algorithm that produces art is not strictly AI art, really. The tool relies on a user to draw a basic sketch first. It may be debatable as to whether AI can create art, but this system simply assists with photorealistic collages.

It is inevitable that programs like Adobe Photoshop will incorporate more AI capabilities. Artificial intelligence can be used to smartly erase pixels, generate parts of an image to save you time and provide clever edits based on how a picture looks. Is it taking the fun out of photo editing? Maybe. AI can make photo editing and drawing far more accessible to the average user, though.

How does AI assist with images?

Cameras obviously take photos by analysing what it sees, pixel by pixel. Most cameras don’t use artificial intelligence though. They simply take an aggregate colour sample for every pixel of their resolution.

This is where AI comes in. Whilst a camera will analyse surroundings to take a sample colour, AI creates the sample colour itself based on data. The data in the case of the NVIDIA system is whether you want to create a rock, tree, plant, river or something else.


Creativity is a difficult thing for science and technology to analyse.


In the case of Photoshop, Adobe introduced a Content-Aware tool a few years ago. Say you have a picture of a person standing on a beach. You can select the subject and use Content-Aware to delete them seamlessly from the image: AI is used by the Content-Aware tool to fill in the pixels with randomly generated colours, based on how the rest of the image looks. So if you’re erasing someone from a beach scene, the chances are that Content-Aware will fill in the space with similar blues and greens of the sea and similar yellows of the beach.

These kinds of tools usually employ Machine Learning (ML). A set of two or more ML algorithms work in conjunction with each other: one tries to produce a realistic image whilst the other one challenges the flaws of that image. It’s similar to the way that deepfake works and by learning from its mistakes, it manages to produce better images based on data.

The more data that you feed an algorithm, the more realistic your finished image will be. The NVIDIA AI system is a little rough so far and doesn’t create the most like-life images so far but it isn’t too much of a stretch to imagine more defined, authentic images with a more powerful processor.

How could this AI system be used?

The tales of what artificial intelligence could do are mind-boggling. There’s talk of how just the picture of an ear could be used to generate an entire face from AI and be used in criminal reconstructions. It’s possible to create extremely realistic computer-generated models for retail.

Creating realistic landscapes with quick sketches opens doors in the worlds of entertainment and digital art. Factor in the imminent rise of VR, thanks to 5G, and it might not be long before we are able to build three-dimensional worlds simply by drawing them out in a Paint-like program first.

In the world of design, AI could be groundbreaking. Architects could use systems like the NVIDIA GauGAN to create designs with realistic textures in a flash. Interior designers could do a similar thing, utilising the speed and smartness of AI to bring in textures to their designs easily. If you’re looking to create a mock-up of anything, from a building to a complete garden, artificial intelligence could well become the default starting block for designers of all walks of life.

For now however, GauGAN is a tool for messing around and having fun. AI might not be inherently creative itself, but there’s no reason that you can’t get creative with it.

ai, AI, NVIDIA's AI system can create photorealistic images from your drawings

Mark White

Mark is a writer/editor who has written online and in print.

The vital role business plays in the business intelligence equation

Phillip Smith • 03rd December 2021

As organizations continue turning their attention to technology advancements, to maintain some form of ‘business as usual’, there has never been a greater need for companies to truly understand the data they hold, and how to interpret it. Phillip Smith, the chief architect at workplace data analytics firm Tiger, explains more.

2,500 years of threat intelligence and its value continues to...

Anthony Perridge • 01st December 2021

Anthony Perridge, VP International at ThreatQuotient discusses how threat intelligence has evolved to form an essential aspect of modern-day cybersecurity. By harking back to the practices of our ancestors, today’s threat hunters can take inspiration about evaluating threat data to maximize the best possible decisions.

The gaming industry’s latest challenge: DDoS protection

Babur Khan • 30th November 2021

As the industry faces a rise in DDoS threats, it is time that gaming companies truly prioritise cybersecurity and business resiliency. In his latest article, Babur Khan,Technical Marketing Engineer at A10 Networks, lays out a framework for the industry to do just this, securing their customers and their own success against the tide of cyberattacks.

The Best Ten Rated Cloud Security Management Options For Business

Erin Laurenson • 24th November 2021

Cloud Security programs that can carry out safety procedures and address or flag potential high-risk elements are now critical, allowing businesses to function normally without fearing a potential breach. To help you find the best Cloud management and security system for your business, we’ve done the research and found the top systems presently available on...