NVIDIA Uses AI To Create Immersive Virtual Game Worlds From Real World Video
It's said that seeing is believing, but can you always believe what you see? That's a question that will come up more and more as companies like NVIDIA push the envelope with graphics rendering and, interestingly enough, artificial intelligence. As it relates to that, in addition to launching a new Titan RTX graphics card today, NVIDIA also announced a new deep learning-based model for generating 3D environments based on real-world data.
A team of NVIDIA researchers led by Bryan Catanzaro, Vice President of Applied Deep Learning at NVIDIA, used a conditional generative neural network as a starting point. From that data, they trained a neural network to render new 3D environments, after it was already trained on existing videos.
“NVIDIA has been creating new ways to generate interactive graphics for 25 years—and this is the first time we can do this with a neural network," Catanzaro said. "Neural networks—specifically—generative models are going to change the way graphics are created."
According to NVIDIA, this "AI breakthrough" will enable developers and artists to create new interactive 3D virtual worlds quicker and cheaper for automotive, gaming, and virtual reality, by training models on videos from the real world.
"One of the main obstacles developers face when creating virtual worlds, whether for game development, telepresence, or other applications is that creating the content is expensive. This method allows artists and developers to create at a much lower cost, by using AI that learns from the real world," Catanzaro said.
According to Catanzaro, this technique would not have been possible before the introduction of Tensor cores, as found in NVIDIA's Volta and Turing GPU architectures.
NVIDIA is presenting a demo of the technology at the NeurIPS conference in Montreal, Canada. It consists of a simple driving game that allows attendees to interactively navigate an AI-generated environment.
This new method is in the early stages of development. However, NVIDIA believes this technology will ultimately make it much cheaper and easier to create virtual environments once it's fleshed out.
