Intel Labs AI Diffusion Model Generates Stunning 3D Environments From Simple Text

hero intel ldm3d grassy
Contemporary AI advancements have been incredible in just the last couple of years. Generative AI has really taken off, proving once again that all you really need to convince consumers of the value of a technology is to give them a toy they can play with. (Just look at 3D video games.) The "toys" in the case of generative AI have been neural networks capable of generating increasingly-convincing text, audio, image, and even video with a simple prompt.

What we haven't seen before is a generative model that can create 3D environments. That's what Intel has just debuted: a diffusion model like Stable Diffusion, except for "realistic 3D visual content." It's called LDM3D, and the environments it presents can be surprisingly complex and detailed, even with simple prompts. As you can see, they're not photographic; they're clearly 3D art. Still, the results are very cool.

intel ldm3d marketplace
The images retain many features of typical Stable Diffusion-style AI generation.

To be clear, this result isn't quite as awesome as it appears at first. While the images are extremely impressive, these aren't truly 3D models being created; you couldn't import the result from this AI into a video game, for example. What's happening is that the AI is generating a 360º 2D image along with a matching depth map that provides the convincing illusion of true 3D. The two are combined to create a 360º 3D image that you can even view in VR.

In fact, Intel has a 360º video up on YouTube that demonstrates the technology, and if you have a VR device, you can even watch it in VR. (If you've never seen one of these videos, click and drag or move your mobile device in 3D space to move the camera around.)

Unsurprisingly, the LDM3D AI model is based on Intel's prior work in estimating 3D depth using 2D images—you may recall Intel's RealSense depth cameras. The blue team's boys say it was trained on a combination of Xeon processors and Gaudi2 accelerators, and that it used some 10,000 samples from the LAION-400M database.

More surprising is that Intel has totally open-sourced the model and it's available for free right now. You can head over to Hugging Face, a site that hosts a great number of open-source AI models (including the original Stable Diffusion and many variants), and download LDM3D if you know what to do with it. However, there's no easy web interface like with Stable Diffusion (yet), so if you're not real handy with Python, then you'd better wait for the packaged solution.
Tags:  Intel, (NASDAQ:INTC), AI