Interview With NVIDIA And CD Projekt Sheds Light on DLSS 3.5 Ray Reconstruction
In case you couldn't tell from our coverage, some of us around here—particularly your present author—are big fans of Cyberpunk 2077. That's not only because it's a legitimately great game these days, but also because it is a showcase for the absolute bleeding edge in graphics technology. That will continue to be the case in coming days as the game gets its 2.0 update and its Phantom Liberty expansion.
The big-ticket item arriving with the 2.0 update tomorrow is DLSS 3.5 with Ray Reconstruction. We've already written about that upcoming feature at length, but a new interview revealed some interesting details about the technique, its creation, the history of DLSS, and its future.
The interview was hosted by Alex Battaglia of Digital Foundry, and featured two fellows from NVIDIA: Bryan Catanzaro, NV's VP of Applied Deep Learning, and Jacob Freeman, who joined the company as "GeForce Evangelist" after departing from EVGA earlier this year. Also present were CD Projekt Red's Jakub Knapik, Global Art Director, and Pedro Valadas, the founder of the /r/pcmr "PC Master Race" subreddit.
The topic of discussion was very distinctly DLSS, particularly (but not exclusively) DLSS 3.5 and its new Ray Reconstruction feature. As an extremely brief recap of our previous coverage, Ray Reconstruction replaces the hand-tuned denoisers that ray-traced games normally use with the same AI that performs upscaling as part of DLSS Super Resolution. This isn't about increasing performance like other DLSS features; as Catanzaro says, there are already "lots of tools to make framerates really high."
Instead, DLSS Ray Reconstruction is about improving image quality. You see, some ray-traced effects don't mesh well with DLSS Super Resolution. That's because the denoising is done separately from the upscaling, so it's done at the lower input resolution. This gives the denoiser less detail to work with, and it's also doing its own temporal integration. Ultimately this means the denoisers have to "blur a lot of the detail away."
The ultimate solution to this conundrum was to create DLSS Ray Reconstruction (RR). In this operational mode, the DLSS AI performs both denoising and super-resolution upscaling. It's all done in one place, which means that not only do you get vastly superior image quality, it's also both more accurate and can even be faster, if it replaces the work of multiple hand-tuned denoisers.
None of this is really new information, but Catanzaro did elucidate on the fact that you don't have to be doing full-on path tracing to make use of DLSS RR. It can work with any ray-traced effect, even if you're only doing shadows or reflections. However, the performance cost is the same either way, and it's about four times as demanding as doing Super Resolution alone. The cost is pretty low overall; he gives a figure of 2 milliseconds per frame at 4K on an RTX 4090; lower resolutions will take less time, although weaker GPUs will take longer.
Unlike DLSS Super Resolution where you are trading quality for performance, Ray Reconstruction has "virtually no downside", according to Freeman. Both he and Catanzaro as well as CDPR's Knapik agreed that DLSS RR actually looks better than rendering the game at native resolution where it presumably falls back to the hand-tuned denoisers. NVIDIA says when using DLAA, but haven't heard back yet.
The reason that DLSS RR apparently looks better than native rendering is because it allows for finer detail in the lighting and shadowing. This particularly applies to characters, according to Knapik; when asked by Battaglia if there was any single element of the game that RR really affected, he immediately replied "characters." He says that they became "victims of the denoising" in the path-traced mode, with too few samples at too low of a resolution to really give good light definition on the faces. Ray Reconstruction solves that.
Catanzaro explained that DLSS RR can look better than native because the AI is able to make smarter decisions than humans can. They described denoisers as a "bag of tricks with a lot of knobs to tweak," and said that it takes skilled professionals a lot of time to tune denoisers by hand. By contrast, AI can do things that we don't know how to write algorithms for by looking at huge datasets and learning much more complicated functions than we can achieve by manually constructing algorithms by hand.
By the way, don't be confused by the "DLSS 3.5" nomenclature. Ray Reconstruction can be used on any GeForce RTX GPU, going all the way back to the "Turing" RTX 20 series. Because the numeric versions can be disorienting, NVIDIA's reps actually recommended sticking with feature names for the various DLSS functions. So far, those are Super Resolution, Frame Generation, and Ray Reconstruction. Only Frame Generation requires a GeForce RTX 40 series GPU.
There's a lot more to the hour-long interview than we'll cover in this post, and it's worth a watch if you're interested in the topic. They go over some pretty insightful questions, like whether developers are using DLSS as a crutch (they say no, of course) and if it's possible to mod DLSS RR into games like people have done with Super Resolution and Frame Generation.
DLSS 3.5 with Ray Reconstruction debuts tomorrow with Cyberpunk 2077's version 2.0 update. You don't have to buy the Phantom Liberty expansion to try it out, but you will need a GPU capable of running the game's path-traced "technology preview". Officially, that means a GeForce RTX 3090 or faster GPU, but realistically speaking you can achieve playable performance with something like a GeForce RTX 3070 Ti if you're willing to drop the resolution enough.
