Watch Us Talk To A Generative AI NPC Inside A Game With NVIDIA’s ACE Tech

nvidia ace demo ces2024
As gamers, we've all had that experience of interacting with a game NPC and wishing that you weren't limited to just the conversation options provided by the developer. Now that generative AI is real and finding its way into nearly every industry, we can solve this problem forever, right? That's certainly the goal of NVIDIA's ACE.

If you're not familiar, we've reported on the Avatar Cloud Engine (ACE) before when NVIDIA showed it off at Computex 2023. Essentially, it's a service offered by NVIDIA and startup Convai that allows developers to integrate cloud-based generative AI into their games to allow open-ended converations with NPCs in the game. NVIDIA demoed the tech last year, and has continued to work on it after that original promising-yet-janky demo.

Later in the year, the company debuted SteerLM, a method for tuning a language model (LM) to behave in a certain way, improving the ability of the service to differentiate characters. This year, at CES, NVIDIA is excitedly demonstrating a new function for ACE: real-time conversation with voice input. Instead of typing responses to the NPCs, you can simply talk to them, and they'll respond, complete with dynamically-generated lip sync.

We got to try out the technology at CES, and aside from a bit of latency caused by the spotty network connection on the show floor, it works pretty darn well. You can check out our experience with the tech in our YouTube video, embedded below:

nvidia ace ramen shop demo
The demo is based on the same ramen shop environment from the previous demos.

All of this is done through AI, of course, and even if you have a beefy GeForce RTX 4090 in your machine, that's still not enough GPU for this kind of workload. Instead, it's handled by the cloud servers run by Convai. This does bring along the usual caveats for any cloud-based gaming technology, including network latency for responsiveness and also a near-total shutdown for anyone who wants to permanently preserve your game as an experience.

Still, AI tech continues to advance at a rapid pace, and it's far from inconceivable that technology like this could find its way into a PC near you sooner than later. Both Intel and AMD are eager to get neural processing accelerators, or NPUs, into every single PC. Of course, NVIDIA itself has been focused AI for quite some time. So saying, it may not be long at all before AI-powered game NPCs are the order of the day rather than a novel curiosity.