NVIDIA Inpainting Uses AI To Magically Rebuild Corrupted Or Damaged Images
This process, which NVIDIA calls image inpainting, can be used not only for restoring missing image pixels, but also for removing an unwanted object from a scene and filling it back in. If this sounds familiar, it's because Photoshop has been able to perform similar operations with Content-Aware Fill, which was introduced in the CS5 release.
However, NVIDIA goes a step further given that inpainting was trained using 55,116 random streaks and holes to improve its performance. Another 25,000 were generated to further test the algorithm's accuracy.
“Our model can robustly handle holes of any shape, size location, or distance from the image borders,” wrote the NVIDIA researchers in a paper on inpainting. “Further, our model gracefully handles holes of increasing size.
“To the best of our knowledge, we are the first to demonstrate the efficacy of deep learning image inpainting models on irregularly shaped holes.”
NVIDIA of course has plenty of high-powered hardware to pull from for its machine learning exercises, and in this case used Tesla V100 GPUs linked up with a cuDNN-accelerated PyTorch deep learning framework.
To see NVIDIA image inpainting in action, simply take a look at the embedded video above. The results are remarkable, especially when looking at woman's "gouged out" eyes being replaced with two new ones. The results aren't always perfect, but it's still a remarkable achievement that shows a lot of promise.
Eventually, NVIDIA says that its image inpainting technique could be integrated into photo editing software.
