NVIDIA unveils AI tool for ultra-realistic and precise 3D editing

Amid artificial intelligence (AI) demonstrating increasingly more capability and usability in various industry branches, particularly image generation and editing, Nvidia has unveiled an AI tool for ultra-realistic and precise 3D editing.

As it happens, at this year’s Conference on Computer Vision and Pattern Recognition (CVPR 2025) in Nashville, Tennessee, June 11-15,  the technology behemoth’s researchers presented a paper on a new machine learning approach that could advance the generation and editing of images.

What Nvidia’s DiffusionRenderer can do

Its name is DiffusionRenderer and it allows users to adjust specific image attributes more precisely than ever. In the words of Sanja Fidler, VP of AI Research at NVIDIA and head of the Spatial Intelligence lab, shared with Tech Xplore and published in a report on July 14:

“Generative AI has made huge strides in visual creation, but it introduces an entirely new creative workflow that differs from classic graphics and still struggles with controllability. (…) With DiffusionRendered, we wanted to bridge that gap by combining the precision of traditional graphics pipelines with the flexibility of AI.”

Specifically, the research team aimed to achieve this with the new approach that can convert individual two-dimensional (2D) videos into graphics-compatible scene representations, such as by adding the possibility to adjust the lighting and materials in the representations, which helps produce new content in line with specific needs and preferences.

Furthermore, the approach relies on diffusion models, a category of deep learning algorithms that can generate images by gradually refining random noise into coherent graphics. Compared to traditional image generation techniques, it works by primarily introducing G-buffers (intermediate image representations outlining specific attributes) and then using these representations to create new and realistic images.

According to Fidler, this is a massive breakthrough as it addresses two longstanding difficulties in computer graphics at the same time. This includes inverse rendering for pulling the geometry and materials from real-life videos, and forward rendering for creating photorealistic images and videos from scene representations. 

In her words:

“One of the most exciting achievements of DiffusionRenderer is that it brings generative AI to the core of graphics workflows and complements it by making traditionally time-consuming tasks like asset creation, relighting, and material editing more efficient.”

Earlier in March, NVIDIA unveiled strategic AI tools to supercharge gaming graphics, including new neural rendering advancements with Unreal Engine 5, Microsoft DirectX, and NVIDIA DLSS 4, which is the most adopted NVIDIA game technology of all time.