- Number
- 12450697
- Published
- 2025-10-21
- Filed
- 2022-09-30
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Papas; Marios et al.
- CPC
- G06T5/70; G06N20/00; G06N20/20; G06T5/60
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Volume-render denoising VFX technique.
Abstract
A system includes a hardware processor and a system memory storing software code and one or more machine learning (ML) models. The hardware processor is configured to execute the software code to train a first ML model of the one or more ML models as a denoising feature selector, generate, using the trained first ML model a plurality of candidate feature sets, and identify a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion. The hardware processor is further configured to execute the software code to train, using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser, receive an image including noise due to rendering, and denoise, using the trained denoiser, the noise due to rendering to produce a denoised image.
Background
BACKGROUND (1) Volumetric effects such as fog, smoke, and clouds play an important role in animated movies and visual effects. However, these volumetric effects are among the most computationally expensive effects to render using conventional production techniques. For example, due to its generality and simplicity, the most widely used technique in production rendering is path tracing, in which the light transport in a virtual scene is simulated using Monte-Carlo integration methods. The main drawback of path tracing is computation cost. For complex scenes, in particular those containing volumetric effects, hundreds of hours of computation time on modern computers are typically required to render a single clean frame at final-production quality. Moreover, if the rendering process is stopped prematurely, the resulting image may undesirably exhibit disturbing noise artifacts. (2) One approach to reducing computation time is denoising. That is, instead of waiting for an image to slowly converge to a clean image, rendering is stopped at a relatively early stage, and the intermediate noisy image can be processed in post-production by an algorithm that removes the residual noise in the rendering. However, at present there is no consensus with respect to establishment of a set of volumetric features for improving the preservation of salient volumetric details during post production volume denoising.
Claims
1. A system comprising: a hardware processor and a system memory storing a software code and one or more machine learning (ML) models; the hardware processor configured to execute the software code to: train a first ML model of the one or more ML models as a denoising feature selector; generate, using the trained first ML model, a plurality of candidate feature sets; identify a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion; train, using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser; receive a noisy image including a noise due to rendering; and transform the noisy image to a denoised image; wherein transforming the noisy image to the denoised image comprises: decomposing color included in the noisy image into a surface contribution to the color and a volumetric contribution to the color; denoising, using the trained denoiser, the volumetric contribution to the color to provide a denoised volumetric color result; separately denoising the surface contribution to the color to provide a denoised surface color result; and combining the denoised surface color result with the denoised volumetric color result. ||
8. A method for use by a system including a hardware processor and a system memory storing a software code and one or more machine learning (ML) models, the method comprising: training, by the software code executed by the hardware processor, a first ML model of the one or more ML models as a denoising feature selector; generating, by the software code executed by the hardware processor and using the trained first ML model, a plurality of candidate feature sets; identifying, by the software code executed by the hardware processor, a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion; training, by the software code executed by the hardware processor and using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser; receiving a noisy image, by the software code executed by the hardware processor, the noisy image including a noise due to rendering; and transforming, by the software code executed by the hardware processor, the noisy image to a denoised image wherein transforming the noisy image to the denoised image comprises: decomposing color included in the noisy image into a surface contribution to the color and a volumetric contribution to the color; denoising, using the trained denoiser, the volumetric contribution to the color to provide a denoised volumetric color result; separately denoising the surface contribution to the color to provide a denoised surface color result; and combining the denoised surface color result with the denoised volumetric color result. ||
15. A computer-readable non-transitory storage medium having stored thereon instructions, which when executed by a hardware processor, instantiates a method comprising: training a first ML model of the one or more ML models as a denoising feature selector; generating, using the trained first ML model, a plurality of candidate feature sets; identifying a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion; training, using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser; receiving a noisy image including a noise due to rendering; and transforming the noisy image to a denoised image; wherein transforming the noisy image to the denoised image comprises: decomposing color included in the noisy image into a surface contribution to the color and a volumetric contribution to the color; denoising, using the trained denoiser, the volumetric contribution to the color to provide a denoised volumetric color result; separately denoising the surface contribution to the color to provide a denoised surface color result; and combining the denoised surface color result with the denoised volumetric color result.