Volume Denoising with Feature Selection

Number: 20230109328
Published: 2023-04-06
Filed: 2022-09-30
Assignee: Disney Enterprises, Inc.
Inventors: Papas; Marios et al.
CPC: G06T5/70; G06N20/00; G06N20/20; G06T5/60
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Volumetric-rendering denoising technique with feature selection.

Abstract

A system includes a hardware processor and a system memory storing software code and one or more machine learning (ML) models. The hardware processor is configured to execute the software code to train a first ML model of the one or more ML models as a denoising feature selector, generate, using the trained first ML model a plurality of candidate feature sets, and identify a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion. The hardware processor is further configured to execute the software code to train, using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser, receive an image including noise due to rendering, and denoise, using the trained denoiser, the noise due to rendering to produce a denoised image.

Background

BACKGROUND

Volumetric effects such as fog, smoke, and clouds play an important role in animated movies and visual effects. However, these volumetric effects are among the most computationally expensive effects to render using conventional production techniques. For example, due to its generality and simplicity, the most widely used technique in production rendering is path tracing, in which the light transport in a virtual scene is simulated using Monte-Carlo integration methods. The main drawback of path tracing is computation cost. For complex scenes, in particular those containing volumetric effects, hundreds of hours of computation time on modern computers are typically required to render a single clean frame at final-production quality. Moreover, if the rendering process is stopped prematurely, the resulting image may undesirably exhibit disturbing noise artifacts.

One approach to reducing computation time is denoising. That is, instead of waiting for an image to slowly converge to a clean image, rendering is stopped at a relatively early stage, and the intermediate noisy image can be processed in post-production by an algorithm that removes the residual noise in the rendering. However, at present there is no consensus with respect to establishment of a set of volumetric features for improving the preservation of salient volumetric details during post production volume denoising.

Claims

1. A system comprising: a hardware processor and a system memory storing a software code and one or more machine learning (ML) models; the hardware processor configured to execute the software code to: train a first ML model of the one or more ML models as a denoising feature selector; generate, using the trained first ML model, a plurality of candidate feature sets; identify a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion; train, using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser; receive an image including a noise due to rendering; and denoise, using the trained denoiser, the image with the noise due to rendering to produce a denoised image. || 8. A method for use by a system including a hardware processor and a system memory storing a software code and one or more machine learning (ML) models, the method comprising: training, by the software code executed by the hardware processor, a first ML model of the one or more ML models as a denoising feature selector; generating, by the software code executed by the hardware processor and using the trained first ML model, a plurality of candidate feature sets; identifying, by the software code executed by the hardware processor, a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion; training, by the software code executed by the hardware processor and using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser; receiving an image, by the software code executed by the hardware processor, the image including a noise due to rendering; and denoising, by the software code executed by the hardware processor and using the trained denoiser, the noise due to rendering in the image to produce a denoised image. || 15. A computer-readable non-transitory storage medium having stored thereon instructions, which when executed by a hardware processor, instantiates a method comprising: training a first ML model of the one or more ML models as a denoising feature selector; generating, using the trained first ML model, a plurality of candidate feature sets; identifying a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion; training, using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser; receiving an image, including a noise due to rendering; and denoising, using the trained denoiser, the noise due to rendering in the image to produce a denoised image.