ADAPTIVE SAMPLING USING DEEP LEARNING

Number: 20230334612
Published: 2023-10-19
Filed: 2022-04-14
Assignee: Disney Enterprises, Inc.
Inventors: PAPAS; Marios, RÖTHLIN; Gerhard, DAHLBERG; Henrik D., SALEHI; Farnood, ADLER; David M., MEYER; Mark A., MAZZONE; Andre C., SCHROERS; Christopher R., MANZI; Marco, VOGELS; Thijs, CHRISTENSEN; Per H.
CPC: G06T5/60; G06T1/20; G06N3/096; G06N3/08; G06N3/045; G06T5/70
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Deep-learning adaptive-sampling technique for VFX rendering.

Abstract

Certain aspects of the present disclosure provide techniques for adaptive sampling for rendering using deep learning. This includes receiving, at a sampler in a rendering pipeline, a plurality of rendered pixel data, wherein the sampler includes a first machine learning (ML) model. It further includes generating a sampling map for the rendering pipeline using the first ML model and the plurality of rendered pixel data, including predicting a plurality of pixel values in the sampling map based on a generated distribution of pixel values. It further includes rendering an image using the sampler, the sampling map, and a denoiser in the rendering pipeline.

Background

BACKGROUND

Generating high-quality animated content with convincing lighting often relies on computationally expensive light transport simulations on computer clusters, commonly referred to as rendering. The light transport simulations during rendering can compute how light interacts with virtual content (e.g., created by artists). But the rendering process can take dozens, or hundreds, of CPU hours of computation time for a single frame of animation. Because many animated productions (e.g., animated feature-length movies or television shows) are made up of hundreds of thousands of frames, reducing the rendering cost per frame plays a significant role in reducing the production costs of movies that include rendered content.

For example, light transport simulation in existing renderers can be performed using Monte-Carlo simulation techniques. This can include repeatedly sampling random paths of light in a given scene reflected in an image, and computing how much light is transported through these paths to a specific pixel in the image. This computation can be performed independently for every pixel in the image, and the more such paths are sampled, the more accurate the color value of a pixel will be. SUMMARY

Embodiments include a method. The method includes receiving at a sampler in a rendering pipeline a plurality of rendered pixel data, wherein the sampler includes a first machine learning (ML) model. The method further includes generating a sampling ma

Claims

1. A method, comprising: receiving at a sampler in a rendering pipeline a plurality of rendered pixel data, wherein the sampler comprises a first machine learning (ML) model; generating a sampling map for the rendering pipeline using the first ML model and the plurality of rendered pixel data, comprising: predicting a plurality of pixel values in the sampling map based on a generated distribution of pixel values; and rendering an image using the sampler, the sampling map, and a denoiser in the rendering pipeline. || 11. A non-transitory computer-readable medium containing computer program code that, when executed by operation of one or more computer processors, performs operations comprising: receiving at a sampler in a rendering pipeline a plurality of rendered pixel data, wherein the sampler comprises a first machine learning (ML) model; generating a sampling map for the rendering pipeline using the first ML model and the plurality of rendered pixel data, comprising: predicting a plurality of pixel values in the sampling map based on a generated distribution of pixel values; and rendering an image using the sampler, the sampling map, and a denoiser in the rendering pipeline. || 17. A system, comprising: a computer processor; and a memory having instructions stored thereon which, when executed on the computer processor, performs operations comprising: receiving at a sampler in a rendering pipeline a plurality of rendered pixel data, wherein the sampler comprises a first machine learning (ML) model; generating a sampling map for the rendering pipeline using the first ML model and the plurality of rendered pixel data, comprising: predicting a plurality of pixel values in the sampling map based on a generated distribution of pixel values; and rendering an image using the sampler, the sampling map, and a denoiser in the rendering pipeline.