Adaptive sampling using deep learning

Number: 12373909
Published: 2025-07-29
Filed: 2022-04-14
Assignee: Disney Enterprises, Inc.
Inventors: Papas; Marios et al.
CPC: G06T5/60; G06T1/20; G06N3/096; G06N3/045; G06N3/08; G06T5/70
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Deep-learning adaptive-sampling rendering technique.

Abstract

Certain aspects of the present disclosure provide techniques for adaptive sampling for rendering using deep learning. This includes receiving, at a sampler in a rendering pipeline, a plurality of rendered pixel data, wherein the sampler includes a first machine learning (ML) model. It further includes generating a sampling map for the rendering pipeline using the first ML model and the plurality of rendered pixel data, including predicting a plurality of pixel values in the sampling map based on a generated distribution of pixel values. It further includes rendering an image using the sampler, the sampling map, and a denoiser in the rendering pipeline.

Background

BACKGROUND (1) Generating high-quality animated content with convincing lighting often relies on computationally expensive light transport simulations on computer clusters, commonly referred to as rendering. The light transport simulations during rendering can compute how light interacts with virtual content (e.g., created by artists). But the rendering process can take dozens, or hundreds, of CPU hours of computation time for a single frame of animation. Because many animated productions (e.g., animated feature-length movies or television shows) are made up of hundreds of thousands of frames, reducing the rendering cost per frame plays a significant role in reducing the production costs of movies that include rendered content. (2) For example, light transport simulation in existing renderers can be performed using Monte-Carlo simulation techniques. This can include repeatedly sampling random paths of light in a given scene reflected in an image, and computing how much light is transported through these paths to a specific pixel in the image. This computation can be performed independently for every pixel in the image, and the more such paths are sampled, the more accurate the color value of a pixel will be. SUMMARY (3) Embodiments include a method. The method includes receiving at a sampler in a rendering pipeline a plurality of rendered pixel data, wherein the sampler includes a first machine learning (ML) model. The method further includes generating a sampling map for the

Claims

1. A method, comprising: receiving at a sampler in a rendering pipeline a plurality of rendered pixel data comprising a distribution of rendered pixels, wherein the sampler comprises a first machine learning (ML) model; generating a sampling map for the rendering pipeline using the first ML model and the plurality of rendered pixel data, comprising: generating an analytical distribution having a shape that matches a shape of the distribution of rendered pixel data; and predicting a plurality of pixel values in the sampling map based on the generated analytical distribution; and rendering an image using the sampler, the sampling map, and a denoiser in the rendering pipeline, wherein the sampling map is generated before rendering the image in the denoiser. || 10. A non-transitory computer-readable medium containing computer program code that, when executed by operation of one or more computer processors, performs operations comprising: receiving at a sampler in a rendering pipeline a plurality of rendered pixel data comprising a distribution of rendered pixels, wherein the sampler comprises a first machine learning (ML) model; generating a sampling map for the rendering pipeline using the first ML model and the plurality of rendered pixel data, comprising: generating an analytical distribution having a shape that matches a shape of the distribution of rendered pixel data; and predicting a plurality of pixel values in the sampling map based on the generated analytical distribution; and rendering an image using the sampler, the sampling map, and a denoiser in the rendering pipeline, wherein the sampling map is generated before rendering the image in the denoiser. || 15. A system, comprising: a computer processor; and a memory having instructions stored thereon which, when executed on the computer processor, performs operations comprising: receiving at a sampler in a rendering pipeline a plurality of rendered pixel data comprising a distribution of rendered pixels, wherein the sampler comprises a first machine learning (ML) model; generating a sampling map for the rendering pipeline using the first ML model and the plurality of rendered pixel data, comprising: generating an analytical distribution having a shape that matches a shape of the distribution of rendered pixel data; and predicting a plurality of pixel values in the sampling map based on the generated analytical distribution; and rendering an image using the sampler, the sampling map, and a denoiser in the rendering pipeline, wherein the sampling map is generated before rendering the image in the denoiser.