Depth-Aware Indexing of Three-Dimensional Image Depth Bins

Number: 20240394845
Published: 2024-11-28
Filed: 2024-05-21
Assignee: Disney Enterprises, Inc.
Inventors: Papas; Marios et al.
CPC: G06T3/067; G06T5/70; G06T5/60; G06T9/00; G06T15/00; G06T19/20
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Depth-aware indexing technique for 3D-rendering pipelines (related cluster).

Abstract

A system includes a hardware processor and a memory storing software code. The hardware processor executes the software code to receive a three-dimensional (3-D) image including a plurality of pixels each having a plurality of bins with respective depth values, select, for a first bin of the plurality of bins of a first pixel, a second bin in each of one or more nearest neighbor pixels of the first pixel, the second bin in each of the one or more nearest neighbor pixels having a most similar depth value to a depth value of the first bin. The hardware processor further executes the software code to generate a first depth-aware bin group including the first bin and the second bin in each of the one or more nearest neighbor pixels, and process, using the first depth-aware bin group, the 3-D image to produce a corresponding 3-D output image.

Background

BACKGROUND

Compositing is an important step in the production of animated films and visual effects, in which different parts of a frame are post-processed and fine-tuned independently before being merged together. Three-dimensional (3-D) images, such as deep-Z images for example, contain a variable number of bins per pixel at different depths, each of which records the color and opacity, or “alpha,” at the corresponding depth. As a result, 3-D images can advantageously provide more accurate opacity and avoid edge artifacts in compositing because those 3-D images can cleanly separate distinct geometric boundaries in different bins.

However, path-traced 3-D images generated by renderers presently used in production suffer from the same problem as flat two-dimensional (2-D) images, i.e., noise. Noise reduces the quality of the compositing operations and increases the difficulty of achieving a desired artistic effect. The absence in the conventional art of a denoising solution for 3-D images that can compete with the quality of denoisers on flat 2-D images is one of the primary factors inhibiting the use of 3-D images in production. For example, the present state-of-the-art deep-Z image denoising approach, which filters each bin based on information from neighboring bins, produces artifacts such as residual noise or splotches and is computationally expensive.

Although it is possible to apply state-of-the-art neural network-based denoisers for flat 2-D images

Claims

1. A system comprising: a hardware processor; and a memory storing a software code; the hardware processor configured to execute the software code to: receive a three-dimensional (3-D) image, the 3-D image including a plurality of pixels each having a plurality of bins with respective depth values; select, for a first bin of the plurality of bins of a first pixel of the plurality of pixels, a second bin in each of one or more nearest neighbor pixels of the first pixel, the second bin in each of the one or more nearest neighbor pixels having a most similar depth value to a depth value of the first bin; generate a first depth-aware bin group including the first bin and the second bin in each of the one or more nearest neighbor pixels; and process, using the first depth-aware bin group, the 3-D image to produce a corresponding 3-D output image. || 8. A method for use by a system including a hardware processor and a memory storing a software code, the method comprising: receiving, by the software code executed by the hardware processor, a three-dimensional (3-D) image, the 3-D image including a plurality of pixels each having a plurality of bins with respective depth values; selecting, by the software code executed by the hardware processor for a first bin of the plurality of bins of a first pixel of the plurality of pixels, a second bin in each of one or more nearest neighbor pixels of the first pixel, the second bin in each of the one or more nearest neighbor pixels having a most similar depth value to a depth value of the first bin; generating, by the software code executed by the hardware processor, a first depth-aware bin group including the first bin and the second bin in each of the one or more nearest neighbor pixels; and processing, by the software code executed by the hardware processor and using the first depth-aware bin group, the 3-D image to produce a corresponding 3-D output image. || 15. A computer-readable non-transitory medium having stored thereon instructions, which when executed by a hardware processor, instantiate a method comprising: receiving a three-dimensional (3-D) image, the 3-D image including a plurality of pixels each having a plurality of bins with respective depth values; selecting, for a first bin of the plurality of bins of a first pixel of the plurality of pixels, a second bin in each of one or more nearest neighbor pixels of the first pixel, the second bin in each of the one or more nearest neighbor pixels having a most similar depth value to a depth value of the first bin; generating a first depth-aware bin group including the first bin and the second bin in each of the one or more nearest neighbor pixels; and processing, using the first depth-aware bin group, the 3-D image to produce a corresponding 3-D output image.