Techniques for performing point-based inverse rendering

Number: 11074743
Published: 2021-07-27
Filed: 2019-09-27
Assignee: Disney Enterprises, Inc.
Inventors: Öztireli; Ahmet Cengiz, Sorkine-Hornung; Olga, Wu; Shihao, Wang; Yifan
CPC: G06T5/73; G06T11/40; G06T7/90; G06T15/205
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Point-based inverse rendering technique (granted).

Abstract

In various embodiments, a differentiable rendering application enables an inverse rendering application to infer attributes associated with a 3D scene. In operation, the differentiable rendering application renders an image based on a first set of points associated with the 3D scene. The differentiable rendering application then generates an artificial gradient that approximates a change in a value of a first pixel included in the image with respect to a change in an attribute of a first point included in the first set of points. Subsequently, the inverse rendering application performs optimization operation(s) on the first point based on the artificial gradient to generate a second set of points. Notably, an error associated with the second set of points is less than an error associated with the first set of points.

Background

BACKGROUND Field of the Various Embodiments (1) Embodiments relate generally to computer science and computer vision and, more specifically, to techniques for performing point-based inverse rendering. Description of the Related Art (2) Neural networks trained to recognize patterns in input data based on example patterns included in training data can be used to solve a wide variety of “computer vision” problems. In computer vision, computers are programmed to infer, analyze, and/or process attributes of three-dimensional (“3D scenes”) from digital two-dimensional (“2D”) images. Some examples of these types of attributes include, without limitation, coordinates, normals, colors of points, camera position and orientation, and lighting. (3) For instance, in an iterative training process that includes a forward rendering pass and a backward inverse rendering pass, a neural network can be trained to infer attributes associated with a 3D scene based on a 2D reference image. In the forward rendering pass, a rendering function can be used to generate a 2D image based on a set of inferred attributes. In the backward inverse rendering pass, an error can be computed based on the difference between the generated image and the reference image and optimization operations can be performed on the set of inferred attributes to reduce that error. (4) With respect to the backward inverse rendering pass, optimizing the inferred attributes may require the rendering function to be differentiable wi

Claims

1. A computer-implemented method for inferring attributes associated with a three-dimensional (3D) scene, the method comprising: generating a first image based on a first plurality of points associated with the 3D scene; generating a gradient that represents a change in a value of a pixel included in the first image with respect to a given point included in the first plurality of points; and performing one or more optimization operations on the given point based on the gradient to generate a second plurality of points, wherein an error associated with the second plurality of points is less than an error associated with the first plurality of points. || 11. One or more non-transitory computer readable media including instructions that, when executed by one or more processors, cause the one or more processors to infer attributes associated with a three-dimensional (3D) scene by performing the steps of: generating a first image based on a first plurality of points associated with the 3D scene; generating a gradient that represents a change in a value of a pixel included in the first image with respect to a given point included in the first plurality of points; and performing one or more optimization operations on the given point based on the gradient to generate a second plurality of points, wherein an error associated with the second plurality of points is less than an error associated with the first plurality of points. || 20. A system, comprising: one or more memories storing instructions; and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to: generate a first image based on a first plurality of points associated with the 3D scene; generating a gradient that represents a change in a value of a pixel included in the first image with respect to a given point included in the first plurality of points; and performing one or more optimization operations on the given point based on the gradient to generate a second plurality of points, wherein an error associated with the second plurality of points is less than an error associated with the first plurality of points.