Appearance capture under multiple lighting conditions

Number: 12086927
Published: 2024-09-10
Filed: 2021-10-20
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Urnau Gotardo; Paulo Fabiano et al.
CPC: G06V20/64; G01N21/55; G06V10/16; G06V10/60; G06T15/50; G06V40/16; G06T15/205
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Multi-lighting-condition appearance-capture rendering technique (granted).

Abstract

One embodiment of the present invention sets forth a technique for performing appearance capture. The technique includes receiving a first sequence of images of an object, wherein the first sequence of images includes a first set of images interleaved with a second set of images, and wherein the first set of images is captured based on illumination of the object using a first lighting pattern and the second set of images is captured based on illumination of the object using one or more lighting patterns that are different from the first lighting pattern. The technique also includes generating a first set of appearance parameters associated with the object based on a first inverse rendering associated with the first sequence of images.

Background

BACKGROUND Field of the Various Embodiments (1) Embodiments of the present disclosure relate generally to computer science and computer graphics and, more specifically, to techniques for appearance capture under multiple lighting conditions. Description of the Related Art (2) Realistic digital faces are required for various computer graphics and computer vision applications. For example, digital faces are oftentimes used in virtual scenes of film or television productions and in video games. (3) A digital face can be synthesized using three-dimensional (3D) geometry of the face and two-dimensional (2D) appearance maps indicating reflectance properties of the face. As used herein, “appearance” refers to reflectance properties characterizing how light interacts with materials before reaching a camera. Example reflectance properties include, without limitation, shininess (also referred to as “specular intensity”) and the shape of a specular reflection lobe (also referred to as “roughness”), which are surface properties of skin models, as well as shading-free color (also referred to as “diffuse albedo”), which is mostly a subsurface property but can include contributions from the surface of skin as well. The reflectance properties of human skin, and more particularly the skin on a face, can vary due to skin type, tanning, blood flow caused by muscle activation or physiological effects, scarring, stretching, oiliness, and sweating, among other things. (4) Facial capture systems ha

Claims

1. A computer-implemented method for performing appearance capture, the method comprising: determining one or more lighting patterns from a plurality of lighting patterns based on a comparison of one or more relighting losses associated with the plurality of lighting patterns; determining a first sequence of images captured by a first set of cameras, wherein the first sequence of images comprises a first set of images interleaved with a second set of images, and wherein the first set of images is captured during illumination of an object using a first lighting pattern and the second set of images is captured during illumination of the object using the one or more lighting patterns that are different from the first lighting pattern; and generating a first set of appearance parameters associated with the object based on a first inverse rendering and the first sequence of images. || 11. One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: receiving a plurality of lighting patterns based on a first set of images, wherein each lighting pattern in the plurality of lighting patterns represents illumination of a first object under a different combination of light sources included in a plurality of light sources, wherein each lighting pattern in the plurality of lighting patterns is associated with a relighting loss, wherein a first relighting loss associated with a first lighting pattern in the plurality of lighting patterns is determined based on a first set of appearance parameters for the first object generated using at least the first lighting pattern; determining a first set of lighting patterns from the plurality of lighting patterns based on a comparison of relighting losses associated with the plurality of lighting patterns; and generating a lighting pattern combination for an object type associated with the first object based on the first set of lighting patterns. || 20. A system, comprising: a memory that stores instructions, and a processor that is coupled to the memory and, when executing the instructions, is configured to: determining one or more lighting patterns from a plurality of lighting patterns based on one or more relighting losses associated with the plurality of lighting patterns; determining a first sequence of images captured by a set of cameras, wherein the first sequence of images comprises a first set of images interleaved with a second set of images, and wherein the first set of images is captured based on illumination of an object using a first lighting pattern and the second set of images is captured based on illumination of the object using the one or more lighting patterns that are different from the first lighting pattern; and generate a first set of appearance parameters associated with the object based on a first inverse rendering associated with the first sequence of images.