TECHNIQUES FOR FACIAL APPEARANCE CAPTURE

Number: 20220092293
Published: 2022-03-24
Filed: 2021-12-02
Assignee: DISNEY ENTERPRISES, INC.
Inventors: RIVIERE; Jeremy, URNAU GOTARDO; Paulo Fabiano, GHOSH; Abhijeet, BRADLEY; Derek Edward, BEELER; Dominik Thabo
CPC: G06V40/169; H04N23/56; H04N23/90; G06V40/166
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Facial appearance/material capture technique (VFX).

Abstract

Techniques are disclosed for capturing facial appearance properties. In some examples, a facial capture system includes light source(s) that produce linearly polarized light, at least one camera that is cross-polarized with respect to the polarization of light produced by the light source(s), and at least one other camera that is not cross-polarized with respect to the polarization of the light produced by the light source(s). Images captured by the cross-polarized camera(s) are used to determine facial appearance properties other than specular intensity, such as diffuse albedo, while images captured by the camera(s) that are not cross-polarized are used to determine facial appearance properties including specular intensity. In addition, a coarse-to-fine optimization procedure is disclosed for determining appearance and detailed geometry maps based on images captured by the cross-polarized camera(s) and the camera(s) that are not cross-polarized.

Background

BACKGROUND Technical Field

Embodiments of the present disclosure relate generally to computer science and computer graphics and, more specifically, to techniques for facial appearance capture. Description of the Related Art

Realistic digital faces are required for various computer graphics and computer vision applications. For example, digital faces are oftentimes used in virtual scenes of film or television productions and in video games.

A digital face can be synthesized using three-dimensional (3D) geometry of the face and two-dimensional (2D) appearance maps indicating reflectance properties of the face. As used herein, “appearance” refers to reflectance properties characterizing how light interacts with materials before reaching a camera. Example reflectance properties include, without limitation, shininess (also referred to as “specular intensity”) and the shape of a specular reflection lobe (also referred to as “roughness”), which are surface properties of skin, as well shading-free color (also referred to as “diffuse albedo”), which is mostly a subsurface property but can include contributions from the surface of skin as well. The reflectance properties of human skin, and more particularly the skin on a face, can vary due to skin type, tanning, blood flow caused by muscle activation or physiological effects, scarring, stretching, oiliness, and sweating, among other things.

Facial capture systems have been used to capture images of individua

Claims

1. A computer-implemented method for generating one or more maps indicating surface properties of an object, the computer-implemented method comprising: receiving a first set of images of the object captured using a first set of cameras, wherein the first set of cameras is configured to capture light that is cross-polarized with respect to a polarization of at least one light source; receiving a second set of images of the object captured using a second set of cameras, wherein the second set of cameras is configured to capture light that is unpolarized; and generating one or more appearance maps associated with the object based on the first set of images and the second set of images. || 11. A system, comprising: a set of light sources configured to generate light having a first polarization during a shot of an object; a first set of cameras configured to capture light that is cross-polarized with respect to the first polarization during the shot; and a second set of cameras configured to capture light that is not cross-polarized with respect to the first polarization during the shot. || 18. One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: receiving a first set of images of an object captured using a first set of cameras, wherein the first set of cameras is configured to capture light that is cross-polarized with respect to a polarization of at least one light source; receiving a second set of images of the object captured using a second set of cameras, wherein the second set of cameras is configured to capture light that is unpolarized; and generating one or more appearance maps associated with the object based on the first set of images and the second set of images.