Object appearance/material capture technique (VFX, pre-grant).
Techniques are disclosed for capturing facial appearance properties. In some examples, a facial capture system includes light source(s) that produce linearly polarized light, at least one camera that is cross-polarized with respect to the polarization of light produced by the light source(s), and at least one other camera that is not cross-polarized with respect to the polarization of the light produced by the light source(s). Images captured by the cross-polarized camera(s) are used to determine facial appearance properties other than specular intensity, such as diffuse albedo, while images captured by the camera(s) that are not cross-polarized are used to determine facial appearance properties including specular intensity. In addition, a coarse-to-fine optimization procedure is disclosed for determining appearance and detailed geometry maps based on images captured by the cross-polarized camera(s) and the camera(s) that are not cross-polarized.
BACKGROUND Technical Field
Embodiments of the present disclosure relate generally to computer science and computer graphics and, more specifically, to techniques for facial appearance capture. Description of the Related Art
Realistic digital faces are required for various computer graphics and computer vision applications. For example, digital faces are oftentimes used in virtual scenes of film or television productions and in video games.
A digital face can be synthesized using three-dimensional (3D) geometry of the face and two-dimensional (2D) appearance maps indicating reflectance properties of the face. As used herein, “appearance” refers to reflectance properties characterizing how light interacts with materials before reaching a camera. Example reflectance properties include, without limitation, shininess (also referred to as “specular intensity”) and the shape of a specular reflection lobe (also referred to as “roughness”), which are surface properties of skin, as well shading-free color (also referred to as “diffuse albedo”), which is mostly a subsurface property but can include contributions from the surface of skin as well. The reflectance properties of human skin, and more particularly the skin on a face, can vary due to skin type, tanning, blood flow caused by muscle activation or physiological effects, scarring, stretching, oiliness, and sweating, among other things.
Facial capture systems have been used to capture images of individua
1. A computer-implemented method for generating one or more maps indicating surface properties of an object, the method comprising: receiving a first set of one or more images of the object captured using a first set of cameras that are cross-polarized with respect to a polarization of at least one light source; receiving a second set of one or more images of the object captured using at one a second set of cameras that are not cross-polarized with respect to the polarization of the at least one light source; receiving a three-dimensional (3D) geometry associated with the object; calibrating the first set of cameras using one camera that is included in the first set of cameras; calibrating the second set of second cameras using a rendering of the object as a calibration target; determining initial appearance maps associated with the object based on the calibrated first set of cameras and the calibrated second set of cameras; generating at least one of an appearance map or a geometry map associated with the object based on the initial appearance maps, the first set of one or more images, the second set of one or more images, and the 3D geometry. ||
11. A system, comprising: at least one light source configured to generate light that is polarized; a first set of cameras configured to capture light that is cross-polarized with respect to a polarization of the light generated by the at least one light source; a second set of cameras configured to capture light that is not cross-polarized with respect to the polarization of the light generated by the at least one light source; and an appearance capture application that, during operation: receives a first set of one or more images of an object captured using the first set of cameras; receives a second set of one or more images of the object captured using the second set of cameras; receives a three-dimensional (3D) geometry associated with the object; calibrates the first set of cameras using one camera that is included in the first set of cameras; calibrates the second set of cameras using a rendering of the object as a calibration target; determines initial appearance maps associated with the object based on the calibrated first set of cameras and the calibrated second set of cameras; generates at least one of an appearance map or a geometry map associated with the object based on the initial appearance maps, the first set of one or more images, the second set of one or more images, and the 3D geometry. ||
18. One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to perform steps for generating one or more maps indicating surface properties of an object, the steps comprising: receiving a first set of one or more images of the object captured using a first set of cameras that are cross-polarized with respect to a polarization of at least one light source; receiving a second set of one or more images of the object captured using a second set of cameras that are not cross-polarized with respect to the polarization of the at least one light source; receiving a three-dimensional (3D) geometry associated with the object; calibrating the first set of cameras using one camera that is included in the first set of cameras; calibrating the second set of second cameras using a rendering of the object as a calibration target; determining initial appearance maps associated with the object based on the calibrated first set of cameras and the calibrated second set of cameras; generating at least one of an appearance map or a geometry map associated with the object based on the initial appearance maps, the first set of one or more images, the second set of one or more images, and the 3D geometry.