Photometric three-dimensional facial capture and relighting

Number: 9883167
Published: 2018-01-30
Filed: 2015-09-25
Assignee: Disney Enterprises, Inc.; Carnegie Mellon University
Inventors: Matthews; Iain; Gotardo; Paulo Urnau; Simon; Tomas; Sheikh; Yaser
CPC: G06T7/586; H04N23/56; H04N13/254; H04N13/239; G06V40/166; G06V40/16; H04N13/257; H04N13/246; G06V10/60
Verdict: High Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Photometric 3D facial capture and relighting.

Abstract

Embodiments disclosed herein relate to a method and apparatus for generating a three-dimensional surface. In one embodiment, there is a method for generating a three-dimensional surface. The method includes capturing a plurality of images of a target object with at least two cameras, the target object illuminated by at least two sets of red-green-blue (RGB) lights positioned in an array about the target object and generating a three-dimensional surface of the target object by iteratively reconstructing a surface estimate of the target object and aligning images of the target object using motion estimation until the images converge, wherein the images are processed in n-frame intervals.

Background

BACKGROUND(1) High-resolution geometry and appearances are invaluable assets in the movie and video game industries. A geometry generally refers to an (three-dimensional) object's structure, i.e., the triangles, lines, or points that compose the object, while an appearance generally refers to the object's shading, including its vertex and fragment shaders, and render state. The quality of a three-dimensional model can make or break the perceived realism of an animation. When it comes to facial models, people have a remarkably low threshold for inaccuracies. Even the smallest details in geometry and appearance are important. A variety of computer vision techniques have been developed for reconstructing high-detail geometry and appearances of real objects. However, each technique for constructing 3D models has its own particular limitations. Even when combined as independent processing stages, the combination of techniques can exacerbate errors introduced by early regularization, such as a smoothness constraint introduced by the early regularization.SUMMARY(2) According to one embodiment, a method for generating a three-dimensional surface is disclosed herein. The method includes capturing a plurality of images of a target object with at least two cameras, the target object illuminated by at least three sets of red-green-blue (RGB) lights positioned in an array about the target object and generating a three-dimensional surface of the target object by iteratively reconstructing

Claims

1. A method for generating a three-dimensional surface, comprising: capturing a plurality of images of a target object with at least two cameras, the target object illuminated by at least two sets of red-green-blue (RGB) lights positioned in an array about the target object; and generating a three-dimensional surface of the target object by iteratively reconstructing a surface estimate of the target object and aligning images of the target object using motion estimation until the images converge, wherein the images are processed in n-frame intervals. 8. A method for generating a three-dimensional surface, comprising: capturing an image of a target object with two cameras, one camera positioned to a left of the target object, one camera position to a right of the target object, wherein the target object is illuminated by three sets of red-green-blue (RGB) lights positioned in an array about the target object; and processing the image in three frame intervals, wherein processing the image comprises: recovering an unknown surface at each frame within a three-frame interval; and recovering a backward motion and forward motion within the three-frame interval. 15. A system for reconstructing a three-dimensional surface, comprising: at least two cameras, wherein a first camera of the at least two cameras is positioned to capture a right image of a target object, and wherein a second camera is positioned to capture a left image of the target object; at least three sets of red-green-blue (RGB) lights positioned in an array about the target object; and a controller configured to generate a three-dimensional surface of the target object by iteratively reconstructing a surface estimate of the target object and aligning images of the target object using motion estimation until the images converge, wherein the images are processed in three-frame intervals.