PHOTOMETRIC THREE-DIMENSIONAL FACIAL CAPTURE AND RELIGHTING

Number: 20170094253
Published: 2017-03-30
Filed: 2015-09-25
Assignee: Disney Enterprises, Inc.
Inventors: MATTHEWS; Iain et al.
CPC: G06V40/166; G06V10/60; H04N13/257; G06V40/16; H04N13/239; G06T7/586; H04N13/254; H04N23/56; H04N13/246
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Photometric 3D facial capture and relighting technique for realistic digital face reconstruction.

Abstract

Embodiments disclosed herein relate to a method and apparatus for generating a three-dimensional surface. In one embodiment, there is a method for generating a three-dimensional surface. The method includes capturing a plurality of images of a target objectwith at least two cameras, the target object illuminated by at least two sets of red-green-blue (RGB) lights positioned in an array about the target object and generating a three-dimensional surface of the target object by iteratively reconstructing a surface estimate of the target object and aligning images of the target object using motion estimation until the images converge, wherein the images are processed in n-frame intervals.

Background

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a block diagram depicting a system for acquiring image data of an object and generating a three-dimensional geometry and appearance of that object, according to one embodiment of the present disclosure.

FIG. 2 is a flow diagram illustrating a method for generating a three-dimensional geometry and appearance, according to one embodiment.

FIG. 3 is a block diagramillustrating a three flame sliding window of images taken from a left camera and a right camera, according to one embodiment.

FIG. 4 is a flow diagram illustrating a method for generating a three-dimensional surface, according to one embodiment of a present disclosure.

FIG. 5 is a block diagram illustrating an example computing system configuredaccording to an embodiment.DETAILED DESCRIPTION

A variety of computer vision techniques have been developed for reconstructing high-detail geometry and appearances of real objects. Photometric stereo is one technique for estimating surface normal of objects by observing the objects under different illumination conditions. Multi-view stereo is another technique that reconstructs a 3D object model from a collection of images taken from knowncamera viewpoints. Photometric stereo is generally used to enhance the detail of an initial low-resolution geometry, most often obtained via the multi-view stereo technique. Although previously computed independently, photometric stereo and m

Claims

1. A method for generating a three-dimensional surface, comprising: capturing a plurality of images of a target object with at least two cameras, the target object illuminated by atleast two sets of red-green-blue (RGB) lights positioned in an array about the target object; and generating a three-dimensional surface of the target object by iteratively reconstructing a surface estimate of the target object and aligning images of the target object using motion estimation until the images converge, wherein the images are processed in n-frame intervals. 8. A method for generating a three-dimensional surface, comprising: capturing an image of a target object with two cameras, one camera positioned to a left of the target object, one camera position to a right of the target object, wherein the target object is illuminated by three sets of red-green-blue (RGB) lights positioned in an array about the target object; and processing the image in three frame intervals, wherein processingthe image comprises: recovering an unknown surface at each frame within the three-frame interval; and recovering a backward motion and forward motion within the three-frame interval. 15. A system for reconstructing a three-dimensional surface, comprising: at least two cameras, wherein a first camera of the at least two cameras is positioned to capture a right image of the target object, and wherein a second camera is positioned to capture a leftimage of the target object; at least three sets of red-green-blue (RGB) lights positionedin an array about the target object; and a controller configured to generate a three-dimensional surface of the target object by iteratively reconstructing a surface estimate of the target object and aligning images of the target object using motion estimation until the images converge, wherein the images are processed in three-frame intervals.