Detailed spatio-temporal reconstruction of eyelids

Number: 9691165
Published: 2017-06-27
Filed: 2015-07-24
Assignee: Disney Enterprises, Inc.
Inventors: Bermano; Amit, Beeler; Thabo, Bradley; Derek, Bickel; Bernd, Gross; Markus, Kozlov; Yeara
CPC: G06T13/40; G06T7/269; G06V40/193; G06V40/19; G06T19/20; G06T17/20
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Reconstructs detailed spatio-temporal eyelid geometry from images for realistic facial animation.

Abstract

Methods and systems of reconstructing an eyelid are provided. A method of reconstructing an eyelid includes obtaining one or more images of the eyelid, generating one or more image input data for the one or more images of the eyelid, generating one or more reconstruction data for the one or more images of the eyelid, and reconstructing a spatio-temporal digital representation of the eyelid using the one or more input image data and the one or more reconstruction data.

Background

BRIEF DESCRIPTION OF DRAWINGS(1) The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.(2) The disclosure will be readily understood by the following detailed description in conjunction with the accompanying drawings, wherein like reference numerals designate like elements, and in which:(3) FIG. 1 illustrates an example of human eyelids.(4) FIG. 2 illustrates an overview of a systemfor reconstructing an eyelid, in accordance with an exemplary embodiment.(5) FIG. 3 illustrates an example configuration of a system for obtaining one or more images, in accordance with an exemplary embodiment.(6) FIG. 4 illustrates a plurality of images which are obtained from an exemplary system, in accordance with an exemplary embodiment.(7) FIG. 5 illustrates a method of performing inpainting, in accordance with an exemplary embodiment.(8) FIG. 6 illustrates an example of creating a template mesh for an eyelid, in accordance withan exemplary embodiment.(9) FIG. 7 illustrates a method of creating a wrinkle probabilitymap, in accordance with an exemplary embodiment.(10) FIG. 8 illustrates optical flow of an eyelid, in accordance with an exemplary embodiment.(11) FIG. 9 illustrates a method of correcting optical flow, in accordance with an exemplary embodiment.(12) FIG. 10 illustrates a flowchart of a method for corr

Claims

1. A computer-implemented method of reconstructing an eyelid comprising: obtaining one or more images of the eyelid; generating one or more image input data for the one or more images of the eyelid; generating one or more reconstruction data for the one or more images of the eyelid, the one or more reconstruction data including one or more eyelid meshes; and reconstructing a spatio-temporal digital representation of the eyelid using theone or more input image data and the one or more reconstruction data, wherein reconstructing the spatio-temporal digital representation of the eyelid includes deforming the one ormore eyelid meshes using the one or more image input data. 12. A system for reconstructing an eyelid comprising: a memory storing a plurality of instructions; and one or more processors configured to: obtain one or more images of the eyelid; generate one or more image input data for the one or more images of the eyelid; generate one or more reconstruction data for the one or more images of the eyelid, the one or more reconstruction data including one or more eyelid meshes; and reconstruct a spatio-temporal digital representation of the eyelid using the one or more input image data and the one or more reconstruction data, wherein reconstructing the spatio-temporal digital representation of the eyelid includes deforming the one or more eyelid meshes using the one or more image input data. 19. A computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising: instructions that cause the one or more processors to obtain one or more images of the eyelid; instructions that cause the one or moreprocessors to generate one or more image input data for the one or more images of the eyelid; instructions that cause the one or more processors to generate one or more reconstruction data for the one or more images of the eyelid, the one or more reconstruction data including one or more eyelid meshes; and instructions that cause the one or more processors to reconstruct a spatio-temporal digital representation of the eyelid using the one or more input image data and the one or more reconstruction data, wherein reconstructing the spatio-temporal digital representation of the eyelid includes deforming the one or more eyelid meshes using the one or more image input data.