- Number
- 10217275
- Published
- 2019-02-26
- Filed
- 2016-07-07
- Assignee
- DISNEY ENTERPRISES, INC.
- Inventors
- Bérard; Pascal, Beeler; Thabo, Bradley; Derek, Gross; Markus
- CPC
- G06T17/00
- Verdict
- Medium Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Eye reconstruction using parametric model (digital human).
Abstract
Systems and techniques for reconstructing one or more eyes using a parametric eye model are provided. The systems and techniques may include obtaining one or more input images that include at least one eye. The systems and techniques may further include obtaining a parametric eye model including an eyeball model and an iris model. The systems and techniques may further include determining parameters of the parametric eye model from the one or more input images. The parameters can be determined to fit the parametric eye model to the at least one eye in the one or more input images. The parameters include a control map used by the iris model to synthesize an iris of the at least one eye. The systems and techniques may further include reconstructing the at least one eye using the parametric eye model with the determined parameters.
Background
FIELD(1) The present disclosure generally relates to the capture and reconstruction of eyes for animation. For example, one or more parts of an eye may be captured and reconstructed using a parametric model for use in rendering an animation of the eye.BACKGROUND(2) Capturing faces using three-dimensional (3D) scanning techniques can be used to build face models for video games, visual effects in films, medical applications, personalized figurines, among other applications. Much research has allowed facial capture technology to achieve a high level of quality, to a point that it is oftentimes difficult to distinguish the difference between digital faces and real faces. Oftentimes, face capture research focuses on the facial skin, ignoring other important characteristics. One example is the eyes, which are an important part of the face. Techniques for producing digital doubles in computer graphics tend to give little attention to details of a subject's eye, with far less attention than capturing the rest of the face. Some efforts to focus on the eye include time consuming and uncomfortable capture set-ups for the actors.SUMMARY(3) Techniques and systems are described for generating and using a parametric eye model for eye capture and reconstruction. The model is a deformable model that can be morphed to represent many or all possible shapes and colors that an eye can have. By being deformable, the model succinctly captures unique variations present across the different componen
Claims
1. A computer-implemented method of reconstructing one or more eyes, comprising: obtaining one or more input images, the one or more input images including at least one eye, wherein the one or more input images include a three-dimensional face scan of at least a portion of a face including the at least one eye, the three-dimensional face scan being from a multi-view scanner; obtaining a parametric eye model, the parametric eye model including an eyeball model and an iris model; determining a geometry for the eyeball model, the geometry being determined by fitting the eyeball model to the at least one eye in the one or more input images, wherein the geometry corresponds to a shape of the eyeball of the at least one eye; determining an iris control map used by the iris model to synthesize an iris of the at least one eye, the iris control map being determined by generating a proxy iris centered on a pupil determined for the eyeball model and determining texture information for the proxy iris from at least one of the one or more input images; determining a vein network for the at least one eye; and reconstructing the at least one eye using the parametric eye model with the geometry and the iris control map, wherein reconstructing the at least one eye includes rendering the vein network onto a sclera backplate determined from the one or more images.
6. A system for reconstructing one or more eyes, comprising: a memory storing a plurality of instructions; and one or more processors configurable to: obtain one or more input images, the one or more input images including at least one eye, wherein the one or more input images include a three-dimensional face scan of at least a portion of a face including the at least one eye, the three-dimensional face scan being from a multi-view scanner; obtain a parametric eye model, the parametric eye model including an eyeball model and an iris model; determine a geometry for the eyeball model, the geometry being determined by fitting the eyeball model to the at least one eye in the one or more input images, wherein the geometry corresponds to a shape of the eyeball of the at least one eye; determine an iris control map used by the iris model to synthesize an iris of the at least one eye, the iris control map being determined by generating a proxy iris centered on a pupil determined for the eyeball model and determining texture information for the proxy iris from at least one of the one or more input images; determining a vein network for the at least one eye; and reconstruct the at least one eye using the parametric eye model with the geometry and the iris control map, wherein reconstructing the at least one eye includes rendering the vein network onto a sclera backplate determined from the one or more images.
11. A non-transitory computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising: instructions that cause the one or more processors to obtain one or more input images, the one or more input images including at least one eye, wherein the one or more input images include a three-dimensional face scan of at least a portion of a face including the at least one eye, the three-dimensional face scan being from a multi-view scanner; instructions that cause the one or more processors to obtain a parametric eye model, the parametric eye model including an eyeball model and an iris model; instructions that cause the one or more processors to determine a geometry for the eyeball model, the geometry being determined by fitting the eyeball model to the at least one eye in the one or more input images, wherein the geometry corresponds to a shape of the eyeball of the at least one eye; instructions that cause the one or more processors to determine an iris control map used by the iris model to synthesize an iris of the at least one eye, the iris control map being determined by generating a proxy iris centered on a pupil determined for the eyeball model and determining texture information for the proxy iris from at least one of the one or more input images; instructions that cause the one or more processor to determine a vein network for the at least one eye; and instructions that cause the one or more processors to reconstruct the at least one eye using the parametric eye model with the geometry and the iris control map, wherein reconstructing the at least one eye includes rendering the vein network onto a sclera backplate determined from the one or more images.