- Number
- 10217265
- Published
- 2019-02-26
- Filed
- 2016-07-07
- Assignee
- DISNEY ENTERPRISES, INC.
- Inventors
- Bérard; Pascal, Beeler; Thabo, Bradley; Derek, Gross; Markus
- CPC
- G06T19/20; G06T15/04
- Verdict
- Medium Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Generating parametric eye model (digital human).
Abstract
Systems and techniques for generating a parametric eye model of one or more eyes are provided. The systems and techniques may include obtaining eye data from an eye model database. The eye data includes eyeball data and iris data corresponding to a plurality of eyes. The systems and techniques may further include generating an eyeball model using the eyeball data. Generating the eyeball model includes establishing correspondences among the plurality of eyes. The systems and techniques may further include generating an iris model using the iris data. Generating the iris model includes sampling one or more patches of one or more of the plurality of eyes using an iris control map and merging the one or more patches into a synthesized texture. The systems and techniques may further include generating the parametric eye model that includes the eyeball model and the iris model.
Background
FIELD(1) The present disclosure generally relates to generating a parametric eye model. For example, one or more parts of an eye may be captured and reconstructed using the parametric model for use in rendering an animation of an eye.BACKGROUND(2) Capturing faces using three-dimensional (3D) scanning techniques can be used to build face models for video games, visual effects in films, medical applications, personalized figurines, among other applications. Much research has allowed facial capture technology to achieve a high level of quality, to a point that it is oftentimes difficult to distinguish the difference between digital faces and real faces. Oftentimes, face capture research focuses on the facial skin, ignoring other important characteristics. One example is the eyes, which are an important part of the face. Techniques for producing digital doubles in computer graphics tend to give little attention to details of a subject's eye, with far less attention than capturing the rest of the face. Some efforts to focus on the eye include time consuming and uncomfortable capture set-ups for the actors.SUMMARY(3) Techniques and systems are described for generating and using a parametric eye model for eye capture and reconstruction. The model is a deformable model that can be morphed to represent many or all possible shapes and colors that an eye can have. By being deformable, the model succinctly captures unique variations present across the different components of the eye. In
Claims
1. A computer-implemented method of generating a parametric eye model of one or more eyes, comprising: obtaining eye data from an eye model database, the eye data including eyeball data and iris data corresponding to a plurality of eyes; generating an eyeball model of a subject using the eyeball data from the eye model database, wherein generating the eyeball model includes iteratively establishing correspondences among the plurality of eyes and modifying the eyeball model based on the established correspondences, wherein establishing correspondences among the plurality of eyes includes: rigidly aligning each shape of each of the plurality of eyes with the eyeball model; fitting the eyeball model to each rigidly aligned shape of the plurality of eyes by identifying a limbus of the at least one eye in one or more input images, and minimizing a distance between a limbus of the eyeball model and the identified limbus of the at least one eye; generating non-rigidly deformed eyeball models by non-rigidly deforming the fitted eyeball model to each shape of each eye of the plurality of eyes; and updating the eyeball model using the non-rigidly deformed eyeball models; generating an iris model of the subject using the iris data from the eye model database and an iris control map, the iris control map including a low-resolution representation of an iris of the subject, wherein generating the iris model includes sampling a plurality of patches from the plurality of eyes and merging the plurality of patches into a high-resolution synthesized texture, and wherein sampling of patches from the plurality of eyes is constrained by the iris control map; and generating the parametric eye model of the subject, the parametric eye model including the eyeball model and the iris model.
9. A system for generating a parametric eye model of one or more eyes, comprising: a memory storing a plurality of instructions; and one or more processors configurable to: obtain eye data from an eye model database, the eye data including eyeball data and iris data corresponding to a plurality of eyes; generate an eyeball model of a subject using the eyeball data from the eye model database, wherein generating the eyeball model includes iteratively establishing correspondences among the plurality of eyes and modifying the eyeball model based on the established correspondences, wherein establishing correspondences among the plurality of eyes includes: rigidly aligning each shape of each of the plurality of eyes with the eyeball model; fitting the eyeball model to each rigidly aligned shape of the plurality of eyes by identifying a limbus of the at least one eye in one or more input images, and minimizing a distance between a limbus of the eyeball model and the identified limbus of the at least one eye; generating non-rigidly deformed eyeball models by non-rigidly deforming the fitted eyeball model to each shape of each eye of the plurality of eyes; and updating the eyeball model using the non-rigidly deformed eyeball models; generate an iris model of the subject using the iris data from the eye model database and an iris control map, the iris control map including a low-resolution representation of an iris of the subject, wherein generating the iris model includes sampling a plurality of patches from the plurality of eyes and merging the plurality of patches into a high-resolution synthesized texture, and wherein sampling of patches from the plurality of eyes is constrained by the iris control map; and generate the parametric eye model of the subject, the parametric eye model including the eyeball model and the iris model.
16. A non-transitory computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising: instructions that cause the one or more processors to obtain eye data from an eye model database, the eye data including eyeball data and iris data corresponding to a plurality of eyes; instructions that cause the one or more processors to generate an eyeball model of a subject using the eyeball data from the eye model database, wherein generating the eyeball model includes iteratively establishing correspondences among the plurality of eyes and modifying the eyeball model based on the established correspondences, wherein establishing correspondences among the plurality of eyes includes: rigidly aligning each shape of each of the plurality of eyes with the eyeball model; fitting the eyeball model to each rigidly aligned shape of the plurality of eyes by identifying a limbus of the at least one eye in one or more input images, and minimizing a distance between a limbus of the eyeball model and the identified limbus of the at least one eye; generating non-rigidly deformed eyeball models by non-rigidly deforming the fitted eyeball model to each shape of each eye of the plurality of eyes; and updating the eyeball model using the non-rigidly deformed eyeball models; instructions that cause the one or more processors to generate an iris model of the subject using the iris data from the eye model database and an iris control map, the iris control map including a low-resolution representation of an iris of the subject, wherein generating the iris model includes sampling a plurality of patches from the plurality of eyes and merging the plurality of patches into a high-resolution synthesized texture, and wherein sampling of patches from the plurality of eyes is constrained by the iris control map; and instructions that cause the one or more processors to generate the parametric eye model of the subject, the parametric eye model including the eyeball model and the iris model.