- Number
- 10489958
- Published
- 2019-11-26
- Filed
- 2017-08-01
- Assignee
- LUCASFILM ENTERTAINMENT COMPANY LTD.
- Inventors
- Mallet; Ronald, Ye; Yuting, Koperwas; Michael, Goldenthal; Adrian R., Bhat; Kiran S.
- CPC
- G06T13/40; G06T19/20
- Verdict
- Medium Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Facial animation models (character animation).
Abstract
A system includes a computing device that includes a memory configured to store instructions. The system also includes a processor configured to execute the instructions to perform a method that includes receiving multiple representations of an object. Each of the representations includes position information of the object and corresponds to an instance in time. For at least one of the representations, the method includes defining a contour that represents a movable silhouette of a surface feature of the object. The method also includes producing a deformable model of the surface of the object from the defined contour and from the at least one representation of the object.
Background
TECHNICAL FIELD(1) This document relates to a system for producing animation models such as facial animation models.BACKGROUND(2) Ever increasing computational power allows complex calculations to be tackled with increased efficiency in less time. For example, robust computer systems can execute applications to simulate objects such as fictional characters and even realistic-looking human forms. To visually present such seemingly realistic characters, a significant amount of detail may be included in the simulated objects (e.g., fine wrinkles on the forehead of a character, laugh lines, etc.). However, even with such detail included, the generated simulations may not appear completely realistic and may lack subtle nuances that even a casual viewer may find glaringly apparent and distracting from their viewing experience.SUMMARY(3) The described systems and techniques are for improving the production of models (e.g., for use in creating an actor's or fictional character's face, etc.) to more closely match the performance of an actor. Along with using basic facial geometry and motion information captured from the performing actor's face, perceptually important contour features of the actor's face are also used for model production. Additionally, by adjusting the size and shape of a mouth included in a model to better match the actor's mouth, the lips represented by the model may better track the dialogue performed by the actor and improve the overall viewing experience.(4) In o
Claims
1. A computer-implemented method, the method being implemented by one or more of a processor configured to execute machine-readable instructions, the method comprising: receiving multiple images of a face of a person, wherein each of the images include position information indicating positions of a set of features on the face of the person and corresponds to an instance in time, wherein the set of features include a first curve in an eye region of the person; for each of the multiple images, defining, based on the position information, a silhouette contour that corresponds to a movable boundary between a visual portion and a hidden portion of an eye of the person, wherein the silhouette contour is defined such that the silhouette contour represents the first curve; producing a deformable model of a surface of the face based on at least one image of the multiple images of the face and the silhouette contour; defining separation thresholds for a portion of the deformable model; generating one or more corrective shapes based on the silhouette contour and correspondence between the silhouette contour and the first curve defined in the images; and correcting the eye region on the deformable model corresponding to the silhouette contour by applying the one or more corrective shapes to the deformable model wherein applying the corrective shapes to the deformable model includes adjusting an eyelid shape or a corner of the eye region of the deformable model based on the separation thresholds.
2. The computer-implemented method of claim 1, wherein producing the deformable model includes defining an association between a first contour that represents the silhouette contour of the eye region and a mesh of vertices that represents the face.
3. The computer-implemented method of claim 1, wherein the silhouette contour further corresponds to a second curve on the face, and wherein the one or more corrective shapes are generated further based on correspondence between silhouette contour and the second curve.
4. The computer-implemented method of claim 1, wherein correcting on the deformable model applying the one or more corrective shapes to the deformable model includes adjusting a geometry of the eye of the person.
5. The computer-implemented method of claim 4, wherein adjusting the geometry of the eye of the person includes closing the portion of the eye of the person.
6. The computer-implemented method of claim 5, wherein adjusting the geometry of the eye of the person includes defining a separation threshold for closing the portion of the eye of the person.
7. A system comprising: a computing device comprising: a memory configured to store instructions; and a processor configured to execute the instructions to perform a method comprising: receiving multiple images of a face of a person, wherein each of the images include position information indicating positions of a set of features on the face of the person and corresponds to an instance in time, wherein the set of features include a first curve in an eye region of the person; for each of the multiple images, defining, based on the position information, a silhouette contour that corresponds to a movable boundary between a visual portion and a hidden portion of an eye of the person, wherein the silhouette contour is defined such that the silhouette contour represents the first curve; producing a deformable model of a surface of the face based on at least one image of the multiple images of the face and the silhouette contour; defining separation thresholds for a portion of the deformable model; generating one or more corrective shapes based on the silhouette contour and correspondence between the silhouette contour and the first curve defined in the images; and correcting the eye region on the deformable model corresponding to the silhouette contour by applying the one or more corrective shapes to the deformable model, wherein applying the corrective shapes to the deformable models to adjust the eye region of the deformable model includes adjusting an eyelid shape or a corner of the eye region based on the separation thresholds.
8. The system of claim 7, wherein producing the deformable model includes defining an association between a first contour that represents the silhouette contour of the eye region and a mesh of vertices that represents the face.
9. The system of claim 7, wherein the silhouette contour further corresponds to a second curve on the face, and wherein the one or more corrective shapes are generated further based on correspondence between silhouette contour and the second curve.
10. The system of claim 7, wherein the processor is further configured to perform: producing an updated deformable model of the surface of the face from the produced deformable model and another contour defined from another one of the received multiple images of the face.
11. The system of claim 7, wherein correcting the eye region on the deformable model applying the one or more corrective shapes to the deformable model includes adjusting a geometry of the eye of the person.
12. The system of claim 11, wherein adjusting the geometry of the eye of the person includes closing the portion of the eye of the person.
13. The system of claim 12, wherein adjusting the geometry of the eye of the person includes defining a separation threshold for closing the portion of the eye of the person.
14. A non-transitory computer readable medium having executable instructions stored thereon that when executed by a processor perform a method comprising: receiving multiple images of a face of a person, wherein each of the images include position information indicating positions of a set of features on the face of the person and corresponds to an instance in time, wherein the set of features include a first curve in an eye region of the person; for each of the multiple images, defining, based on the position information, a silhouette contour that corresponds to a movable boundary between a visual portion and a hidden portion of an eye of the person, wherein the silhouette contour is defined such that the silhouette contour represents the first curve; producing a deformable model of a surface of the face based on at least one image of the multiple images of the face and the silhouette contour; defining separation thresholds for a portion of the deformable model; generating one or more corrective shapes based on the silhouette contour and correspondence between the silhouette contour and the first curve defined in the images; and correcting the eye region on the deformable model corresponding to the silhouette contour by applying the one or more corrective shapes to the deformable model, wherein applying the corrective shapes to the deformable models to adjust the eye region of the deformable model includes adjusting an eyelid shape or a corner of the eye region based on the separation thresholds.
15. The non-transitory computer readable medium of claim 14, wherein producing the deformable model includes defining an association between a first contour that represents the silhouette contour of the eye region and a mesh of vertices that represents the face.
16. The non-transitory computer readable medium of claim 14, wherein the silhouette contour further corresponds to a second curve on the face, and wherein the one or more corrective shapes are generated further based on correspondence between silhouette contour and the second curve.
17. The non-transitory computer readable medium of claim 14, having executable instructions stored thereon executable instructions stored thereon that when executed by a processor perform a method further comprising: producing an updated deformable model of the surface of the face from the produced deformable model and another contour defined from another one of the received multiple images of the face.
18. The non-transitory computer readable medium of claim 14, wherein correcting the eye region on the deformable model applying the one or more corrective shapes to the deformable model includes adjusting a geometry of the eye of the person.
19. The non-transitory computer readable medium of claim 18, wherein adjusting the geometry of the eye of the person includes closing the portion of the eye of the person based on a separation threshold.