Animation rig lip-synchronization.
In some embodiments a method of transferring facial expressions from a subject to a computer-generated character is provided where the method includes receiving positional information from a motion capture session of the subject representing a performance having facial expressions to be transferred to the computer-generated character, receiving a first animation model that represents the subject, and receiving a second animation model that represents the computer-generated character. Each of the first and second animation models can include a plurality of adjustable controls that define geometries of the model and that can be adjusted to present different facial expressions on the model, and where the first and second animation models are designed so that setting the same values for the same set of adjustable controls in each model generates a similar facial poses on the models. The method further includes determining a solution, including values for at least some of the plurality of controls, that matches the first animation model to the positional information to reproduce the facial expressions from the performance to the first animation model, retargeting the facial expressions from the performance to the second animation model using the solution; and thereafter, synchronizing lip movement of the second animation model with lip movement from the first animation model.
BACKGROUND OF THE INVENTION(1) The present disclosure relates generally to performance capture, and more specifically to methods and techniques and systems for transferring facial expressions from an actor to a computer-generated character.(2) Facial expression transfer is the act of adapting the facial expressions of an actor to a computer-generated (CG) target character. The expression transfer process plays an important role in transferring subtle facial expressions of the actor to the CG character giving the CG character natural, life-like expressions. Accurately transferring the nuances of an actor's facial expressions, including accurately transferring lip movement, to the CG character can be a complex and difficult process. Embodiments of the invention address these and other problems both individually and collectively.BRIEF SUMMARY OF THE INVENTION(3) Techniques for transferring facial expressions from an actor to a computer-generated (CG) character are provided herein. One or more embodiments described herein include techniques for accurately transferring lip and jaw movements of an actor (sometimes referred to herein as a “subject”) to a CG character to produce believable, robust, and editable animations in an automated manner. In some examples, one or more techniques may be provided for improving the synchronization of lip and jaw movements of a CG character to the lip and jaw movements of the subject.(4) Some embodiments provide a method of transferring facial exp
1. A method of transferring facial expressions from a subject to a computer-generated character, the method comprising: receiving positional information from a motion capture session of the subject representing a performance having facial expressions to be transferred to the computer-generated character; receiving a first animation model that represents the subject and a second animation model that represents the computer-generated character, each of the first and second animation models including a plurality of adjustable controls that define geometries of the model and that can be adjusted to present different facial expressions on the model, wherein setting the same values for corresponding sets of adjustable controls in the models generates a similar facial poses on the models; determining a solution that matches the first animation model to the positional information to reproduce the facial expressions from the performance to the first animation model, the solution including values for at least some of the plurality of adjustable controls; retargeting the facial expressions from the performance to the second animation model using the solution; and thereafter, synchronizing lip movement of the second animation model with lip movement from the first animation model.
16. A system for transferring facial expressions from a subject to a computer-generated character, the system comprising: a memory storing a plurality of instructions; and one or more processors configurable to: receive positional information from a motion capture session of the subject representing a performance having facial expressions to be transferred to the computer-generated character; receive a first animation model that represents the subject and a second animation model that represents the computer-generated character, each of the first and second animation models including a plurality of adjustable controls that define geometries of the model and that can be adjusted to present different facial expressions on the model, wherein setting the same values for corresponding sets of adjustable controls in the models generates similar facial poses on the models; determine a solution that matches the first animation model to the positional information to reproduce the facial expressions from the performance to the first animation model, the solution including values for at least some of the plurality of adjustable controls; retarget the facial expressions from the performance to the second animation model using the solution; and thereafter, synchronize lip movement of the second animation model with lip movement from the first animation model.
19. A computer-readable memory storing a plurality of instructions executable by one or more processors for transferring facial expressions from a subject to a computer-generated character, the plurality of instructions comprising: instructions that cause the one or more processors to receive positional information from a motion capture session of the subject representing a performance having facial expressions to be transferred to the computer-generated character; instructions that cause the one or more processors to receive a first animation model that represents the subject and a second animation model that represents the computer-generated character, each of the first and second animation models including a plurality of adjustable controls that define geometries of the animation model and that can be adjusted to represent different facial expressions on the model, wherein setting the same values for corresponding sets of adjustable controls in the models generates similar facial poses on the models; instructions that cause the one or more processors to determine a solution that matches the animation model to the positional information to reproduce the facial expressions from the performance to an first animation model that represents the subject using the positional information and animation model, the solution including values for at least some of the plurality of controls; instructions that cause the one or more processors to retarget the facial expressions from the performance to the second animation model using the solution; and instructions that cause the one or more processors to synchronize lip movement of the creature with lip movement from the first animation model after the facial expressions are retargeted from the performance to the second animation model.