Lucasfilm lip-synchronization technique transferring performance between animation rigs.
In some embodiments a method of transferring facial expressions from a subject to a computer-generated character is provided where the method includes receiving positional information from a motion capture session of the subject representing a performance having facial expressions to be transferred to the computer-generated character, receiving a first animation model that represents the subject, and receiving a second animation model that represents the computer-generated character. Each of the first and second animation models can include a plurality of adjustable controls that define geometries of the model and that can be adjusted to present different facial expressions on the model, and where the first and second animation models are designed so that setting the same values for the same set of adjustable controls in each model generates a similar facial poses on the models. The method further includes determining a solution, including values for at least some of the plurality of controls, that matches the first animation model to the positional information to reproduce the facial expressions from the performance to the first animation model, retargeting the facial expressions from the performance to the second animation model using the solution; and thereafter, synchronizing lip movement of the second animation model with lip movement from the first animation model.
BRIEF DESCRIPTION OF THE DRAWINGS
Illustrative embodiments of the present invention are described in detail below with reference to the following drawing figures:
FIG. 1 is a block diagram of an animation system with which the facial performance of a subject can be captured and re-targeted to an animated character according to some embodiments of the invention;
FIG. 2 is a flow diagram depicting a method according to some embodiments of the invention;
FIG. 3 illustrates the blocks performed in accordance with a lip synchronization process according to some embodiments of the invention;
FIG. 4 is a flow diagram depicting a process for matching the relative amount of visible teeth in a creature rigto the relative amount of visible teeth in an actor rig according to some embodiments of the invention;
FIG. 5 is a flow diagram depicting a process for determining the relative amount of visible teeth in a rig, such as an actor rig or a creature rig, according to some embodiments of the invention;
FIG. 6A is a simplified representation of a computer-generated character, such as an actor rig or a creature rig;
FIG. 6B is a simplified, expanded view of a portion within dotted line A of the simplified representation of a computer-generated character shown in FIG. 6A;
FIG. 7 is a simplified flow diagram depicting a process for matching the jaw opening in a creature rig to the jaw opening in an actor rig according to some embodiments
1. A method of transferring facial expressions from a subject to a computer-generated character, the method comprising: receiving positional information from a motion capture session of the subject representing a performance having facial expressions to be transferred to the computer-generated character; receiving a first animation model that represents the subject and a second animation model that represents the computer-generated character, each of the first and second animation models including a plurality of adjustable controls that define geometries of the model and that can be adjusted to present different facial expressions on the model, wherein setting the same values for corresponding sets of adjustable controls in the models generates a similar facial poses on the models; determining a solution that matches the first animation model to the positional information to reproduce the facial expressions from the performance to the first animation model, the solution including values for at least some of the plurality of adjustable controls; retargetingthe facial expressions from the performance to the second animation model using the solution; and thereafter, synchronizing lip movement of the second animation model with lip movement from the first animation model.
16. A system for transferring facial expressions from a subject to a computer-generated character, the system comprising: a memory storing a plurality of instructions; and one or more processors configurable to: receive positional information from a motion capture session of the subject representing a performance having facial expressions to be transferred to the computer-generated character; receive a first animation model that represents the subject and a second animation model that represents the computer-generated character, each of the first and second animation models including aplurality of adjustable controls that define geometries of the model and that can be adjusted to present different facial expressions on the model, wherein setting the same valuesfor corresponding sets of adjustable controls in the models generates similar facial poses on the models; determine a solution that matches the first animation model to the positional information to reproduce the facial expressions from the performance to the first animation model, the solution including values for at least some of the plurality of adjustable controls; retarget the facial expressions from the performance to the second animation model using the solution; and thereafter, synchronize lip movement of the second animationmodel with lip movement from the first animation model.
19. A computer-readable memory storing a plurality of instructions executable by one or more processors for transferring facial expressions from a subject to a computer-generated character, the plurality of instructions comprising: instructions that cause the one or more processors to receive positional information from a motion capture session of the subject representing a performance having facial expressions to be transferred to the computer-generated character; instructions that cause the one or more processors to receive a first animation model that represents the subject and a second animation model that represents the computer-generated character, each of the first and second animation models including a plurality of adjustable controlsthat define geometries of the animation model and that can be adjusted to represent different facial expressions on the model, wherein setting the same values for corresponding sets of adjustable controls in the models generates similar facial poses on the models; instructions that cause the one or more processors to determine a solution that matches the animation model to the positional information to reproduce the facial expressions from the performance to an first animation model that represents the subject using the positional information and animation model, the solution including values for at least some of the plurality of controls; instructions that cause the one or more processors to retarget the facial expressions from the performance to the second animation model using the solution; and instructions that cause the one or more processors to synchronize lip movement of the creature with lip movement from the first animation model after the facial expressions are retargeted from the performance to the second animation model.