- Number
- 9600742
- Published
- 2017-03-21
- Filed
- 2015-05-05
- Assignee
- LUCASFILM ENTERTAINMENT COMPANY LTD.
- Inventors
- Bhat; Kiran et al.
- CPC
- G06V40/176; G06T13/80; G06T13/40; G06T7/75; G06F18/22; G06T7/60
- Verdict
- Medium Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Lucasfilm performance-capture control system (same family as gate-verified patent #7) — core VFX pipeline IP.
Abstract
Performance capture systemsand techniques are provided for capturing a performance of a subject and reproducing an animated performance that tracks the subject's performance. For example, systems and techniques are provided for determining control values for controlling an animation model to define features of a computer-generated representation of a subject based on the performance.A method may include obtaining input data corresponding to a pose performed by the subject, the input data including position information defining positions on a face of the subject. The method may further include obtaining an animation model for the subject that includes adjustable controls that control the animation model to define facial features of the computer-generated representation of the face, and matching one or more of the positions on the face with one or more corresponding positions on the animation model. The matching includes using an objective function to project an error onto a control space of the animation model. The method may further include determining, using the projected error and one or more constraints on the adjustable controls, one or more values for one or more of the adjustable controls. The values are configured to control the animation model to cause the computer-generated representation to perform a representation of the pose using the one ormore adjustable controls.
Background
BRIEF DESCRIPTION OF DRAWINGS(1) Illustrative embodiments of the present invention are described in detail below with reference to the following drawing figures:(2) FIG. 1 illustrates an example of asystem for capturing a performance of a subject using facial motion information and an animation model.(3) FIG. 2 illustrates an example of a graphical interface with graphical elements representing adjustable controls of an animation model.(4) FIG. 3 illustrates an example of a system for capturing a performance of a subject by determining control values of an animation model.(5) FIG. 4 illustrates an example of a solver used in capturing a performance of a subject.(6) FIGS. 5(a)-5(c) illustrate collected information used for producing a model capable of representing facial expressions of a subject.(7) FIGS. 6(a)-6(d) illustrate examples of determined positions of a marker.(8) FIG. 7 is an example of a diagram illustrating a projection technique.(9) FIGS. 8(a)-8(d) illustrate contours of a subject's mouth.(10) FIG. 9 illustrates editing expressions provided by a model of a subject's face.(11) FIGS. 10(a)-10(c) illustrate mapping a subject's facial expression to a representation of the subject and a representation of a character.(12) FIG. 11 is a flow chart illustrating a process for determining control values for controlling features of a computer-generated representation of a subject based on a performance of the subject.(13) FIG. 12 shows an example of a computer system that m
Claims
1. A computer-implemented method of determining control values for controlling features of a computer-generated representation of a subject based on a performance of the subject, comprising: obtaining input data corresponding to a pose performed by the subject, the input data including position information defining positions on a face of the subject; obtaining an animation model for the subject, the animationmodel including adjustable controls that control the animation model to define facial features of the computer-generated representation of the face; matching one or more of the positions on the face with one or more corresponding positions on the animation model, wherein matching includes using an objective function to project an error onto a control space of the animation model; determining, using the projected error and one or more constraintson the adjustable controls, one or more values for one or more of the adjustable controls, the one or more values being configured to control the animation model to cause the computer-generated representation of the face to perform a representation of the pose using the one or more adjustable controls; and modifying the one or more adjustable controls to include the one or more values based on the projected error and the one or more constraints,wherein the modified one or more adjustable controls cause the computer-generated representation of the face to perform the representation of the pose.
15. A system, comprising: amemory storing a plurality of instructions; and one or more processors configurable to: obtain input data corresponding to a pose performed by the subject, the input data including position information defining positions on a face of the subject; obtain an animation model for the subject, the animation model including adjustable controls that control the animation model to define facial features of the computer-generated representation of the face; match one or more of the positions on the face with one or more corresponding positions on the animation model, wherein matching includes using an objective function to projectan error onto a control space of the animation model; determine, using the projected error and one or more constraints on the adjustable controls, one or more values for one or more of the adjustable controls, the one or more values being configured to control the animation model to cause the computer-generated representation of the face to perform a representation of the pose using the one or more adjustable controls; and modify the one or moreadjustable controls to include the one or more values based on the projected error and the one or more constraints, wherein the modified one or more adjustable controls cause the computer-generated representation of the face to perform the representation of the pose.
18. A computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising: instructions that cause the one ormore processors to obtain input data corresponding to a pose performed by the subject, the input data including position information defining positions on a face of the subject; instructions that cause the one or more processors to obtain an animation model for the subject, the animation model including adjustable controls that control the animation model to define facial features of the computer-generated representation of the face; instructions that cause the one or more processors to match one or more of the positions on the face with one or more corresponding positions on the animation model, wherein matching includesusing an objective function to project an error onto a control space of the animation model; instructions that cause the one or more processors to determine, using the projected error and one or more constraints on the adjustable controls, one or more values for one ormore of the adjustable controls, the one or more values being configured to control the animation model to cause the computer-generated representation of the face to perform a representation of the pose using the one or more adjustable controls; and instructions that cause the one or more processors to modify the one or more adjustable controls to include the one or more values based on the projected error and the one or more constraints, wherein the modified one or more adjustable controls cause the computer-generated representationof the face to perform the representation of the pose.