Determining control values of an animation model using performance capture

Number: 10147219
Published: 2018-12-04
Filed: 2017-02-03
Assignee: LUCASFILM ENTERTAINMENT COMPANY LTD.
Inventors: Bhat; Kiran; Koperwas; Michael; Yost; Jeffery; Yu; Ji Hun; Santos; Sheila
CPC: G06T13/40; G06V40/176; G06T7/75; G06T7/60; G06F18/22; G06T13/80
Verdict: High Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Performance-capture animation control.

Abstract

Performance capture systems and techniques are provided for capturing a performance of a subject and reproducing an animated performance that tracks the subject's performance. For example, systems and techniques are provided for determining control values for controlling an animation model to define features of a computer-generated representation of a subject based on the performance. A method may include obtaining input data corresponding to a pose performed by the subject, the input data including position information defining positions on a face of the subject. The method may further include obtaining an animation model for the subject that includes adjustable controls that control the animation model to define facial features of the computer-generated representation of the face, and matching one or more of the positions on the face with one or more corresponding positions on the animation model. The matching includes using an objective function to project an error onto a control space of the animation model. The method may further include determining, using the projected error and one or more constraints on the adjustable controls, one or more values for one or more of the adjustable controls. The values are configured to control the animation model to cause the computer-generated representation to perform a representation of the pose using the one or more adjustable controls.

Background

FIELD(1) The present disclosure generally relates to capturing performance characteristics of a subject. For example, performance capture systems and techniques may be provided for capturing performance characteristics of a subject.BACKGROUND(2) Performance capture is an important aspect in animation. Providing a realistic animation of a subject or a fictional character representing the subject requires that a subject's performance is effectively captured. For example, an effective performance capture ensures that the nuances of the subject's performance are conveyed in the animation. Transfer of the performance of the subject onto a computer-generated representation of the subject or fictional character can be a complex and difficult process. Embodiments of the invention address these and other problems both individually and collectively.SUMMARY(3) Techniques and systems are described for capturing performance characteristics of a subject. Using input data obtained from video images of the subject performing one or more poses, a solver may determine values for adjustable controls of an animation model for the subject. In some examples, the input data can include position information defining positions on the subject and may further include contour information defining contour features of the subject. The animation model includes various adjustable controls that control the animation model. Different control values for the adjustable controls allow the animation model to defi

Claims

1. A computer-implemented method comprising: obtaining input data corresponding to a pose of a subject, the input data including position information for points on a face of the subject; obtaining an animation model, the animation model being associated with adjustable controls usable for adjusting the animation model to define poses for a computer representation of a face; determining, using the position information, a correspondence between one or more of the points on the face of the subject and one or more points of the computer representation of the face, wherein the determining includes using an objective function to project an error onto a control space of the animation model; and setting the one or more adjustable controls to have one or more values based on the projected error and one or more constraints for the one or more adjustable controls, wherein the setting of the one or more adjustable controls causes the computer representation of the face to have a pose based on the pose of the subject. 15. A system, comprising: a memory storing a plurality of instructions; and one or more processors configurable to: obtain input data corresponding to a pose of a subject, the input data including position information for points on a face of the subject; obtain an animation model, the animation model being associated with adjustable controls usable for adjusting the animation model to define poses for a computer representation of a face; determine, using the position information, a correspondence between one or more of the points on the face of the subject and one or more points of the computer representation of the face, wherein the determining includes using an objective function to project an error onto a control space of the animation model; and set the one or more adjustable controls to have one or more values based on the projected error and one or more constraints for the one or more adjustable controls, wherein the setting of the one or more adjustable controls causes the computer representation of the face to have a pose based on the pose of the subject. 18. A non-transitory computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising: instructions that cause the one or more processors to obtain input data corresponding to a pose of a subject, the input data including position information for points on a face of the subject; instructions that cause the one or more processors to obtain an animation model, the animation model being associated with adjustable controls usable for adjusting the animation model to define poses for a computer representation of a face; instructions that cause the one or more processors to determine, using the position information, a correspondence between one or more of the points on the face of the subject and one or more points of the computer representation of the face, wherein the determining includes using an objective function to project an error onto a control space of the animation model; and instructions that cause the one or more processors to set the one or more adjustable controls to have one or more values based on the projected error and one or more constraints for the one or more adjustable controls, wherein the setting of the one or more adjustable controls causes the computer representation of the face to have a pose based on the pose of the subject.