DATA-DRIVEN EXTRACTION AND COMPOSITION OF SECONDARY DYNAMICS IN FACIAL PERFORMANCE CAPTURE

Number: 20210166458
Published: 2021-06-03
Filed: 2019-12-03
Assignee: DISNEY ENTERPRISES, INC.
Inventors: BEELER; Dominik Thabo, BRADLEY; Derek Edward, Sifakis; Eftychios Dimitrios, Zoss; Gaspard
CPC: G06T13/80; G06V40/176; G06F18/214; G06T13/40
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Facial performance-capture secondary-dynamics extraction technique.

Abstract

A modeling engine generates a prediction model that quantifies and predicts secondary dynamics associated with the face of a performer enacting a performance. The modeling engine generates a set of geometric representations that represents the face of the performer enacting different facial expressions under a range of loading conditions. For a given facial expression and specific loading condition, the modeling engine trains a Machine Learning model to predict how soft tissue regions of the face of the performer change in response to external forces applied to the performer during the performance. The modeling engine combines different expression models associated with different facial expressions to generate a prediction model. The prediction model can be used to predict and remove secondary dynamics from a given geometric representation of a performance or to generate and add secondary dynamics to a given geometric representation of a performance.

Background

BACKGROUND Field of the Various Embodiments

The various embodiments relate generally to computer science and computer animation and, more specifically, to data-driven extraction and composition of secondary dynamics in facial performance capture. Description of the Related Art

Certain types of computer animation pipelines include a motion capture phase and a digital rendering phase. During the motion capture phase, a human performer enacts a performance within a motion capture environment. The motion capture environment typically includes multiple video cameras that are positioned at different angles relative to the performer and are configured to capture three-dimensional (3D) motion capture data as the performer enacts the performance. Subsequently, during the digital rendering phase, digital rendering techniques are used to process the 3D motion capture data to generate a 3D geometric model of the performer enacting the performance. A computer animation of the performance is then rendered based on the 3D geometric model.

Computer animation pipelines also can be implemented to generate computer animations representing the faces of digital characters exhibiting various facial expressions. For example, the motion capture phase of a computer animation pipeline could be implemented to generate 3D motion capture data representing a performer enacting a sequence of facial expressions during a performance. Subsequently, the digital rendering phase of the compu

Claims

1. A computer-implemented method for generating computer animations of facial expressions, the method comprising: generating a first geometric model of a first facial expression made by a performer during a performance, wherein the first geometric model includes a first level of secondary dynamics; generating a first set of values that correspond to one or more non-rigid portions of a facial area associated with the performer based on a second set of values that correspond to one or more rigid portions of the facial area; correcting the first geometric model based on the first set of values to generate a second geometric model that includes a second level of secondary dynamics; and generating a computer animation that is based on the second geometric model and depicts the first facial expression having the second level of secondary dynamics. || 11. A non-transitory computer-readable medium storing program instructions that, when executed by a processor, cause the processor to generate computer animations of facial expressions by performing the steps of: generating a first geometric model of a first facial expression made by a performer during a performance, wherein the first geometric model includes a first level of secondary dynamics; generating a first set of values that correspond to one or more non-rigid portions of a facial area associated with the performer based on a second set of values that correspond to one or more rigid portions of the facial area; correcting the first geometric model based on the first set of values to generate a second geometric model that includes a second level of secondary dynamics; and generating a computer animation that is based on the second geometric model and depicts the first facial expression having the second level of secondary dynamics. || 20. A system, comprising: a memory storing a software application; and a processor that, when executing the software application, is configured to perform the steps of: generating a first geometric model of a first facial expression made by a performer during a performance, wherein the first geometric model includes a first level of secondary dynamics, generating a first set of values that correspond to one or more non-rigid portions of a facial area associated with the performer based on a second set of values that correspond to one or more rigid portions of the facial area, correcting the first geometric model based on the first set of values to generate a second geometric model that includes a second level of secondary dynamics, and generating a computer animation that is based on the second geometric model and depicts the first facial expression having the second level of secondary dynamics.