Secondary-dynamics extraction technique for facial performance capture.
A modeling engine generates a prediction model that quantifies and predicts secondary dynamics associated with the face of a performer enacting a performance. The modeling engine generates a set of geometric representations that represents the face of the performer enacting different facial expressions under a range of loading conditions. For a given facial expression and specific loading condition, the modeling engine trains a Machine Learning model to predict how soft tissue regions of the face of the performer change in response to external forces applied to the performer during the performance. The modeling engine combines different expression models associated with different facial expressions to generate a prediction model. The prediction model can be used to predict and remove secondary dynamics from a given geometric representation of a performance or to generate and add secondary dynamics to a given geometric representation of a performance.
BACKGROUND Field of the Various Embodiments (1) The various embodiments relate generally to computer science and computer animation and, more specifically, to data-driven extraction and composition of secondary dynamics in facial performance capture. Description of the Related Art (2) Certain types of computer animation pipelines include a motion capture phase and a digital rendering phase. During the motion capture phase, a human performer enacts a performance within a motion capture environment. The motion capture environment typically includes multiple video cameras that are positioned at different angles relative to the performer and are configured to capture three-dimensional (3D) motion capture data as the performer enacts the performance. Subsequently, during the digital rendering phase, digital rendering techniques are used to process the 3D motion capture data to generate a 3D geometric model of the performer enacting the performance. A computer animation of the performance is then rendered based on the 3D geometric model. (3) Computer animation pipelines also can be implemented to generate computer animations representing the faces of digital characters exhibiting various facial expressions. For example, the motion capture phase of a computer animation pipeline could be implemented to generate 3D motion capture data representing a performer enacting a sequence of facial expressions during a performance. Subsequently, the digital rendering phase of the computer anima
1. A computer-implemented method for generating computer animations of facial expressions, the method comprising: generating a first geometric model of a first facial expression made by a performer during a performance, wherein the first facial expression in the first geometric model includes (i) primary dynamics that are caused by the performer making the first facial expression and (ii) a first level of secondary dynamics that are caused by one or more physical external forces on a facial area associated with the performer; generating a first set of values that quantify effects of the one or more physical external forces on one or more non-rigid portions of the facial area associated with the performer based on a second set of values that correspond to one or more rigid portions of the facial area; correcting the first geometric model based on the first set of values to generate a second geometric model of the first facial expression, wherein the first facial expression in the second geometric model includes a second level of secondary dynamics, and wherein one or more portions of the facial area associated with the performer respond differently to the second level of secondary dynamics relative to the first level of secondary dynamics; and generating a computer animation that is based on the second geometric model and depicts the first facial expression having the second level of secondary dynamics. ||
11. A non-transitory computer-readable medium storing program instructions that, when executed by a processor, cause the processor to generate computer animations of facial expressions by performing the steps of: generating a first geometric model of a first facial expression made by a performer during a performance, wherein the first facial expression in the first geometric model includes (1) primary dynamics that are caused by the performer making the first facial expression and (ii) a first level of secondary dynamics that are caused by one or more physical external forces on a facial area associated with the performer; generating a first set of values that quantify effects of the one or more physical external forces on one or more non-rigid portions of the facial area associated with the performer based on a second set of values that correspond to one or more rigid portions of the facial area; correcting the first geometric model based on the first set of values to generate a second geometric model of the first facial expression, wherein the first facial expression in the second geometric model includes a second level of secondary dynamics, and wherein one or more portions of the facial area associated with the performer respond differently to the second level of secondary dynamics relative to the first level of secondary dynamics; and generating a computer animation that is based on the second geometric model and depicts the first facial expression having the second level of secondary dynamics. ||
20. A system, comprising: a memory storing a software application; and a processor that, when executing the software application, is configured to perform the steps of: generating a first geometric model of a first facial expression made by a performer during a performance, wherein the first facial expression in the first geometric model includes (i) primary dynamics that are caused by the performer making the first facial expression and (ii) a first level of secondary dynamics that are caused by one or more physical external forces on a facial area associated with the performer, generating a first set of values that quantify effects of the one or more physical external forces on one or more non-rigid portions of the facial area associated with the performer based on a second set of values that correspond to one or more rigid portions of the facial area, correcting the first geometric model based on the first set of values to generate a second geometric model of the first facial expression, wherein the first facial expression in the second geometric model includes a second level of secondary dynamics, and wherein one or more portions of the facial area associated with the performer respond differently to the second level of secondary dynamics relative to the first level of secondary dynamics, and generating a computer animation that is based on the second geometric model and depicts the first facial expression having the second level of secondary dynamics.