DATA-DRIVEN PHYSICS-BASED FACIAL ANIMATION RETARGETING

Number: 20240249459
Published: 2024-07-25
Filed: 2024-01-24
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Bradley; Derek Edward et al.
CPC: G06T13/40
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Data-driven physics-based facial-animation retargeting technique.

Abstract

One embodiment of the present invention sets forth a technique for retargeting a facial expression to a different facial identity. The technique includes generating, based on an input target facial identity, a facial identity code in an input identity latent space. The technique further includes converting a spatial input point from an input facial identity space of the input target facial identity to a canonical-space point in a canonical space. The technique still further includes generating one or more canonical simulator control values based on the facial identity code, an input source facial expression, and the canonical-space point. The technique still further includes generating a simulated active soft body based on one or more identity-specific control values, wherein each identity-specific control value corresponds to one or more of the canonical simulator control values and is in an output facial identity space associated with an output target facial identity.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to computer science and computer-generated graphics and, more specifically, to data-driven physics-based facial animation retargeting. DESCRIPTION OF THE RELATED ART

Facial animation retargeting refers to the changing of the facial identity of a representation of a “target’ individual to produce a modified representation having an identity of a “source” individual while maintaining the performance of the target individual. The representation of each individual can be a mesh such as a geometric surface mesh or other 3D geometric representation stored in memory of a computer system. The geometric surface mesh can have triangular mesh elements, for example. Each mesh element can be specified by a number of vertices. For example, a triangular mesh element can be specified by three vertices, and a tetrahedral element can by four vertices, or a hexahedral element by eight vertices. The facial identity includes aspects of a facial appearance that arise from differences in personal identities, such as ages, eye colors, and/or other factors. For example, two different facial identities can be attributed to two different individuals, the same individual under different lighting conditions, or the same individual at different ages. The performance of an individual includes the identity of the individual and the facial expressions and poses of the individual encoded in the rep

Claims

1. A computer-implemented method comprising: generating, based on an input target facial identity, a facial identity code in an input identity latent space; converting a spatial input point from an input facial identity space of the input target facial identity to a canonical-space point in a canonical space; generating one or more canonical simulator control values based on the facial identity code, an input source facial expression, and the canonical-space point; and generating a simulated active soft body based on one or more identity-specific control values, wherein each identity-specific control value corresponds to one or more of the canonical simulator control values and is in an output facial identity space associated with an output target facial identity. || 19. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: generating, based on an input target facial identity, a facial identity code in an input identity latent space; converting a spatial input point from an input facial identity space of the input target facial identity to a canonical-space point in a canonical space; generating one or more canonical simulator control values based on the facial identity code, an input source facial expression, and the canonical-space point; and generating a simulated active soft body based on one or more identity-specific control values, wherein each identity-specific control value corresponds to one or more of the canonical simulator control values and is in an output facial identity space associated with an output target facial identity. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: generating, based on an input target facial identity, a facial identity code in an input identity latent space; converting a spatial input point from an input facial identity space of the input target facial identity to a canonical-space point in a canonical space; generating one or more canonical simulator control values based on the facial identity code, an input source facial expression, and the canonical-space point; and generating a simulated active soft body based on one or more identity-specific control values, wherein each identity-specific control value corresponds to one or more of the canonical simulator control values and is in an output facial identity space associated with an output target facial identity.