SYNTHESIZING SEQUENCES OF 3D GEOMETRIES FOR MOVEMENT-BASED PERFORMANCE

Number: 20230154089
Published: 2023-05-18
Filed: 2021-11-15
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Bradley; Derek Edward et al.
CPC: G06T9/002; G06T13/40; G06N3/044; G06N3/045; G06N3/047; G06N3/08; G06N3/084; G06N3/088; G06T3/00
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Movement/performance-driven 3D geometry synthesis technique.

Abstract

A technique for generating a sequence of geometries includes converting, via an encoder neural network, one or more input geometries corresponding to one or more frames within an animation into one or more latent vectors. The technique also includes generating the sequence of geometries corresponding to a sequence of frames within the animation based on the one or more latent vectors. The technique further includes causing output related to the animation to be generated based on the sequence of geometries.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and animation and, more specifically, to synthesizing sequences of three-dimensional (3D) geometries for movement-based performance. Description of the Related Art

Realistic digital faces are required for various computer graphics and computer vision applications. For example, digital faces are oftentimes used in virtual scenes of film or television productions and in video games.

To capture photorealistic faces, a typical facial capture system employs a specialized light stage and hundreds of lights that are used to capture numerous images of an individual face under multiple illumination conditions. The facial capture system additionally employs multiple calibrated camera views, uniform or controlled patterned lighting, and a controlled setting in which the face can be guided into different expressions to capture images of individual faces. These images can then be used to determine three-dimensional (3D) geometry and appearance maps that are needed to synthesize digital versions of the faces.

Machine learning models have also been developed to synthesize digital faces. These machine learning models can include a large number of tunable parameters and thus require a large amount and variety of data to train. However, collecting training data for these machine learning models can be time- and resource-intensive. For example, a dee

Claims

1. A computer-implemented method for generating a sequence of geometries, the computer-implemented method comprising: converting, via an encoder neural network, one or more input geometries corresponding to one or more frames within an animation into one or more latent vectors; generating the sequence of geometries corresponding to a sequence of frames within the animation based on the one or more latent vectors; and causing output related to the animation to be generated based on the sequence of geometries. || 11. One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: converting, via an encoder neural network, one or more input geometries corresponding to one or more frames within an animation into one or more latent vectors; generating a sequence of geometries corresponding to a sequence of frames within the animation based on the one or more latent vectors and one or more positions of the one or more frames within the animation; and causing output related to the animation to be generated based on the sequence of geometries. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to: convert, via an encoder neural network, one or more input geometries corresponding to one or more frames within an animation into one or more latent vectors; generate a sequence of geometries corresponding to a sequence of frames within the animation based on the one or more latent vectors and one or more positions of the one or more frames within the animation; and cause output related to the animation to be generated based on the sequence of geometries.