TRAINING FOR NEURAL SPLINE DEFORMATION

Number: 20250356645
Published: 2025-11-20
Filed: 2025-05-16
Assignee: DISNEY ENTERPRISES, INC.
Inventors: ZHANG; Yang et al.
CPC: G06F16/735; G06F16/738; G06F16/7867; G06V10/75; G06V10/82
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Neural spline deformation training technique (character rigging/rendering).

Abstract

One embodiment of the present invention sets forth a technique for generating a neural deformation model. The technique includes inputting, into a machine learning model, (i) a set of canonical coordinates in a scene and (ii) one or more times included in a temporal trajectory of the scene. The technique also includes generating, via execution of the machine learning model, one or more sets of attributes associated with the set of canonical coordinates and the one or more times. The technique further includes computing one or more losses based on (i) a velocity included in the one or more sets of attributes and (ii) one or more representations of the scene at the one or more times, and training the machine learning model based on the one or more losses.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to training for neural spline deformation. Description of the Related Art

Films, video games, virtual reality (VR) systems, augmented reality (AR) systems, mixed reality (MR) systems, motion capture, and/or other types of applications frequently involve generating and/or making changes to depictions of 3D scenes over time. Traditionally, a visual representation of a given scene is generated and/or edited via a time-consuming, iterative, and/or laborious process. For example, a conventional visual effects workflow may involve a visual effects artist adding special effects and/or posing or animating a virtual character on a frame-by-frame basis.

More recently, advancements in machine learning and deep learning have led to the development of neural deformation models, which include deep neural networks that learn implicit representations of non-rigid and/or time-varying scenes. These neural deformation models commonly include coordinate neural networks that map coordinates in a canonical space to corresponding deformed coordinates at various temporal offsets. The deformed coordinates can then be used to render and/or reconstruct the corresponding scenes at the temporal offsets.

However, conventional neural deformation models are associated with a tradeoff between performance and ability to g

Claims

1. A computer-implemented method for generating a neural deformation model, the method comprising: inputting, into a machine learning model, (i) a set of canonical coordinates in a scene and (ii) one or more times included in a temporal trajectory of the scene; generating, via execution of the machine learning model, one or more sets of attributes associated with the set of canonical coordinates and the one or more times; computing one or more losses based on (i) a velocity included in the one or more sets of attributes and (ii) one or more representations of the scene at the one or more times; and training the machine learning model based on the one or more losses. || 11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: inputting, into a machine learning model, (i) a set of canonical coordinates in a scene and (ii) one or more times included in a temporal trajectory of the scene; generating, via execution of the machine learning model, one or more sets of attributes associated with the set of canonical coordinates and the one or more times; computing one or more losses based on (i) a velocity included in the one or more sets of attributes and (ii) one or more representations of the scene generated from the one or more sets of attributes and a 3D Gaussian parameterization of the scene; and training the machine learning model based on the one or more losses. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: inputting, into a machine learning model, (i) a set of canonical coordinates in a scene and (ii) one or more times included in a temporal trajectory of the scene; generating, via execution of the machine learning model, one or more sets of attributes associated with the set of canonical coordinates and the one or more times; computing one or more losses based on (i) a velocity included in the one or more sets of attributes, (ii) an acceleration included in the one or more sets of attributes, and (iii) one or more representations of the scene generated from the one or more sets of attributes and a 3D Gaussian parameterization of the scene; and training the machine learning model based on the one or more losses.