REDUCING DOMAIN SHIFT IN NEURAL MOTION CONTROLLERS

Number: 20240282028
Published: 2024-08-22
Filed: 2023-02-21
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Guay; Martin et al.
CPC: G06T13/00; G06T13/40; G06T7/248; G06T7/74
Verdict: Medium Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Neural motion-controller technique for character/robotics animation.

Abstract

One embodiment of the present invention sets forth a technique for training a neural motion controller. The technique includes determining a first set of features associated with a first control signal for a virtual character. The technique also includes matching the first set of features to a first sequence of motions included in a plurality of sequences of motions. The technique further includes training the neural motion controller based on one or more motions included in the first sequence of motions and the first control signal.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and neural motion controllers and, more specifically, to techniques for reducing domain shift in neural motion controllers. Description of the Related Art

Neural motion controllers are neural networks that can be used to animate virtual characters in real-time, given input signals for controlling the movements of the virtual characters. For example, a neural motion controller can include a deep learning model that generates a sequence of poses (i.e., positions and orientations) used to animate a virtual character. The resulting animation can then be used in a game, a previsualization of a film or television show, or another application involving the virtual character.

A neural motion controller can be trained using training input data that is generated from motion capture data. For example, training input data for the neural motion controller could include a “root trajectory” that is computed based on positions associated with the hips of the virtual character. Given this root trajectory, the neural motion controller would be trained to output poses for the virtual character that cause the virtual character to follow the root trajectory.

However, during inference, input data into the trained neural motion controller is commonly derived from a control signal generated by an input device, which differs from the training input data pr

Claims

1. A computer-implemented method for training a neural motion controller, the method comprising: determining a first set of features associated with a first control signal for a virtual character; matching the first set of features to a first sequence of motions included in a plurality of sequences of motions; and training the neural motion controller based on one or more motions included in the first sequence of motions and the first control signal. || 11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: determining a first set of features associated with a first control signal for a virtual character; matching the first set of features to a first sequence of motions included in a plurality of sequences of motions; and training a neural motion controller to generate one or more motions included in the first sequence of motions based on the first control signal. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: determining a first set of features associated with a first control signal for a virtual character; matching the first set of features to a first sequence of motions included in a plurality of sequences of motions; and training a neural motion controller to generate one or more motions included in the first sequence of motions based on the first control signal.