Reducing domain shift in neural motion controllers

Number: 12718450
Published: 2026-08-25
Filed: 2023-02-21
Assignee: Disney Enterprises, INC.
Inventors: Guay; Martin, Agrawal; Dhruv, Borer; Dominik Tobias, Buhmann; Jakob Joachim, Ryffel; Mattia Gustavo Bruno Paolo, Sumner; Robert Walker
CPC: G06T13/00; G06T13/40; G06T7/248; G06T7/74; G06T2207/10016; G06T2207/20081; G06T2207/20084; G06T2207/30196
Verdict: Low Notable software
First reported: 2026-W36 (2026-09-06)
Source: Google Patents · FreePatentsOnline

The keeper's note

Trains a neural controller to translate a control signal into realistic character motion — locomotion-control ML relevant to animatronics/robot movement research.

Abstract

One embodiment of the present invention sets forth a technique for training a neural motion controller. The technique includes determining a first set of features associated with a first control signal for a virtual character. The technique also includes matching the first set of features to a first sequence of motions included in a plurality of sequences of motions. The technique further includes training the neural motion controller based on one or more motions included in the first sequence of motions and the first control signal.

Background

BACKGROUND Field of the Various Embodiments (1) Embodiments of the present disclosure relate generally to machine learning and neural motion controllers and, more specifically, to techniques for reducing domain shift in neural motion controllers. Description of the Related Art (2) Neural motion controllers are neural networks that can be used to animate virtual characters in real-time, given input signals for controlling the movements of the virtual characters. For example, a neural motion controller can include a deep learning model that generates a sequence of poses (i.e., positions and orientations) used to animate a virtual character. The resulting animation can then be used in a game, a previsualization of a film or television show, or another application involving the virtual character. (3) A neural motion controller can be trained using training input data that is generated from motion capture data. For example, training input data for the neural motion controller could include a “root trajectory” that is computed based on positions associated with the hips of the virtual character. Given this root trajectory, the neural motion controller would be trained to output poses for the virtual character that cause the virtual character to follow the root trajectory. (4) However, during inference, input data into the trained neural motion controller is commonly derived from a control signal generated by an input device, which differs from the training input data previously use

Claims

1. A computer-implemented method for training a neural motion controller, the method comprising: determining a first set of features associated with a first control signal for a virtual character, the first control signal generated via an input device, wherein the determining comprises computing the first set of features based on a weighted combination of multiple sets of features associated with the first control signal; matching the first set of features to a first sequence of motions included in a plurality of sequences of motions, wherein the matching comprises determining a match between the first set of features and a second set of features associated with the first sequence of motions; and training the neural motion controller based on one or more motions included in the first sequence of motions and the first control signal. || 9. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: determining a first set of features associated with a first control signal for a virtual character, the first control signal generated via an input device; matching the first set of features to a first sequence of motions included in a plurality of sequences of motions, wherein the matching comprises determining a distance between the first set of features and a second set of features associated with a first motion included in the first sequence of motions; and training a neural motion controller to generate one or more motions included in the first sequence of motions based on the first control signal. || 17. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: determining a first set of features associated with a first control signal for a virtual character, the first control signal generated via an input device; matching the first set of features to a first sequence of motions included in a plurality of sequences of motions, wherein the matching comprises determining a distance between the first set of features and a second set of features associated with a first motion included in the first sequence of motions; and training a neural motion controller to generate one or more motions included in the first sequence of motions based on the first control signal.