DYNAMIC WEIGHT ADJUSTMENT FOR MOTION CONTROL VIA MULTI-OBJECTIVE REINFORCEMENT LEARNING

Number: 20260212200
Published: 2026-07-23
Filed: 2026-01-22
Assignee: DISNEY ENTERPRISES, INC.
Inventors: BÄCHER; Moritz Niklaus, KNOOP; Lars Espen, GRANDIA; Ruben Jelle, SERIFI; Agon, ALEGRE; Lucas Nunes, CAO; David Benjamin
CPC: G06N3/006; G06N3/092; G06N3/008; G06N3/044; G06N3/045; G06N3/047; G06N3/0475; G06N3/08; G06N3/084; G06N3/088; G06N7/01; G06N20/00
Verdict: High Notable software
First reported: 2026-W30 (2026-07-24)
Source: Google Patents · FreePatentsOnline

The keeper's note

One embodiment of the present invention sets forth a technique for controlling motion in an articulated object.

Abstract

One embodiment of the present invention sets forth a technique for controlling motion in an articulated object. The technique includes generating, via execution of a machine learning model based on a first state of the articulated object at a first time step, a first plurality of weights for a plurality of rewards, and determining (i) one or more actions at the first time step based on the first plurality of weights and (ii) a second state of the articulated object at a second time step based on the first state and the action(s). The technique also includes generating, via execution of the machine learning model based on the second state, a second plurality of weights for the plurality of rewards, and determining one or more additional actions based on the second plurality of weights. The technique further includes generating the motion based on the action(s) and additional action(s).

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to motion tracking and reinforcement learning and, more specifically, to dynamic weight adjustment for motion control via multi-objective reinforcement learning. Description of the Related Art

Physics-based character control is a technique for generating motion in physical and/or virtual characters in a physically realistic and robust manner. To achieve this type of motion, a controller computes actions (e.g., joint torques, target positions, actuator commands, etc.) that cause a character to move in a desired manner while respecting physics constraints such as (but not limited to) gravity, momentum, friction, and/or contact forces. The actions are used to update joints of the character and produce physically plausible motion in a robot, game, animation, simulation, and/or another application involving the character.

Existing approaches for performing physics-based character control include the use of reinforcement learning (RL) to train a control policy to output actions that maximize a reward function. The reward function can include one or more objectives related to the accuracy with which a reference motion is tracked. When multiple objectives are included in the reward function, a set of weights is used to control the relative priorities and/or effects of the objectives on the outputted actions.

However, these approaches have traditionally used r

Claims

1. A computer-implemented method for controlling motion in an articulated object, the method comprising: generating, via execution of a machine learning model based on a first state of the articulated object at a first time step, a first plurality of weights for a plurality of rewards associated with the motion in the articulated object; determining (i) one or more actions associated with the motion of the articulated object at the first time step based on the first plurality of weights and (ii) a second state of the articulated object at a second time step based on the first state and the one or more actions; generating, via execution of the machine learning model based on the second state, a second plurality of weights for the plurality of rewards; determining one or more additional actions associated with the motion of the articulated object based on the second plurality of weights; and generating the motion based on the one or more actions and the one or more additional actions. || 11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: generating, via execution of a machine learning model based on a first state of an articulated object at a first time step, a first plurality of weights for a plurality of rewards associated with motion in an articulated object; generating, via execution of an additional machine learning model based on the first state and the first plurality of weights, one or more actions associated with a motion of the articulated object at the first time step; and causing a task associated with the articulated object to be performed based on the one or more actions. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: generating, via execution of a machine learning model based on a first state of an articulated object at a first time step, a first plurality of weights for a plurality of rewards associated with motion in the articulated object; determining (i) one or more actions associated with a motion of the articulated object at the first time step based on the first plurality of weights and (ii) a second state of the articulated object at a second time step based on the first state and the one or more actions; generating, via execution of the machine learning model based on the second state, a second plurality of weights for the plurality of rewards; determining one or more additional actions associated with the motion of the articulated object based on the second plurality of weights; and training the machine learning model based on one or more additional rewards associated with the one or more actions and the one or more additional actions.