AUTONOMOUS HUMAN-ROBOT INTERACTION VIA DIFFUSION-BASED OPERATOR IMITATION

Number: 20260257351
Published: 2026-09-03
Filed: 2026-02-26
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Christen; Sammy Joe, Serifi; Agon, Mueller; David, Grandia; Ruben Jelle, Knoop; Lars Espen, Bächer; Moritz Niklaus, Wiedebach; Georg, Hopkins; Michael Anthony, Wang; Jenny
CPC: B25J9/163; B25J9/1689; G06N3/08; G05B2219/40116
Verdict: High Hardware
First reported: 2026-W36 (2026-09-05)
Source: Google Patents · FreePatentsOnline

The keeper's note

Trains a diffusion model so a robot can autonomously read and respond to a human's actions — the control layer for a free-roaming character droid.

Abstract

A method of training a diffusion model to enable autonomous interaction between a robotic device and a human includes receiving an operator command for the robotic device and pose data including a history of poses. The method includes applying noise to the operator command and encoding the noisy operator command, pose data, and history of operator commands into input tokens. The method includes inputting the tokens into a transformer encoder of the diffusion model and training the model to perform denoising to generate an autonomous command from the noisy operator command based on the pose data and command history. The method includes generating the autonomous command based on the transformer encoder output and executing the command to cause the robotic device to perform an action during autonomous interaction with the human.

Background

FIELD

The present application relates to systems and methods for autonomously controlling robotic device/human interactions. BACKGROUND

As robots become more ubiquitous, the situations in which a robotic device may interact with a human increase. Traditional training of robots focuses on enabling the robotic device to move stably and reliably within its environment. However, interactions between robots and humans are either avoided (e.g., the robotic device is physically separated from people), or the robotic device is at least partially controlled by an operator who guides the interaction of the robotic device with other people via a remote control. Better systems are needed to enable robots to autonomously interact with people.

In human-robotic device interaction (HRI), existing platforms often either define heuristics or rely on remote operation of such robots by an operator, through simple-to-use interfaces like gamepads. However, transitioning towards autonomous human-robotic device interactions through imitation learning presents significant challenges, primarily due to the need for expensive data collection. This process itself depends on starting with a stable and expressive robotic device. Advancements in robotics have greatly enhanced their expressiveness, enabling robots to perform a diverse range of complex motions under remote operation. Results have been achieved in training robots in simulation and transferring them to a variety of legged r

Claims

1. A method of training a diffusion model to enable autonomous interaction between a robotic device and a human, the method comprising: receiving, via a processing element, an operator command for the robotic device; receiving, via the processing element, pose data including a history of poses; applying, via the processing element, noise to the operator command to generate a noisy operator command; encoding, via the processing element, the noisy operator command, the pose data, and a history operator commands into a plurality of input tokens; inputting, via the processing element, the plurality of input tokens into a transformer encoder of the diffusion model; training, via the processing element, the diffusion model to perform a denoising operation to generate an autonomous command from the noisy operator command based on the pose data and the history of operator commands; generating, via the processing element, the autonomous command based on an output of the transformer encoder; and executing, via the robotic device, the autonomous command to cause the robotic device to perform an action during the autonomous interaction between the robotic device and the human. || 11. A system for training a robotic device to autonomously interact with a human, the system comprising: a robotic device; a controller in communication with the robotic device and configured to receive an operator command for the robotic device; a perceptive input system configured to determine pose data including a history of poses; and a processing element configured to: apply noise to the operator command to generate a noisy operator command, encode the noisy operator command, the pose data, and a history of operator commands into a plurality of input tokens, input the plurality of input tokens into a transformer encoder of a diffusion model, train the diffusion model to perform a denoising operation to generate an autonomous command from the noisy operator command based on the pose data and the history of operator commands; and generate the autonomous command based on an output of the transformer encoder, wherein the robotic device is configured to: execute the autonomous command to cause the robotic device to perform an action during the autonomous interaction with the human.