- Number
- 11747890
- Published
- 2023-09-05
- Filed
- 2021-11-02
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Papon; Jeremie A. et al.
- CPC
- G02B27/0179; B25J13/00; G06N20/00; G06F3/016; G06F3/011; G06T13/40; G06T19/006; B25J9/0081; B25J11/0005; G06F3/167; G02B27/017
- Verdict
- High Hardware
- Source
- Google Patents · FreePatentsOnline
The keeper's note
AR-enhanced interactive robotic/animatronic animation system (granted).
Abstract
A system includes a computing platform having processing hardware, one or more sensor(s), and a memory storing perception software and animation software, as well as an augmented reality (AR) headset including another processing hardware. The computing platform is configured to obtain an environmental data using the sensor(s), determine the location and orientation of the AR headset using the perception software and the environmental data, identify an action for execution by the computing platform, using the animation software and the location and orientation of the AR headset, and transmit performative data corresponding to the identified action to the AR headset. The AR headset is configured to receive, from the computing platform, the performative data, and to render at least one AR effect corresponding to the identified action using the performative data.
Background
BACKGROUND (1) Animating and testing lifelike interactive robotic characters is a challenging problem at least in part because it involves a feedback loop between the human and robot actions. That is to say, both the human and the robot are continuously reacting to each other. Because of this feedback loop, standard animation tools do not give a true-to-life view of what the animated character will ultimately look like. (2) One possible solution to this problem is to provide a simulator and procedural animation system with live recorded human inputs from a sensor, such as a webcam, for example, but this still does not fully close the feedback loop, since the input sensors are not moving as they would on the physical hardware. Consider, for instance, a camera placed in a robot's head: as the robot moves, what the camera sees is influenced by the movement itself. As a result, for example, when a robot glances at a person, the animation of the glance itself will influence the performance of the system as whole. Consequently, there is a need in the art for a simulation solution that integrates the human and robotic perspectives of a mutual interaction.
Claims
1. A system comprising: an automaton having a computing platform including a first processing hardware, at least one sensor, and a memory storing a perception software and an animation software; and an augmented reality (AR) headset including a second processing hardware; the computing platform configured to: obtain an environmental data, using the at least one sensor under the control of the first processing hardware; determine a location and an orientation of the AR headset, using the first processing hardware, the perception software, and the environmental data; identify a real-world action for execution by the computing platform, using the first processing hardware, the animation software, and the location and the orientation of the AR headset; execute the identified real-world action, wherein the identified real-world action comprises a physical movement by the automaton; and transmit, to the AR headset, a performative data corresponding to the identified real-world action, using the first processing hardware; the AR headset configured to: receive, from the computing platform, the performative data, using the second processing hardware; and render at least one AR effect corresponding to the identified real-world action, using the second processing hardware and the performative data. ||
9. A method for use by a system comprising an automaton having a computing platform including a first processing hardware, at least one sensor, and a memory storing a perception software and an animation software, the system further comprising an augmented reality (AR) headset including a second processing hardware, the method comprising: obtaining an environmental data, by the computing platform using the at least one sensor under the control of the first processing hardware; determining a location and an orientation of the AR headset, by the computing platform using the first processing hardware, the perception software, and the environmental data; identifying a real-world action for execution by the computing platform, by the computing platform using the first processing hardware, the animation software, and the location and the orientation of the AR headset; executing the identified real-world action, by the computing platform using the first processing hardware, wherein the identified real-world action comprises a physical movement by the automaton; and transmitting, to the AR headset, a performative data corresponding to the identified real-world action, by the computing platform using the first processing hardware; receiving from the computing platform the performative data, by the AR headset using the second processing hardware; and rendering at least one AR effect corresponding to the identified real-world action, by the AR headset using the second processing hardware and the performative data. ||
17. An automaton for providing augmented reality (AR) effects to an AR headset, the automaton comprising: a computing platform including a processing hardware and a memory storing an animation software; and the computing platform configured to: obtain environmental data describing a plurality of environmental features, using the processing hardware; determine a location of the AR headset, using the processing hardware; identify a real-world action for execution by the computing platform, using the processing hardware, the animation software, the environmental data and the location; execute the identified real-world action, using the processing hardware, wherein the identified real-world action comprises a physical movement by the automaton; and transmit, to the AR headset, a performative data corresponding to the identified real-world action, using the processing hardware.