- Number
- 10937220
- Published
- 2021-03-02
- Filed
- 2019-04-22
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Mitchell; Kenneth J.
- CPC
- A63F13/573; G06T13/40; A63F13/355; A63F13/358
- Verdict
- Set aside animation content streaming, delivery plumbing
- Source
- Google Patents · FreePatentsOnline
Abstract
Embodiments provide for animation streaming for media interaction by receiving, at a generator, inputs from a target device presenting of a virtual environment; updating, based on the user inputs, a model of the virtual environment; determining network conditions between the generator and target device; generating a packet that includes a forecasted animation set for a virtual object in the updated model that comprises rig updates for the virtual object for at least two different states, and a number of states included in the packet is based on the network conditions; and streaming the packet to the target device, where the target device: receives a second input to interact with the virtual environment that changes the virtual environment to a given state; selects and applies a rig update associated with the given state a local model of the virtual object; and outputs the updated local model on the target device.
Background
BACKGROUND (1) The animation of immersive content, such as virtual objects in a video game, Augmented Reality (AR) application, Virtual Reality (VR) application, or other context includes providing a sequence of images for display to a user. When animated, consecutive images of the virtual objects provide changes in the appearance and/or position of the virtual objects in which the virtual objects may be repositioned within an environment (either a purely virtual environment or a virtual environment mapped to a real-world environment), modeled in a different pose, affected by changing lighting/texturing conditions, or moved into/out of a frame of view for the user. Users increasingly desire that the animations be provided at a consistent rate between the various animated virtual objects, at a steady rate of animation (e.g., avoiding jitter in animating the virtual objects), a high rate of refresh (e.g., at least X Frames per Second (FPS)), and at a high level of graphical detail. SUMMARY (2) The present disclosure, in one embodiment, provides a method of animation streaming for media interaction, comprising: receiving, at a generator, user inputs from a target device displaying a presentation of a virtual environment; updating, at the generator based on the user inputs, an environmental model of the virtual environment; determining network conditions between the generator and the target device; generating a packet that includes a forecasted animation set for a virtual object
Claims
1. A method, comprising: receiving, at a generator, user inputs from a target device displaying a presentation of a virtual environment; updating, at the generator based on the user inputs, an environmental model of the virtual environment; determining network conditions between the generator and the target device; generating a packet that includes a forecasted animation set for a virtual object in the updated environmental model, wherein the forecasted animation set comprises a plurality of rig updates for the virtual object for at least two different states of the environmental model, and a number of states comprising the at least two states is based on the network conditions, wherein the number of states comprising the at least two different states is based on at least one of an available bandwidth between the target device and the generator and a latency between the target device and the generator; and streaming the packet to the target device. ||
8. A method, comprising: receiving, at a target device displaying a virtual environment, a packet stream from a remote generating device, wherein each packet in the packet stream includes a forecasted animation set for a virtual object included in the virtual environment, wherein the forecasted animation set comprises a first rig update for a first state of the virtual environment at a subsequent time and a second rig update for a second state of the virtual environment at the subsequent time; receiving, at the target device, a user input to interact with the virtual environment that changes the virtual environment to a given state of the first state and the second state; selecting a given rig update from the packet stream associated with the given state; applying the given rig update for the virtual object to a local model of the virtual object displayed by the target device; outputting, for the subsequent time, the local model on the target device according to the given rig update; in response to determining that network conditions for the target device indicate one of an increase or a decrease in at least one of: latency for communication with the remote generating device, and bandwidth for communication with the remote generating device, requesting a corresponding increase or decrease in a number of branching states included in each packet; and transmitting the user input to the remote generating device. ||
17. A system, comprising: a generating device, including a generating processor and a generating memory including generating instructions that when performed by the generating processor enable the generating device to perform an animation operation comprising: generating a plurality of packets, wherein each packet of the plurality of packets includes a forecasted animation set for a virtual object, wherein the forecasted animation set comprises a plurality of rig updates for the virtual object for at least two different states of a virtual environment for one time; adjusting a number of rig updates in the forecasted animation set of a given packet based on at least one of a latency and an available bandwidth of a network over which the plurality of packets are streamed; and streaming the plurality of packets via the network; a target device, in communication with the generating device via the network, the target device including a target processor and a target memory including target instructions that when performed by the target processor enable the target device to perform an operation comprising: displaying a local model of the virtual environment that includes the virtual object; receiving the plurality of packets streamed from the generating device; receiving a user input to interact with the virtual environment; transmitting the user input to the generating device; selecting, based on the user input, a given rig update from the plurality of packets received from the generating device; and updating the local model according to the given rig update for the virtual object; and wherein the generating device generates the forecasted animation sets via a first graphics engine, and the target device updates the local model via a second graphics engine that is different from the first graphics engine, wherein the target device translates the given rig update from a coordinate system used by the first graphics engine to a coordinate system used by the second graphics engine.