Animation streaming technique for interactive media.
Embodiments provide for animation streaming for media interaction by receiving, at a generator, inputs from a target device presenting of a virtual environment; updating, based on the user inputs, a model of the virtual environment; determining network conditions between the generator and target device; generating a packet that includes a forecasted animation set for a virtual object in the updated model that comprises rig updates for the virtual object for at least two different states, and a number of states included in the packet is based on the network conditions; and streaming the packet to the target device, where the target device: receives a second input to interact with the virtual environment that changes the virtual environment to a given state; selects and applies a rig update associated with the given state a local model of the virtual object; and outputs the updated local model on the target device.
BACKGROUND
The animation of immersive content, such as virtual objects in a video game, Augmented Reality (AR) application, Virtual Reality (VR) application, or other context includes providing a sequence of images for display to a user. When animated, consecutive images of the virtual objects provide changes in the appearance and/or position of the virtual objects in which the virtual objects may be repositioned within an environment (either a purely virtual environment or a virtual environment mapped to a real-world environment), modeled in a different pose, affected by changing lighting/texturing conditions, or moved into/out of a frame of view for the user. Users increasingly desire that the animations be provided at a consistent rate between the various animated virtual objects, at a steady rate of animation (e.g., avoiding jitter in animating the virtual objects), a high rate of refresh (e.g., at least X Frames per Second (FPS)), and at a high level of graphical detail.SUMMARY
The present disclosure, in one embodiment, provides a method of animation streaming for media interaction, comprising: receiving, at a generator, user inputs from a target device displaying a presentation of a virtual environment; updating, at the generator based on the user inputs, an environmental model of the virtual environment; determining network conditions between the generator and the target device; generating a packet that includes a forecasted animation set for a virtual obje
1. A method, comprising: receiving, at a generator, user inputs from a target device displaying a presentation of a virtual environment; updating, at the generator based on the user inputs, an environmental model of the virtual environment; determining network conditions between the generator and the target device; generating a packet that includes a forecasted animation set for a virtual object in the updated environmental model, wherein the forecasted animation set comprises a plurality of rig updates for the virtual object for at least two different states of the environmental model, and a number of states comprising the at least two states is based on the network conditions; and streaming the packet to the target device.
9. A method, comprising: receiving, at a target device displaying a virtual environment, a packet stream from a remote generating device, wherein each packet in the packet stream includes a forecasted animation set for a virtual object included in the virtual environment, wherein the forecasted animation set comprises a first rig update for a first state of the virtual environment at a subsequent time and a second rig update for a second state of the virtual environment at the subsequent time; receiving, at the target device, a user input to interact with the virtual environment that changes the virtual environment to a given state of the first state and the second state; selecting a given rig update from the packet stream associated with the given state; applying the given rig update for the virtual object to a local model of the virtual object displayed by the target device; outputting, for the subsequent time, the local model on the target device according to the given rig update; and transmitting the user input to the remote generating device.
18. A system, comprising: a generating device, including a generating processor and a generating memory including generating instructions that when performed by the generating processor enable the generating device to perform an animation operation comprising: generating a plurality of packets, wherein each packet of the plurality of packets includes a forecasted animation set for a virtual object, wherein the forecasted animation set comprises a plurality of rig updates for the virtual object for at least two different states of a virtual environment for one time; adjusting a number of rig updates in the forecasted animation set of a given packet based on at least one of a latency and an available bandwidth of a network over which the plurality of packets are streamed; and streaming the plurality of packets via the network; and a target device, in communication with the generating device via the network, the target device including a target processor and a target memory including target instructions that when performed by the target processor enable the target device to perform an operation comprising: displaying a local model of the virtual environment that includes the virtual object; receiving the plurality of packets streamed from the generating device; receiving a user input to interact with the virtual environment; transmitting the user input to the generating device; selecting, based on the user input, a given rig update from the plurality of packets received from the generating device; and updating the local model according to the given rig update for the virtual object.