- Number
- 10796489
- Published
- 2020-10-06
- Filed
- 2018-09-13
- Assignee
- Lucasfilm Entertainment Company Ltd.
- Inventors
- Cordes; Roger, Brickhill; David
- CPC
- A63F13/213; G06T19/20; G02B27/017; A63F13/60; A63F13/35; G06T15/08; G06T7/73; G06V40/20; A63F13/25; G06F3/012; G06T13/40; G06V40/23; H04L65/611; A63F13/65; A63F13/53; G06T7/596; G06T19/006; G06F3/011; G06T7/246; A63F13/655
- Verdict
- Medium Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Motion-capture-driven game engine for mixed-reality environments.
Abstract
An immersive content presentation system can capture the motion or position of a performer in a real-world environment. A game engine can be modified to receive the position or motion of the performer and identify predetermined gestures or positions that can be used to trigger actions in a 3-D virtual environment, such as generating a digital effect, transitioning virtual assets through an animation graph, adding new objects, and so forth. The use of the 3-D environment can be rendered and composited views can be generated. Information for constructing the composited views can be streamed to numerous display devices in many different physical locations using a customized communication protocol. Multiple real-world performers can interact with virtual objects through the game engine in a shared mixed-reality experience.
Background
TECHNICAL FIELD(1) This application discloses technology related to the fields of computer animation, virtual reality environments, and digital content generation. Specifically, this application discloses technology for using a real-time gestures to drive computer-generated assets in a multi-device viewing environment.BACKGROUND(2) Augmented reality includes a live view of a real-world environment that is augmented by computer generated sensory input(s), such as GPS graphics, video, sound, data statistics, and so forth. In contrast to virtual reality, which replaces the real-world environment with a simulated one, augmented reality elements are often displayed in real time in semantic context with elements of the real-world environment. For example, sports scores can be displayed on a television during a basketball game on a same screen. Headmounted displays can also be used to place the virtual images over a view of the physical world such that both are in the user's field of view.(3) Virtual reality is very similar to augmented reality in that it entails an interactive computer generated experience. However, virtual reality takes place within a simulated, immersive environment. Current virtual reality technology commonly uses virtual-reality headset or multi-projected environments to create realistic sensations, sales, and images that simulate the user having a physical presence in the virtual or imaginary environment. This effect is usually implemented using a virtual-real
Claims
1. A method comprising: causing, by a game engine, a virtual object in a 3-D virtual environment to be in a first animation state in an animation graph; receiving, by a plug-in of the game engine, one or more 3-D frames from a motion-capture camera system, wherein the plug-in is configured to: receive 3-D frames from the motion-capture camera system as an input; identify a motion or position of a performer in real time in a first real-world environment represented in the one or more 3-D frames; and provide the motion or position of a performer as an input to the game engine; maintaining a library of predefined motions or positions, wherein each of the predefined motions or positions includes a corresponding virtual asset or action; identifying, by the game engine, the motion or position as one of the predefined motions or positions in the library; and causing, by the game engine, the virtual object in the 3-D virtual environment to transition to a second animation state in the animation graph based on the corresponding action in the library, wherein the transition is triggered by identifying the motion or position as the one of the predefined motions or positions in the library.
7. A system comprising: one or more processors; and one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: causing, by a game engine, a virtual object in a 3-D virtual environment to be in a first animation state in an animation graph; receiving, by a plug-in of the game engine, one or more 3-D frames from a motion-capture camera system, wherein the plug-in is configured to: receive 3-D frames from the motion-capture camera system as an input; identify a motion or position of a performer in real time in a first real-world environment represented in the one or more 3-D frames; and provide the motion or position of a performer as an input to the game engine; maintaining a library of predefined motions or positions, wherein each of the predefined motions or positions includes a corresponding virtual asset or action; identifying, by the game engine, the motion or position as one of the predefined motions or positions in the library; and causing, by the game engine, the virtual object in the 3-D virtual environment to transition to a second animation state in the animation graph based on the corresponding action in the library, wherein the transition is triggered by identifying the motion or position as the one of the predefined motions or positions in the library.
14. A non-transitory, computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: causing, by a game engine, a virtual object in a 3-D virtual environment to be in a first animation state in an animation graph; receiving, by a plug-in of the game engine, one or more 3-D frames from a motion-capture camera system, wherein the plug-in is configured to: receive 3-D frames from the motion-capture camera system as an input; identify a motion or position of a performer in real time in a first real-world environment represented in the one or more 3-D frames; and provide the motion or position of a performer as an input to the game engine; maintaining a library of predefined motions or positions, wherein each of the predefined motions or positions includes a corresponding virtual asset or action; identifying, by the game engine, the motion or position as one of the predefined motions or positions in the library; and causing, by the game engine, the virtual object in the 3-D virtual environment to transition to a second animation state in the animation graph based on the corresponding action in the library, wherein the transition is triggered by identifying the motion or position as the one of the predefined motions or positions in the library.