COLLABORATIVE MULTI-MODAL MIXED-REALITY SYSTEM AND METHODS LEVERAGING RECONFIGURABLE TANGIBLE USER INTERFACES FOR THE PRODUCTION OF IMMERSIVE, CINEMATIC, AND INTERACTIVE CONTENT

Number: 20190066387
Published: 2019-02-28
Filed: 2018-02-20
Assignee: Disney Enterprises, Inc.
Inventors: GROSSMANN; Benjamin, LEGATO; Robert
CPC: G06F3/011; G06T19/003; G06F3/0346; G06T19/20; G02B27/017; G06T19/00; G06F3/017
Verdict: Medium Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Collaborative multi-modal mixed-reality tangible UI system (Grossmann).

Abstract

The disclosure is directed to collaborative multi-modal mixed-reality systems and methods leveraging reconfigurable tangible user interfaces for the production of immersive, cinematic, and interactive content. As described herein, content may be created as either a mixture of live action media and computer generated media, combined together in context; or content made up entirely from computer-generated media alone. The technology described herein may leverage physical and software-based user interfaces that merge and unite the physical world and the virtual world, which allow users to produce new forms of content with a user experience and user interactions that emulate traditional processes.

Background

BRIEF SUMMARY OF THE DISCLOSURE

In one embodiment, a method includes: initializing a virtual production application instance on a virtual reality (VR) client device; loading scene files and dependencies at the virtual production application; based on the loaded scene files and dependencies, rendering a virtual world for display by the VR client device; and the VR client device making changes to the scene files using a virtual media production tool rendered in the virtual world or a physical media production tool communicatively coupled to the VR client device.

In some implementations, the method further includes: receiving data corresponding to user input actuating a control of the physical media production tool; and registering the data with the virtual world. In some implementations, receiving data corresponding to user input actuating a control of the physical media production tool and registering the data with the virtual world includes: tracking movement of the physical media production tool using a motion sensor mounted on the physical media production tool; encoding the tracked movement; and transmitting the encoded tracked movement to the VR device for registration with the virtual world.

In some implementations, the VR client device incl

Claims

1. A method, comprising: initializing a virtual production application instance on a virtual reality (VR) client device; loading scene files and dependencies at the virtual production application; based on the loaded scene files and dependencies, rendering a virtual world for display by the VR client device; and the VR client device making changes to the scene files using a virtual media production tool rendered in the virtual world or a physical media production tool communicatively coupled to the VR client device. 13. A system, comprising: a virtual reality (VR) head-mounted display (HMD); one or more non-transitory computer-readable mediums having instructions stored thereon that, when executed by one or more processors, causes the system to: initialize a virtual production application instance; load scene files and dependencies at the virtual production application; based on the loaded scene files and dependencies, render a virtual world for display by the VR HMD; and the VR HMD making changes to the scene files using a virtual media production tool rendered in the virtual world or a physical media production tool communicatively coupled to the VR HMD. 21. A method, comprising: consolidating scene files and dependencies at a master media production device; transmitting the consolidated scene files and dependencies from the master media production device to each of a plurality of virtual reality (VR) client media production devices, wherein the plurality of VR client media production devices comprise a plurality of VR head-mounted displays (HMD); transmitting the consolidated scene files and dependencies from the master media production device to a server; each of the plurality of VR client media production devices loading the consolidated scene files and dependencies to a render a virtual world, wherein a view of the rendered virtual world is displayed by each of the plurality of VR HMD; the server synchronizing across the plurality of VR client media production devices, any changes made to the consolidated scene files and dependencies by one of the plurality of VR client media production devices.