Collaborative multi-modal mixed-reality system and methods leveraging reconfigurable tangible user interfaces for the production of immersive, cinematic, and interactive content

Number: 10565802
Published: 2020-02-18
Filed: 2018-02-20
Assignee: Disney Enterprises, Inc.
Inventors: Grossmann; Benjamin, Legato; Robert
CPC: G06F3/017; G06F3/011; G06T19/00; G06T19/003; G06T19/20; G02B27/017; G06F3/0346
Verdict: Low Hardware
Source: Google Patents · FreePatentsOnline

The keeper's note

Tangible physical UI hardware for mixed-reality production.

Abstract

The disclosure is directed to collaborative multi-modal mixed-reality systems and methods leveraging reconfigurable tangible user interfaces for the production of immersive, cinematic, and interactive content. As described herein, content may be created as either a mixture of live action media and computer generated media, combined together in context; or content made up entirely from computer-generated media alone. The technology described herein may leverage physical and software-based user interfaces that merge and unite the physical world and the virtual world, which allow users to produce new forms of content with a user experience and user interactions that emulate traditional processes.

Background

BRIEF SUMMARY OF THE DISCLOSURE(1) The disclosure is directed to collaborative multi-modal mixed-reality systems and methods leveraging reconfigurable tangible user interfaces for the production of immersive, cinematic, and interactive content.(2) In one embodiment, a method includes: initializing a virtual production application instance on a virtual reality (VR) client device; loading scene files and dependencies at the virtual production application; based on the loaded scene files and dependencies, rendering a virtual world for display by the VR client device; and the VR client device making changes to the scene files using a virtual media production tool rendered in the virtual world or a physical media production tool communicatively coupled to the VR client device.(3) In some implementations, the method further includes: receiving data corresponding to user input actuating a control of the physical media production tool; and registering the data with the virtual world. In some implementations, receiving data corresponding to user input actuating a control of the physical media production tool and registering the data with the virtual world includes: tracking movement of the physical media production tool using a motion sensor mounted on the physical media production tool; encoding the tracked movement; and transmitting the encoded tracked movement to the VR device for registration with the virtual world.(4) In some implementations, the VR client device includes a VR he

Claims

1. A method, comprising: initializing a virtual production application instance on a virtual reality (VR) client device; loading scene files and dependencies at the virtual production application; based on the loaded scene files and dependencies, rendering a virtual world for display by the VR client device; and the VR client device making changes to the scene files using a virtual media production tool rendered in the virtual world, wherein the virtual media production tool corresponds to a physical media production tool communicatively coupled to the VR client device. 13. A system, comprising: a virtual reality (VR) client device; one or more non-transitory computer-readable mediums having instructions stored thereon that, when executed by one or more processors, causes the system to: initialize a virtual production application instance; load scene files and dependencies at the virtual production application; based on the loaded scene files and dependencies, render a virtual world for display by the VR client device; and the VR client device making changes to the scene files using a virtual media production tool rendered in the virtual world, wherein the virtual media production tool corresponds to a physical media production tool communicatively coupled to the VR client device. 21. A method, comprising: consolidating scene files and dependencies at a master media production device; transmitting the consolidated scene files and dependencies from the master media production device to each of a plurality of virtual reality (VR) client devices; transmitting the consolidated scene files and dependencies from the master media production device to a server; loading, at each of the plurality of VR client devices, the consolidated scene files and dependencies to render a virtual world, wherein a view of the rendered virtual world is displayed by each of the plurality of VR client devices; and synchronizing, via the server, across the plurality of VR client devices, any changes made to the consolidated scene files and dependencies by one of the plurality of VR client devices.