REAL-TIME PROJECTION IN A MIXED REALITY ENVIRONMENT

Number: 20210110598
Published: 2021-04-15
Filed: 2019-10-10
Assignee: Disney Enterprises, Inc.
Inventors: Evans; Leslie M., Scerbo; Siroberto, Carroll; Clare M., Hager, IV; Joseph G., Newberg; Nicholas S., Wieland; Alexis P., Becker; Jonathan
CPC: G06F3/011; G06T19/006; G06T15/50
Verdict: Low Hardware
Source: Google Patents · FreePatentsOnline

The keeper's note

Real-time projection mapping hardware for mixed reality.

Abstract

The present disclosure relates generally to systems and methods for creating a mixed reality environment. A mixed reality system includes a performance area for generating a mixed reality environment, a motion determination module that determines and tracks the motion of an object within the performance area, a physical article module that generates a physical article within the performance area, and a mixed reality display that displays a virtual article within the performance area, wherein the physical article and the virtual article are correlated to the motion of the object.

Background

TECHNICAL FIELD

The technology described herein generally relates systems and methods to project light onto and around objects in real time in a mixed reality environment. BACKGROUND

Traditional mixed reality, virtual reality, and augmented reality environments involve the merging of real and virtual components in a user environment such that virtual and physical objects may coexist and interact. Many mixed reality implementations do not include virtual object to physical object interactions (e.g., virtual light emitted from a virtual object does not wash adjacent physical objects with light), since such interactions are difficult to generate virtually in a realistic manner. In particular, such interactions may require high-resolution three-dimensional scans of the environment that are time and computing resource intensive, as well as modeling of surface reflections, refractions, and other light interactions that are difficult and computationally intensive to determine.

For example, with “see through” mixed reality systems, such as head worn displays that pass through light from the environment, these systems can generally perform “additive effects,” such as adding effects or objects to a scene, but subtractive effects, such as creating shadows, are difficult to produce realistically and typically will not generate downstream or supplemental effects that would be expected in object-to-object interactions in the physical world. Similarly, traditional techn

Claims

1. A mixed reality system comprising: an environment model including data corresponding to a physical environment and a virtual environment; a physical article module that generates a physical article within the physical environment; and a mixed reality display that displays a virtual article within the virtual environment, wherein an actuation of the physical article and an actuation of the virtual article are correlated with one another based on the environment model. || 23. A method of generating a mixed reality environment comprising: determining a position of a physical article within a performance area; determining a position of a virtual article within a virtual environment that overlays the performance area; rendering the virtual article within the virtual environment; rendering the physical article within the performance area; and rendering the virtual article within the virtual environment synchronized with the physical article in the performance area to generate a mixed reality experience for a user visualizing the virtual environment and the performance area. || 28. A method to generate a mixed reality experience comprising: detecting state information corresponding to a physical light source and a virtual object; based on the state information, actuating a light source to project light on to a location within a physical environment; rendering a virtual object within a virtual environment, wherein the virtual object and the light source correspond to one another; and updating the location of the physical light source based on changes to the virtual object.