INTRODUCING REAL-TIME LIGHTING EFFECTS TO ILLUMINATE REAL-WORLD PHYSICAL OBJECTS IN SEE-THROUGH AUGMENTED REALITY DISPLAYS

Number: 20210097757
Published: 2021-04-01
Filed: 2019-09-27
Assignee: Disney Enterprises, Inc.
Inventors: YEUNG; Jason A., MITCHELL; Kenneth J., PANEC; Timothy M., BAUMBACH; Elliott H., DRAKE; Corey D.
CPC: G06T15/005; G06T17/00; G06T15/80; G02B27/0172; G06T19/006; G06T15/506; G06T7/13
Verdict: High Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Real-time AR lighting rendering illuminating physical objects.

Abstract

Embodiments provide for the rendering of illumination effects on real-world objects in augmented reality systems. An example method generally includes overlaying a shader on the augmented reality display. The shader generally corresponds to a three-dimensional geometry of an environment in which the augmented reality display is operating, and the shader generally comprises a plurality of vertices forming a plurality of polygons. A computer-generated lighting source is introduced into the augmented reality display. One or more polygons of the shader are illuminated based on the computer-generated lighting source, thereby illuminating one or more real-world objects in the environment with direct lighting from the computer-generated lighting source and reflected and refracted lighting from surfaces in the environment.

Background

BACKGROUND Field

Aspects of the present disclosure relate to augmented reality displays, and more specifically to rendering lighting effects on real-world objects in an augmented reality display. Description of the Related Art

Head-mounted augmented reality devices (also referred to as head-mounted displays, or “HMDs”) generally are display devices that are worn on a user's head and display, to the wearer of the device, an image of the real-world environment in which the HMD is being used with additional information superimposed on the image of the real-world environment. Some of these devices may be opaque to the outside world. Using a camera coupled with the HMD, these devices generally capture images of the real-world environment in which the HMD is being used and combine these captured images with augmented reality content rendered in a layer with a transparent background (e.g., a layer with a transparent alpha channel). Other devices, known as see-through HMDs, may have semi-transparent lenses and a projector configured to display augmented reality content on the semi-transparent lenses, which may allow a user to view the real-world environment in which the HMD is being used with additional content superimposed over the real-world environment.

The real-world environment in which an HMD is being used generally includes a variety of objects made from different materials. Some of these materials may have low reflectivity (e.g., having matte finishes or

Claims

1. A method for rendering lighting effects in an augmented reality display, comprising: overlaying, on an augmented reality display, a shader corresponding to a three-dimensional geometry of an environment in which the augmented reality display is operating, the shader comprising a plurality of vertices forming a plurality of polygons and a flat texture on which lighting effects can be rendered; introducing a computer-generated lighting source into the augmented reality display; and illuminating one or more polygons of the shader, thereby illuminating one or more real-world objects in the environment with direct lighting from the computer-generated lighting source and reflected and refracted lighting from surfaces in the environment. || 11. A system, comprising: a processor; and a memory having instructions stored thereon which, when executed by the processor, performs an operation for rendering lighting effects in an augmented reality display, the operation comprising: overlaying, on an augmented reality display, a shader corresponding to a three-dimensional geometry of an environment in which the augmented reality display is operating, the shader comprising a plurality of vertices forming a plurality of polygons and a flat texture on which lighting effects can be rendered; introducing a computer-generated lighting source into the augmented reality display; and illuminating one or more polygons of the shader, thereby illuminating one or more real-world objects in the environment with direct lighting from the computer-generated lighting source and reflected and refracted lighting from surfaces in the environment. || 20. A head-mounted display for rendering augmented reality experiences, comprising: a non-blocking display screen in which an environment in which the head-mounted display is visible through the non-blocking display screen; and a system for rendering content on the non-blocking display, the system being configured to: overlay, on the non-blocking display screen, a shader corresponding to a three-dimensional geometry of an environment in which the augmented reality display is operating, the shader comprising a plurality of vertices forming a plurality of polygons and a flat texture on which lighting effects can be rendered; rendering a computer-generated lighting source on the non-blocking display; and illuminating one or more polygons of the shader, thereby illuminating one or more real-world objects in the environment with direct lighting from the computer-generated lighting source and reflected and refracted lighting from surfaces in the environment rendered on the non-blocking display.