- Number
- 11087529
- Published
- 2021-08-10
- Filed
- 2019-09-27
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Yeung; Jason A., Mitchell; Kenneth J., Panec; Timothy M., Baumbach; Elliott H., Drake; Corey D.
- CPC
- G06T15/80; G06T17/00; G02B27/0172; G06T7/13; G06T15/506; G06T19/006; G06T15/005
- Verdict
- High Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Real-time AR lighting rendering illuminating physical objects (granted).
Abstract
Embodiments provide for the rendering of illumination effects on real-world objects in augmented reality systems. An example method generally includes overlaying a shader on the augmented reality display. The shader generally corresponds to a three-dimensional geometry of an environment in which the augmented reality display is operating, and the shader generally comprises a plurality of vertices forming a plurality of polygons. A computer-generated lighting source is introduced into the augmented reality display. One or more polygons of the shader are illuminated based on the computer-generated lighting source, thereby illuminating one or more real-world objects in the environment with direct lighting from the computer-generated lighting source and reflected and refracted lighting from surfaces in the environment.
Background
BACKGROUND Field (1) Aspects of the present disclosure relate to augmented reality displays, and more specifically to rendering lighting effects on real-world objects in an augmented reality display. Description of the Related Art (2) Head-mounted augmented reality devices (also referred to as head-mounted displays, or “HMDs”) generally are display devices that are worn on a user's head and display, to the wearer of the device, an image of the real-world environment in which the HMD is being used with additional information superimposed on the image of the real-world environment. Some of these devices may be opaque to the outside world. Using a camera coupled with the HMD, these devices generally capture images of the real-world environment in which the HMD is being used and combine these captured images with augmented reality content rendered in a layer with a transparent background (e.g., a layer with a transparent alpha channel). Other devices, known as see-through HMDs, may have semi-transparent lenses and a projector configured to display augmented reality content on the semi-transparent lenses, which may allow a user to view the real-world environment in which the HMD is being used with additional content superimposed over the real-world environment. (3) The real-world environment in which an HMD is being used generally includes a variety of objects made from different materials. Some of these materials may have low reflectivity (e.g., having matte finishes or other non
Claims
1. A method for rendering lighting effects in an augmented reality display, comprising: overlaying, on an augmented reality display, a shader corresponding to a three-dimensional geometry of an environment in which the augmented reality display is operating, wherein the shader comprises a plurality of vertices forming a plurality of polygons, a flat texture on which lighting effects can be rendered, and a plurality of vectors, each vector defining a light reflectivity value corresponding to a respective surface in the environment in which the augmented reality display is operating; introducing a computer-generated lighting source into the augmented reality display, wherein the computer-generated lighting source comprises a light source that is absent from the environment in which the augmented reality display is operating and is rendered as an object overlaid in the environment in which the augmented reality display is operating; and illuminating one or more polygons of the shader, thereby illuminating, in the augmented reality display, one or more real-world objects in the environment with direct lighting from the computer-generated lighting source and reflected and refracted lighting from surfaces in the environment wherein illuminating the one or more polygons comprises generating an illumination effect for at least one of the one or more real-world objects in the environment in which the augmented reality display is operating such that illumination effects against objects with higher reflectivity surfaces appear brighter than illuminations against objects with lower reflectivity surfaces. ||
11. A system, comprising: a processor; and a memory having instructions stored thereon which, when executed by the processor, performs an operation for rendering lighting effects in an augmented reality display, the operation comprising: overlaying, on an augmented reality display, a shader corresponding to a three-dimensional geometry of an environment in which the augmented reality display is operating, wherein the shader comprises a plurality of vertices forming a plurality of polygons, a flat texture on which lighting effects can be rendered, and a plurality of vectors, each vector defining a light reflectivity value corresponding to a respective surface in the environment in which the augmented reality display is operating; introducing a computer-generated lighting source into the augmented reality display, wherein the computer-generated lighting source comprises a light source that is absent from the environment in which the augmented reality display is operating and is rendered as an object overlaid in the environment in which the augmented reality display is operating; and illuminating one or more polygons of the shader, thereby illuminating, in the augmented reality display, one or more real-world objects in the environment with direct lighting from the computer-generated lighting source and reflected and refracted lighting from surfaces in the environment wherein illuminating the one or more polygons comprises generating an illumination effect for at least one of the one or more real-world objects in the environment in which the augmented reality display is operating such that illumination effects against objects with higher reflectivity surfaces appear brighter than illuminations against objects with lower reflectivity surfaces. ||
20. A head-mounted display for rendering augmented reality experiences, comprising: a non-blocking display screen through which an environment in which the head-mounted display is operating is visible; and a system for rendering content on the non-blocking display screen, the system being configured to: overlay, on the non-blocking display screen, a shader corresponding to a three-dimensional geometry of the environment in which the head-mounted display is operating, wherein the shader comprises a plurality of vertices forming a plurality of polygons, a flat texture on which lighting effects can be rendered, and a plurality of vectors, each vector defining a light reflectivity value corresponding to a respective surface in the environment in which the head-mounted display is operating; rendering a computer-generated lighting source on the non-blocking display screen, wherein the computer-generated lighting source comprises a light source that is absent from the environment in which the head-mounted display is operating and is rendered as an object overlaid in the environment in which the head-mounted display is operating; and illuminating one or more polygons of the shader, thereby illuminating, on the non-blocking display screen, one or more real-world objects in the environment with direct lighting from the computer-generated lighting source and reflected and refracted lighting from surfaces in the environment rendered on the non-blocking display screen, wherein illuminating the one or more polygons comprises generating an illumination effect for at least one of the one or more real-world objects in the environment in which the head-mounted display is operating such that illumination effects against objects with higher reflectivity surfaces appear brighter than illuminations against objects with lower reflectivity surfaces.