Granted patent
Ambient light characterization
- Number
- 10438404
- Published
- 2019-10-08
- Filed
- 2017-10-13
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Trowbridge; Robert Scott
- CPC
- G06T15/80; H04B10/116; G06T15/506; G06T19/006
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Ambient light characterization grant dup.
Abstract
Embodiments herein describe an AR system which communicates with light sources in a real world environment to simulate the light sources having an effect on AR content being displayed in the AR system. In one embodiment, the AR system determines the light characteristics of the light sources such as their color, specularity, beam spread, brightness, or illumination pattern. These light characteristics can contribute to the ambient light in the real world environment which affects the appearance of the physical objects in the environment. The AR system can adjust how the AR content is displayed to simulate the AR content being affected in a similar manner by the light sources. As such, the AR content appears to be more like a physical object (rather than a virtual object) thereby providing the user with a more immersive experience.
Background
BACKGROUNDField of the Invention(1) The embodiments herein are generally directed to characterizing an effect of a light source in a real world environment on augmented reality (AR) content displayed using an AR system.Description of the Related Art(2) AR displays insert virtual content into real-world views of the user's environment. For example, a user may wear a headset that captures views of the real world and includes a lens and a display device that outputs AR content. For example, the AR content may include a virtual object such as a cartoon character that is inserted into the real world environment. The AR display can cause the cartoon character to move and interact with physical objects in the real world environment such as standing on a table, climbing on a box, etc.(3) In some AR systems, the appearance of the AR content is improved by first occluding a portion of the real-world image before combining the real-world image and the AR content. For example, the AR display can control a first display to generate an opaque portion with the same outline as the cartoon character. Because the AR content is combined with the real-world image at the location of the black silhouette, the appearance of the AR content is more vibrant, high-contrast, and can include the color black. Although this technique can make the AR content appear more realistic, it does not consider the ambient light. For example, a light source in the real world environment can be a colored light source
Claims
1. An augmented reality (AR) system, comprising: a display configured to output AR content to overlay a portion of a real world environment viewable through the display; a visible light communication (VLC) communicator configured to receive VLC signals transmitted from a VLC light source in the real world environment, wherein the VLC signals comprise data indicating light source characteristics of the VLC light source; and an AR generator configured to: determine, based on the light source characteristics, display characteristics for the AR content to simulate an effect of the VLC light source on the AR content; and identify a respective location of a plurality of VLC light sources, which includes the VLC light source, in the real world environment, wherein the display characteristics for the AR content are determined based on a distance between a simulated location of the AR content in the real world environment and the respective location of at least one of the plurality of VLC light sources.
2. The AR system of claim 1, wherein the VLC communicator is configured to: transmit VLC signals to a plurality of VLC light sources in the real world environment to repeatedly turn off all but one of the plurality of VLC light sources until each of the plurality of VLC light sources has been turned on in isolation, wherein the AR system is configured to identify an individual contribution of the VLC light sources on ambient light in the real world environment.
3. The AR system of claim 2, wherein the AR generator is configured to: identify a shadow caused by at least one of the plurality of VLC light sources in the real world environment; and upon determining a simulated location of the AR content is within a location of the shadow in the real world environment, update the display characteristics for the AR content to simulate an effect of the at least one of the plurality of VLC light sources on the AR content.
4. The AR system of claim 1, wherein the light source characteristics comprises a flickering pattern performed by the VLC light source, wherein simulating the effect of the VLC light source on the AR content comprises: changing the display characteristics for the AR content in tandem with the flickering pattern of the VLC light source.
5. The AR system of claim 1, wherein the AR generator is configured to: identify a lighting action in the AR content, and determine a lighting parameter for the VLC light source to simulate an effect of the lighting action on a physical object in the real world environment, wherein the VLC communicator is configured to transmit VLC signals to the VLC light source to change the light source characteristics of the VLC light source based on the lighting parameter to simulate the lighting action in the real world environment.
6. The AR system of claim 5, wherein the lighting action is an animation outputted in the display.
7. A non-transitory computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code, when executed by a processor, performs an operation, the operation comprising: outputting AR content in a display, wherein the AR content overlays a portion of a real world environment viewable through the display; receiving data transmitted from a VLC light source in the real world environment using VLC signals, wherein the data comprises light source characteristics of the VLC light source; determining, based on the light source characteristics, display characteristics for the AR content to simulate an effect of the VLC light source on the AR content; and identifying a respective location of a plurality of VLC light sources, which includes the VLC light source, in the real world environment, wherein the display characteristics for the AR content are determined based on a distance between a simulated location of the AR content in the real world environment and the respective location of at least one of the plurality of VLC light sources.
8. The computer-readable storage medium of claim 7, wherein the operation comprises: transmitting VLC signals to a plurality of VLC light sources in the real world environment to repeatedly turn off all but one of the plurality of VLC light sources until each of the plurality of VLC light sources has been turned on in isolation; and identifying an individual contribution of the VLC light sources on ambient light in the real world environment.
9. The computer-readable storage medium of claim 8, wherein the operation comprises: identify a shadow caused by at least one of the plurality of VLC light sources in the real world environment; and upon determining a simulated location of the AR content is within a location of the shadow in the real world environment, updating the display characteristics for the AR content to simulate an effect of the at least one of the plurality of VLC light sources on the AR content.
10. The computer-readable storage medium of claim 7, wherein the light source characteristics comprises a flickering pattern performed by the VLC light source, wherein simulating the effect of the VLC light source on the AR content comprises: changing the display characteristics for the AR content in tandem with the flickering pattern of the VLC light source.
11. The computer-readable storage medium of claim 7, wherein the operation comprises: identifying a lighting action in the AR content, and determining a lighting parameter for the VLC light source to simulate an effect of the lighting action on a physical object in the real world environment, transmitting VLC signals to the VLC light source to change the light source characteristics of the VLC light source based on the lighting parameter to simulate the lighting action in the real world environment.
12. The computer-readable storage medium of claim 11, wherein the lighting action is an animation outputted in the display.
13. A communication system, comprising: an intelligent light source; and an AR system, comprising: a display configured to output AR content to overlay a portion of a real world environment viewable through the display; a communicator configured to receive signals transmitted from the intelligent light source, wherein the signals comprise data indicating a light source characteristic of the intelligent light source; and an AR generator configured to: determine, based on the light source characteristic, display characteristics for the AR content to simulate an effect of the intelligent light source on the AR content; and identify a respective location of a plurality of intelligent light sources, which includes the intelligent light source, in the real world environment, wherein the display characteristics for the AR content are determined based on a distance between a simulated location of the AR content in the real world environment and the respective location of at least one of the plurality of intelligent light sources.
14. The communication system of claim 13, wherein the intelligent light source is configured to use VLC to transmit the signals to the AR system.
15. The communication system of claim 13, wherein the intelligent light source is configured to use an Internet of Things (IoT) communication protocol to transmit the signals to the AR system.
16. The communication system of claim 13, wherein the communicator is configured to: transmit signals to a plurality of intelligent light sources in the real world environment to repeatedly turn off all but one of the plurality of intelligent light sources until each of the plurality of intelligent light sources has been turned on in isolation, wherein the AR system is configured to identify an individual contribution of the intelligent light sources on ambient light in the real world environment.
17. The communication system of claim 13, wherein the AR generator is configured to: identify a lighting action in the AR content, and determine a lighting parameter for the intelligent light source to simulate an effect of the lighting action on a physical object in the real world environment, wherein the communicator is configured to transmit signals to the intelligent light source to change the light source characteristic of the intelligent light source based on the lighting parameter to simulate the lighting action in the real world environment.
18. An augmented reality (AR) system, comprising: a display configured to output AR content to overlay a portion of a real world environment viewable through the display; a visible light communication (VLC) communicator configured to receive VLC signals transmitted from a VLC light source in the real world environment, wherein the VLC signals comprise data indicating light source characteristics of the VLC light source; and an AR generator configured to: determine, based on the light source characteristics, display characteristics for the AR content to simulate an effect of the VLC light source on the AR content; identify a lighting action in the AR content; and determine a lighting parameter for the VLC light source to simulate an effect of the lighting action on a physical object in the real world environment, wherein the VLC communicator is configured to transmit VLC signals to the VLC light source to change the light source characteristics of the VLC light source based on the lighting parameter to simulate the lighting action in the real world environment.