AR masking rendering technique.
To provide masks for augmented reality (AR) effects in real-time, the embodiments described herein capture a plurality of images of an environment that includes one or more real-world objects after initiating an AR session with an AR device located in the environment. Upon detecting a trigger to generate an AR effect targeting a first real-world object of the one or more real-world objects, a first virtual representation of the first real-world object is generated, based on the plurality of images of the environment, and a second virtual representation of a second real-world object of the one or more real-world objects is generated based on the plurality of images of the environment, wherein the second real-world object is at least partially occluded by the first real-world object.
BACKGROUND
Augmented Reality (AR) uses a video device and a display device to display a series of images of the environment to a user, wherein the images have various virtual objects inserted into appropriate places in the environment. For example, a video device may identify a real-world object of a table, so that a virtual object of a flower vase may be displayed on the display device as appearing on the table. In another example, a video device may identify a real-world object of a trashcan and overlay an image of a monster, so that a user of the display device is provided an amusing scene of a creature devouring rubbish when the mundane scene of a person placing rubbish into a proper receptacle is played out. AR can be used for navigational purposes, for games, construction/architectural projects, for educational purposes, and the like.
An AR system may use an integrated video and display device (or separate video devices and display devices) that output the perspective of the video device. The AR system can use the knowledge and perspective of the video device to inform the output of the display device. For example, a headset display may include an outwardly mounted camera as a video device that provides a video feed to one or more display devices positioned in front of a wearer's eyes. SUMMARY
In one embodiment, a computer-implemented method includes initiating an augmented reality (AR) session within an environment comprising one or more real-world
1. A computer-implemented method, comprising: initiating an augmented reality (AR) session within an environment comprising one or more real-world objects, wherein an environmental map of the environment is unavailable for generating AR effects for the AR session at a time of the initiation of the AR session; after initiating the AR session, detecting at least a first trigger in the environment for capturing a plurality of images of the environment; capturing the plurality of images of the environment in response to detecting at least the first trigger; and upon detecting a second trigger to generate an AR effect targeting a first real-world object of the one or more real-world objects: generating a first virtual representation of the first real-world object, based on the plurality of images of the environment; and generating a second virtual representation of a second real-world object of the one or more real-world objects, based on the plurality of images of the environment, wherein the second real-world object is at least partially occluded by the first real-world object. ||
10. A computer-readable storage medium comprising computer-readable program code embodied therewith, the computer-readable program code is configured to perform, when executed by a processor, an operation, the operation comprising: initiating an augmented reality (AR) session within an environment comprising one or more real-world objects, wherein an environmental map of the environment is unavailable for generating AR effects for the AR session at a time of the initiation of the AR session; after initiating the AR session, detecting at least a first trigger in the environment for capturing a plurality of images of the environment; capturing the plurality of images of the environment in response to detecting at least the first trigger; and upon detecting a second trigger to generate an AR effect targeting a first real-world object of the one or more real-world objects: generating a first virtual representation of the first real-world object, based on the plurality of images of the environment; and generating a second virtual representation of a second real-world object of the one or more real-world objects, based on the plurality of images of the environment, wherein the second real-world object is at least partially occluded by the first real-world object. ||
17. A system, comprising: a processor; and a memory comprising instructions which, when executed on the processor, performs an operation, the operation comprising: initiating an augmented reality (AR) session within an environment comprising one or more real-world objects, wherein an environmental map of the environment is unavailable for generating AR effects for the AR session at a time of the initiation of the AR session; after initiating the AR session, detecting at least a first trigger in the environment for capturing a plurality of images of the environment; capturing the plurality of images of the environment in response to detecting at least the first trigger; and upon detecting a second trigger to generate an AR effect targeting a first real-world object of the one or more real-world objects: generating a first virtual representation of the first real-world object, based on the plurality of images of the environment; and generating a second virtual representation of a second real-world object of the one or more real-world objects, based on the plurality of images of the environment, wherein the second real-world object is at least partially occluded by the first real-world object.