- Number
- 11042747
- Published
- 2021-06-22
- Filed
- 2019-10-02
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Goslin; Michael P., Baumbach; Elliott H.
- CPC
- G06V20/20; G06F3/03; G06F3/011; G06T19/006; G06F3/04815; G06F3/012
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
AR masking rendering technique (granted).
Abstract
To provide masks for augmented reality (AR) effects in real-time, the embodiments described herein capture a plurality of images of an environment that includes one or more real-world objects after initiating an AR session with an AR device located in the environment. Upon detecting a trigger to generate an AR effect targeting a first real-world object of the one or more real-world objects, a first virtual representation of the first real-world object is generated, based on the plurality of images of the environment, and a second virtual representation of a second real-world object of the one or more real-world objects is generated based on the plurality of images of the environment, wherein the second real-world object is at least partially occluded by the first real-world object.
Background
BACKGROUND (1) Augmented Reality (AR) uses a video device and a display device to display a series of images of the environment to a user, wherein the images have various virtual objects inserted into appropriate places in the environment. For example, a video device may identify a real-world object of a table, so that a virtual object of a flower vase may be displayed on the display device as appearing on the table. In another example, a video device may identify a real-world object of a trashcan and overlay an image of a monster, so that a user of the display device is provided an amusing scene of a creature devouring rubbish when the mundane scene of a person placing rubbish into a proper receptacle is played out. AR can be used for navigational purposes, for games, construction/architectural projects, for educational purposes, and the like. (2) An AR system may use an integrated video and display device (or separate video devices and display devices) that output the perspective of the video device. The AR system can use the knowledge and perspective of the video device to inform the output of the display device. For example, a headset display may include an outwardly mounted camera as a video device that provides a video feed to one or more display devices positioned in front of a wearer's eyes. SUMMARY (3) In one embodiment, a computer-implemented method includes initiating an augmented reality (AR) session within an environment comprising one or more real-world objects,
Claims
1. A computer-implemented method, comprising: initiating an augmented reality (AR) session within an environment comprising one or more real-world objects, wherein an environmental map of the environment is unavailable for generating AR effects for the AR session at a time of the initiation of the AR session; after initiating the AR session, capturing a plurality of images of the environment upon detecting at least a first trigger in the environment; and upon detecting a second trigger to generate an AR effect targeting a first real-world object of the one or more real-world objects: generating a first virtual representation of the first real-world object, based on the plurality of images; upon determining that image data of a second real-world object of the one or more real-world objects partially occluded by the first real-world object is unavailable within the plurality of images: identifying at least one third real-world object of the one or more real-world objects behind the first real-world object, based on the plurality of images; determining one or more first attributes of the at least one third real-world object; and predicting one or more second attributes of the second real-world object based on the one or more first attributes; and generating a second virtual representation of the second real-world object, based on the plurality of images and the one or more second attributes. ||
8. A non-transitory computer-readable storage medium comprising computer-readable program code embodied therewith, the computer-readable program code is configured to perform, when executed by a processor, an operation, the operation comprising: initiating an augmented reality (AR) session within an environment comprising one or more real-world objects, wherein an environmental map of the environment is unavailable for generating AR effects for the AR session at a time of the initiation of the AR session; after initiating the AR session, capturing a plurality of images of the environment upon detecting at least a first trigger in the environment; and upon detecting a second trigger to generate an AR effect targeting a first real-world object of the one or more real-world objects: generating a first virtual representation of the first real-world object, based on the plurality of images; upon determining that image data of a second real-world object of the one or more real-world objects partially occluded by the first real-world object is unavailable within the plurality of images: identifying at least one third real-world object of the one or more real-world objects behind the first real-world object, based on the plurality of images; determining one or more first attributes of the at least one third real-world object; and predicting one or more second attributes of the second real-world object based on the one or more first attributes; and generating a second virtual representation of the second real-world object, based on the plurality of images and the one or more second attributes. ||
13. A system, comprising: at least one visual sensor; a processor; and a memory comprising instructions which, when executed on the processor, performs an operation, the operation comprising: initiating an augmented reality (AR) session within an environment comprising one or more real-world objects, wherein an environmental map of the environment is unavailable for generating AR effects for the AR session at a time of the initiation of the AR session; after initiating the AR session, capturing, via the at least one visual sensor, a plurality of images of the environment upon detecting at least a first trigger in the environment; and upon detecting a second trigger to generate an AR effect targeting a first real-world object of the one or more real-world objects: generating a first virtual representation of the first real-world object, based on the plurality of images; upon determining that image data of a second real-world object of the one or more real-world objects partially occluded by the first real-world object is unavailable within the plurality of images: identifying at least one third real-world object of the one or more real-world objects behind the first real-world object, based on the plurality of images; determining one or more first attributes of the at least one third real-world object; and predicting one or more second attributes of the second real-world object based on the one or more first attributes; and generating a second virtual representation of the second real-world object, based on the plurality of images and the one or more second attributes.