- Number
- 10692288
- Published
- 2020-06-23
- Filed
- 2017-06-27
- Assignee
- LUCASFILM ENTERTAINMENT COMPANY LTD.
- Inventors
- Rasmussen; Nicholas, Koperwas; Michael, Alexander, IV; Earle M.
- CPC
- A63F13/213; G06T15/04; G06F3/011; A63F13/42; G06T19/20; G06T15/60; G06T15/205; G06T19/006
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
AR image compositing/rendering technique.
Abstract
A method may include capturing a first image of a physical environment using a mobile device. The mobile device may include a physical camera and a display. The method may also include receiving a second image from a content provider system. The second image may be generated by the content provider system by rendering a view from a virtual camera in a virtual environment. The virtual environment may represent at least a portion of the physical environment. A location of the virtual camera in the virtual environment may correspond to a location of the physical camera in the physical environment. The second image may include a view of a computer-generated object. The method may additionally include generating a third image by compositing the first image and the second image, and causing the third image to be displayed on the display of the mobile device.
Background
FIELD(1) The present disclosure generally relates to compositing images for usage in augmented reality experiences.BACKGROUND(2) Augmented reality includes a live view of a real-world environment that is augmented by computer generated sensory input(s), such as GPS graphics, video, sound, data statistics, and so forth. In contrast to virtual reality, which replaces the real-world environment with a simulated one, augmented reality elements are often displayed in real time in semantic context with elements of the real-world environment. For example, sports scores can be displayed on a television during a basketball game on a same screen. Head-mounted displays can also be used to place the virtual images over a view of the physical world such that both are in the user's field of view.BRIEF SUMMARY(3) In some embodiments, a method may include capturing a first image of a physical environment using a mobile device. The mobile device may include a physical camera and a display. The method may also include receiving a second image from a content provider system. The second image may be generated by the content provider system by rendering a view from a virtual camera in a virtual environment. The virtual environment may represent at least a portion of the physical environment. A location of the virtual camera in the virtual environment may correspond to a location of the physical camera in the physical environment. The second image may include a view of a computer-generated object.
Claims
1. A method comprising: capturing a first image of a physical environment using a mobile device, wherein the mobile device comprises a physical camera and a display; receiving a second image from a content provider system, wherein: the second image is generated by the content provider system by rendering a view from a virtual camera in a virtual environment, wherein the virtual environment represents at least a portion of the physical environment, and a location of the virtual camera in the virtual environment corresponds to a location of the physical camera in the physical environment; and the second image comprises a view of a computer-generated object comprising a virtual character, wherein: the virtual character is moved according to a live motion-capture performance; a time that the second image is generated in the virtual environment corresponds to a time when the first image is captured in the physical environment, such that when a third image is generated by compositing the first image and the second image, the first image and the second image are synchronized in time; the second image comprises a four-channel image with an alpha channel that indicates portions of the second image that should be at least semi-transparent such that corresponding portions of the first image are at least partially visible as background through the second image; and the portions of the second image that should be at least semi-transparent include a shadow of the computer-generated object; generating the third image by compositing the first image and the second image; and causing the third image to be displayed on the display of the mobile device.
7. A mobile device comprising: a display; a physical camera; one or more processors; and one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: causing the physical camera to capture a first image of a physical environment; receiving a second image from a content provider system, wherein: the second image is generated by the content provider system by rendering a view from a virtual camera in a virtual environment, wherein the virtual environment represents at least a portion of the physical environment, and a location of the virtual camera in the virtual environment corresponds to a location of the physical camera in the physical environment; and the second image comprises a view of a computer-generated object comprising a virtual character, wherein: the virtual character is moved according to a live motion-capture performance; a time that the second image is generated in the virtual environment corresponds to a time when the first image is captured in the physical environment, such that when a third image is generated by compositing the first image and the second image, the first image and the second image are synchronized in time; the second image comprises a four-channel image with an alpha channel that indicates portions of the second image that should be at least semi-transparent such that corresponding portions of the first image are at least partially visible as background through the second image; and the portions of the second image that should be at least semi-transparent include a shadow of the computer-generated object; generating the third image by compositing the first image and the second image; and causing the third image to be displayed on the display.
13. A non-transitory, computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: capturing a first image of a physical environment using a mobile device, wherein the mobile device comprises a physical camera and a display; receiving a second image from a content provider system, wherein: the second image is generated by the content provider system by rendering a view from a virtual camera in a virtual environment, wherein the virtual environment represents at least a portion of the physical environment, and a location of the virtual camera in the virtual environment corresponds to a location of the physical camera in the physical environment; and the second image comprises a view of a computer-generated object comprising a virtual character, wherein: the virtual character is moved according to a live motion-capture performance; a time that the second image is generated in the virtual environment corresponds to a time when the first image is captured in the physical environment, such that when a third image is generated by compositing the first image and the second image, the first image and the second image are synchronized in time; the second image comprises a four-channel image with an alpha channel that indicates portions of the second image that should be at least semi-transparent such that corresponding portions of the first image are at least partially visible as background through the second image; and the portions of the second image that should be at least semi-transparent include a shadow of the computer-generated object; generating the third image by compositing the first image and the second image; and causing the third image to be displayed on the display of the mobile device.