- Number
- 10706634
- Published
- 2020-07-07
- Filed
- 2019-07-16
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Baumbach; Elliott, Nocon; Nathan, Gibson; Hunter
- CPC
- G06T19/006; G06V20/20; H04W4/021; H04W4/40
- Verdict
- Low Hardware
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Drone-based perspective AR content generation.
Abstract
A system has an augmented reality device and an unmanned aerial vehicle. The augmented reality device captures, from a perspective view of the augmented reality device, real-world imagery of a user within an augmented reality environment. Further, the augmented reality device generates virtual imagery to overlay the real-world imagery captured by the augmented reality image capture device from the perspective view of the augmented reality device. Finally, the augmented reality device determines a position and an orientation of an augmented reality device accessory within a common coordinate space. Moreover, the unmanned aerial vehicle captures, from a perspective view of the unmanned aerial vehicle, real-world imagery of the user within the augmented reality environment. The unmanned aerial vehicle determines a position of the unmanned aerial vehicle within the common coordinate space. Further, the unmanned aerial vehicle generates virtual imagery to overlay the real-world imagery captured by the unmanned aerial vehicle.
Background
BACKGROUND1. Field(1) This disclosure generally relates to the field of augmented reality (“AR”), often referred to as mixed reality (“MR”) when an experience is provided via a head-mounted display (“HMD”) device. More particularly, the disclosure relates to a system that generates AR content.2. General Background(2) AR is used in a variety of contexts (e.g., gaming, medicine, etc.) to provide a virtual experience in conjunction with a real-world experience (e.g., an overlay of various text and/or images over a real-world object, person, place, etc.). A user may use various devices, which are worn or held by the user, to allow the user to view real-world elements along with virtual elements during an AR experience.(3) Unlike a VR experience, which only encompasses virtual content, an AR experience poses specific challenges for recording purposes. For example, if an image capture device (e.g., camera) was worn in conjunction with an AR display device by the user, the recorded content would typically include shaky movements resulting from the user's natural head and/or limb articulations; this is in contrast to a VR experience in which the recorded content may be confined to virtual imagery in isolation of real-world artifacts. Therefore, conventional AR systems do not adequately capture AR content during an AR experience with sufficient quality to allow for simultaneous, or subsequent, viewing by another user at a viewing device other than the AR device used by the user.SUMMAR
Claims
1. A computer program product comprising a non-transitory computer readable storage device having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to: capture, with a UAV image capture device from a perspective view of a UAV, real-world imagery of a user within an augmented reality environment, the perspective view of the UAV being distinct from a perspective view of an augmented reality device that occurs simultaneously with the perspective view of the UAV, the augmented reality device being in contact with the user; generate, with a UAV processor based on the perspective view of the UAV, virtual imagery to overlay the real-world imagery; receive, with the UAV processor from the augmented reality device, virtual imagery manipulation data, the virtual imagery manipulation data indicating a virtual interactive effect that is rendered by the augmented reality device from the perspective of the augmented reality device based on a position and an orientation of an augmented reality device accessory, the virtual imagery manipulation data further indicating the position and the orientation of the augmented reality device accessory within a common coordinate space, the common coordinate space being common to the UAV and the augmented reality device; modify, with the UAV processor, the virtual imagery based on the position and the orientation of the augmented reality device accessory within the common coordinate space; and generate, with the UAV processor, a composite of the captured real-world imagery and the modified virtual imagery.
11. A computer program product comprising a non-transitory computer readable storage device having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to: capture, with an augmented reality image capture device from a perspective view of an augmented reality device, real-world imagery of a user within an augmented reality environment; generate, with an augmented reality processor, virtual imagery to overlay the captured real-world imagery; determine, with the augmented reality processor, a position and an orientation of an augmented reality device accessory within a common coordinate space, the common coordinate space being common to the augmented reality device and a UAV that simultaneously captures the real-world imagery from a perspective view of the UAV, the perspective view of the augmented reality device being distinct from the perspective view of the UAV; and provide, from the augmented reality processor to a UAV processor, virtual imagery manipulation data, the virtual imagery manipulation data indicating a virtual interactive effect that is rendered by the augmented reality device from the perspective view of the augmented reality device based on the position and the orientation of the augmented reality device accessory, the virtual imagery manipulation data further indicating the position and the orientation of the augmented reality device within the common coordinate space so that the UAV processor modifies the virtual imagery from the perspective view of the UAV based on the position and the orientation of the augmented reality device accessory within the common coordinate space and generates a composite of the captured real-world imagery from the perspective view of the UAV and the modified virtual imagery from the perspective view of the UAV, the UAV processor being integrated within the UAV.
20. A system comprising: an augmented reality device that captures, from a perspective view of a the augmented reality device, real-world imagery of a user within an augmented reality environment, generates virtual imagery to overlay the captured real-world imagery from the perspective view of the augmented reality device, and determines a position and an orientation of an augmented reality device accessory within a common coordinate space; and a UAV that captures, from a perspective view of the UAV, real-world imagery of the user within the augmented reality environment, determines a position of the UAV within the common coordinate space, generates virtual imagery to overlay the captured real-world imagery from the perspective view of the UAV, receives the position and the orientation from the augmented reality device accessory, modifies the virtual imagery based on the position and the orientation of the augmented reality device accessory within the common coordinate space, and generates a composite of the captured real-world imagery from the perspective view of the UAV and the modified virtual imagery from the perspective view of the UAV.