Outer Rim Archives
Archives · 2019 · 20190379878

Application (pre-grant publication)

SEE-THROUGH OPERATOR AWARENESS TOOL USING SYNTHESIZED VIEWPOINTS

Number
20190379878
Published
2019-12-12
Filed
2018-06-12
Assignee
Disney Enterprises, Inc.
Inventors
CHAPMAN; Steven, ARANA; Mark, GOSLIN; Michael, POPP; Joseph, PATEL; Mehul
CPC
G06F3/012; H04N13/204; H04N5/2628; H04N5/272; H04N13/117; H04N13/243; H04N13/344; H04N23/54
Verdict
Low Notable software
Source
Google Patents · FreePatentsOnline

The keeper's note

See-through operator awareness synthesized viewpoints.

Abstract

The disclosure is directed to providing operator visibility through an object that occludes the view of the operator by using a HMD system in communication with one or more imaging devices. A method of doing so may include: using one or more imaging devices coupled to an exterior of an object to capture image data of a real-world environment surrounding the object: calculating an orientation of a HMD positioned in an interior of the object; using at least the calculated orientation of the HMD, synthesizing a FOV of the image data with a FOV of the HMD; and rendering the synthesized FOV of the image data to the HMD. In some instances, the captured image data is used to create a three-dimensional model of the real-world environment, and synthesizing the FOV of the image data includes synthesizing a FOV of the 3D model with the FOV of the HMD.

Background

BRIEF SUMMARY OF THE DISCLOSURE

The disclosure is directed to providing operator visibility through an object that occludes the view of the operator by using a head mounted display (HMD) system in communication with one or more imaging devices.

In one embodiment, a method includes: using one or more imaging devices coupled to an exterior of an object to capture image data of a real-world environment surrounding the object; calculating an orientation of a HMD positioned in an interior of the object; using at least the calculated orientation of the HMD, synthesizing a FOV of the image data with a FOV of the HMD; and rendering the synthesized FOV of the image data to the HMD.

In some implementations, the one or more image devices include one or more video cameras, and the image data of the real-world environment includes video of the real-world environment captured by the one or more video cameras, where synthesizing the FOV of the image data includes synthesizing a FOV of the video captured by the one or more video cameras with the FOV of the HMD. In particular implementations, the one or more video cameras include a plurality of omnidirectional cameras.

In some implementations, the method further includes: overlaying the synthesized FOV of the video captured by the one or more video cameras over video of a FOV of the HMD; or overlaying video of a FOV of the HMD over the synthesized FOV of the video captured by the one or more video cameras. The overlay

Claims

1. A method, comprising: using one or more imaging devices coupled to an exterior of an object to capture image data of a real-world environment surrounding the object; calculating an orientation of a head mounted display (HMD) positioned in an interior of the object; using at least the calculated orientation of the HMD, synthesizing a field of view (FOV) of the image data with a FOV of the HMD; and rendering the synthesized FOV of the image data to the HMD. 2. The method of claim 1, wherein the one or more image devices comprise one or more video cameras, wherein the image data of the real-world environment comprises video of the real-world environment captured by the one or more video cameras, wherein synthesizing the FOV of the image data comprises synthesizing a FOV of the video captured by the one or more video cameras with the FOV of the HMD. 3. The method of claim 2, wherein the one or more video cameras comprise a plurality of omnidirectional cameras. 4. The method of claim 3, further comprising: overlaying the synthesized FOV of the video captured by the one or more video cameras over video of a FOV of the HMD; or overlaying video of a FOV of the HMD over the synthesized FOV of the video captured by the one or more video cameras. 5. The method of claim 4, further comprising: render the overlay of the synthesized FOV of the video captured by the one or more video cameras and the video of the FOV of the HMD. 6. The method of claim 5 wherein the video of the FOV of the HMD is ghosted in the overlay. 7. The method of claim 5, wherein the object comprises a vehicle or costumed character. 8. The method of claim 5, further comprising: presenting a graphical user interface to a user of the HMD, the graphical user interface comprising controls to modify the appearance of the overlay. 9. The method of claim 1, further comprising: using the captured image data to create a three-dimensional model of the real-world environment, wherein synthesizing the FOV of the image data comprises synthesizing a FOV of the 3D model with the FOV of the HMD. 10. The method of claim 9, wherein the one or more image devices comprise one or more video cameras, wherein the image data of the real-world environment comprises video of the real-world environment captured by the one or more video cameras. 11. A non-transitory computer-readable medium having executable instructions stored thereon that, when executed by one or more processors, cause a system to: use one or more imaging devices coupled to an exterior of an object to capture image data of a real-world environment surrounding the object; calculate an orientation of a head mounted display (HMD) positioned in an interior of the object; use at least the calculated orientation of the HMD to synthesize a field of view (FOV) of the image data with a FOV of the HMD; and render the synthesized FOV of the image data to the HMD. 12. The non-transitory computer-readable medium of claim 11, wherein the one or more image devices comprise one or more video cameras, wherein the image data of the real-world environment comprises video of the real-world environment captured by the one or more video cameras, wherein synthesizing the FOV of the image data comprises synthesizing a FOV of the video captured by the one or more video cameras with the FOV of the HMD. 13. The non-transitory computer-readable medium of claim 12, wherein the one or more video cameras comprise a plurality of omnidirectional cameras. 14. The non-transitory computer-readable medium of claim 13, wherein the instructions, when executed by the one or more processors, further cause the system to: overlay the synthesized FOV of the video captured by the one or more video cameras over video of a FOV of the HMD; or overlay video of a FOV of the HMD over the synthesized FOV of the video captured by the one or more video cameras. 15. The non-transitory computer-readable medium of claim 14, wherein the instructions, when executed by the one or more processors, further cause the system to: render the overlay of the synthesized FOV of the video captured by the one or more video cameras and the video of the FOV of the HMD. 16. The non-transitory computer-readable medium of claim 15 wherein the video of the FOV of the HMD is ghosted in the overlay. 17. The non-transitory computer-readable medium of claim 15, wherein the object comprises a vehicle or costumed character. 18. The non-transitory computer-readable medium of claim 15, wherein the instructions, when executed by the one or more processors, further cause the system to: present a graphical user interface to a user of the HMD, the graphical user interface comprising controls to modify the appearance of the overlay. 19. The non-transitory computer-readable medium of claim 11, wherein the instructions, when executed by the one or more processors, further cause the system to: use the captured image data to create a three-dimensional model of the real-world environment, wherein synthesizing the FOV of the image data comprises synthesizing a FOV of the 3D model with the FOV of the HMD. 20. The non-transitory computer-readable medium of claim 17, wherein the one or more image devices comprise one or more video cameras, wherein the image data of the real-world environment comprises video of the real-world environment captured by the one or more video cameras. 21. A system, comprising: a plurality of omnidirectional video cameras adapted to be mounted on an exterior of an object; and a head mounted display (HMD) system comprising a HMD, the HMD system configured to: receive video captured by the plurality of omnidirectional video cameras of a real-world environment surrounding the object; calculate an orientation of the HMD while it is positioned in an interior of the object; using at least the calculated orientation of the HMD, synthesize a field of view (FOV) of the captured video with a FOV of the HMD; and render the synthesized FOV of the video to the HMD.