- Number
- 11394949
- Published
- 2022-07-19
- Filed
- 2018-06-12
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Chapman; Steven, Arana; Mark, Goslin; Michael, Popp; Joseph, Patel; Mehul
- CPC
- G06F3/012; H04N13/204; H04N5/2628; H04N5/272; H04N13/117; H04N13/243; H04N13/344; H04N23/54
- Verdict
- Low Hardware
- Source
- Google Patents · FreePatentsOnline
The keeper's note
See-through synthesized-viewpoint awareness display for costume/vehicle operators.
Abstract
The disclosure is directed to providing operator visibility through an object that occludes the view of the operator by using a HMD system in communication with one or more imaging devices. A method of doing so may include: using one or more imaging devices coupled to an exterior of an object to capture image data of a real-world environment surrounding the object: calculating an orientation of a HMD positioned in an interior of the object; using at least the calculated orientation of the HMD, synthesizing a FOV of the image data with a FOV of the HMD; and rendering the synthesized FOV of the image data to the HMD. In some instances, the captured image data is used to create a three-dimensional model of the real-world environment, and synthesizing the FOV of the image data includes synthesizing a FOV of the 3D model with the FOV of the HMD.
Background
BRIEF SUMMARY OF THE DISCLOSURE (1) The disclosure is directed to providing operator visibility through an object that occludes the view of the operator by using a head mounted display (HMD) system in communication with one or more imaging devices. (2) In one embodiment, a method includes: using one or more imaging devices coupled to an exterior of an object to capture image data of a real-world environment surrounding the object; calculating an orientation of a HMD positioned in an interior of the object; using at least the calculated orientation of the HMD, synthesizing a FOV of the image data with a FOV of the HMD; and rendering the synthesized FOV of the image data to the HMD. (3) In some implementations, the one or more image devices include one or more video cameras, and the image data of the real-world environment includes video of the real-world environment captured by the one or more video cameras, where synthesizing the FOV of the image data includes synthesizing a FOV of the video captured by the one or more video cameras with the FOV of the HMD. In particular implementations, the one or more video cameras include a plurality of omnidirectional cameras. (4) In some implementations, the method further includes: overlaying the synthesized FOV of the video captured by the one or more video cameras over video of a FOV of the HMD; or overlaying video of a FOV of the HMD over the synthesized FOV of the video captured by the one or more video cameras. The overlay may be r
Claims
1. A method, comprising: using a first imaging device and a second imaging device coupled to an exterior of a vehicle to capture image data of a real-world environment, wherein: the real-world environment surrounds the vehicle, the image data includes information corresponding to a point occluded by an obstacle from a point of view of an occupant of the vehicle, the first imaging device is positioned on the vehicle on a first side with respect to the occupant, the second imaging device is positioned on the vehicle on a second side with respect to the occupant, the second side being opposite the first side, the image data includes a plurality of intersecting pixels captured by both the first imaging device and the second imaging device, and the plurality of intersecting pixels comprises a representation of the point occluded by the obstacle; calculating a relative spacing between a head mounted display (HMD) worn by the occupant and at least one of the first imaging device or the second imaging device; using at least the relative spacing, synthesizing from the point of view of the occupant, a field of view (FOV) of the image data with a FOV of the HMD to generate a synthesized FOV, wherein the information corresponding to the point occluded by the obstacle is mapped into the synthesized FOV as a patch including the plurality of intersecting pixels of the image data such that the synthesized FOV includes a view from the point of view of the occupant as though the obstacle were not present; and rendering the synthesized FOV on the HMD. ||
9. A non-transitory computer-readable medium having executable instructions stored thereon that, when executed by one or more processors, cause a system to: use a first imaging device and a second imaging device coupled to an exterior of a vehicle to capture image data of a real-world environment, wherein: the real-world environment surrounds the vehicle, the image data includes information corresponding to a point occluded by an obstacle from a point of view of an occupant of the vehicle, the first imaging device is positioned on the vehicle on a first side with respect to the occupant, the second imaging device is positioned on the vehicle on a second side of the point with respect to the occupant, the second side being opposite the first side, the image data includes a plurality of intersecting pixels captured by both the first imaging device and the second imaging device, and the plurality of intersecting pixels comprises a representation of the point occluded by the obstacle; calculate a relative spacing between at least one of the first or second imaging devices and a head mounted display (HMD) worn by the occupant; use at least the relative spacing to synthesize from the point of view of the occupant, a field of view (FOV) of the image data with a FOV of the HMD to generate a synthesized FOV, wherein the the information corresponding to the point occluded by the obstacle is mapped into the synthesized FOV as a patch including the plurality of intersecting pixels of the image data such that the synthesized FOV includes a view from the point of view of the occupant as though the obstacle were not present; and render the synthesized FOV on the HMD. ||
16. A system, comprising: a first video camera and a second video camera configured to be mounted on an exterior of a vehicle; and a head mounted display (HMD) system comprising a HMD, the HMD system configured to: receive, while the first and second video cameras are mounted on the exterior of the vehicle, video captured by the first and second video cameras of a real-world environment, wherein: the real-world environment surrounds the vehicle, the video includes information corresponding to a point occluded by an obstacle from a point of view of an occupant of the vehicle, the first video camera is positioned on the vehicle on a first side with respect to the occupant, the second video camera is positioned on the vehicle on a second side with respect to the occupant, the second side being opposite the first side, the video includes a plurality of intersecting pixels captured by both of the first video camera and the second video camera, and the plurality of intersecting pixels comprises a representation of the point occluded by the obstacle; calculate a relative spacing between at least one of the first or second video cameras and the HMD worn by the occupant; using at least the relative spacing, synthesize from the point of view of the occupant, a field of view (FOV) of the video with a FOV of the HMD to generate a synthesized FOV, wherein the information corresponding to the point occluded by the obstacle is mapped into the synthesized FOV as a patch including the plurality of intersecting pixels of the video such that the synthesized FOV includes a view from the point of view of the occupant as though the obstacle were not present; and render the synthesized FOV of the video on the HMD.