- Number
- 10419669
- Published
- 2019-09-17
- Filed
- 2017-01-17
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Hornung; Alexander Sorkine, Schroers; Christopher, Bazin; Jean-Charles
- CPC
- G06T3/4038; H04N13/00; H04N13/111; H04N13/243; H04N23/45; H04N23/661; H04N23/698; H04N23/90
- Verdict
- Medium Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Omnistereoscopic panoramic video (novel capture/rendering).
Abstract
Systems and methods to generate omnistereoscopic panoramic videos are presented herein. Depth information, flow fields, and/or other information may be used to determine interpolated frame images between adjacent frame images. An omnistereoscopic panoramic video may be used in a real-world VR application.
Background
FIELD OF THE DISCLOSURE(1) The present invention relates generally to generating omnistereoscopic panoramic video.BACKGROUND(2) There is a strong consumer interest in a more immersive experience of content, such as real-world virtual reality (VR) video.SUMMARY(3) One aspect of the disclosure relates to a system configured to generate omnistereoscopic panoramic video. Omnistereoscopic panoramic video generation may be based on determining omnistereoscopic panoramic frame images. In some implementations, omnistereoscopic panoramic frame image generation may be based on creating two sets of light rays: one for the left eye and one for the right eye, for example with the rays and/or camera directions tangential to a circle. In some implementations, omnistereoscopic panoramic frame image generation may be based on reconstructing a full lightfield and selecting two sets of light rays from lightfield.(4) The system may include one or more of one or more physical processors, one or more cameras, and/or other components. The one or more physical processors may be configured by machine-readable instructions. Executing the machine-readable instructions may cause one or more physical processors to facilitate generating omnistereoscopic panoramic video. The machine-readable instructions may include one or more of a video component, a depth component, a flow field component, an interpolation component, a stitching component, and/or other components.(5) The video component may be configured
Claims
1. A system configured to generate omnistereoscopic panoramic video, the system comprising: one or more physical processors configured by machine-readable instructions to: obtain information defining a set of videos, individual videos in the set of videos including visual content, the visual content of an individual video including frame images, individual frame images being captured at individual points in time, the individual videos depicting at least part of a three-dimensional scene, the set of videos being captured simultaneously by a camera array such that for an individual point in time: individual videos in the set of videos include at least one frame image captured at the individual point in time, and at least some of the frame images captured at the individual point in time form an individual set of adjacent frame images at the individual point in time; determine depth information by reconstructing the three-dimensional scene using the obtained information defining the set of videos, the depth information including three-dimensional coordinates of points lying on surfaces within the three-dimensional scene; determine optical flow fields between related pixels in adjacent frame images in the individual sets of adjacent frame images formed at the individual points in time, the determination of the optical flow fields being based on a penalization function, a Jacobian, and a homography between the adjacent frame images induced by a plane at infinity; determine interpolated frame images between adjacent frame images in the individual sets of adjacent frame images formed at the individual points in time, the interpolated frame images being determined based on the information defining the set of videos, the depth information, and/or the optical flow fields; and generate an omnistereoscopic panoramic video, the omnistereoscopic panoramic video comprising the frame images included in the obtained set of videos images and the interpolated frame images.
9. A method of generating omnistereoscopic panoramic video, the method being implemented in a computer system comprising one or more physical processors and storage media storing machine-readable instructions, the method comprising: obtaining information defining a set of videos, individual videos in the set of videos including visual content, the visual content of an individual video including frame images, individual frame images being captured at individual points in time, the individual videos depicting at least part of a three-dimensional scene, the set of videos being captured simultaneously by a camera array such that for an individual point in time: individual videos in the set of videos include at least one frame image captured at the individual point in time, and at least some of the frame images captured at the individual point in time form an individual set of adjacent frame images at the individual point in time; determining depth information by reconstructing the three-dimensional scene using the obtained information defining the set of videos, the depth information including three-dimensional coordinates of points lying on surfaces within the three-dimensional scene; determining optical flow fields between related pixels in adjacent frame images in the individual sets of adjacent frame images formed at the individual points in time, the determination of the optical flow fields being based on a penalization function, a Jacobian, and a homography between the adjacent frame images induced by a plane at infinity; determining interpolated frame images between adjacent frame images in the individual sets of adjacent frame images formed at the individual points in time, the interpolated frame images being determined based on the information defining the set of videos, the depth information, and/or the optical flow fields; and generating an omnistereoscopic panoramic video, the omnistereoscopic panoramic video comprising the frame images included in the obtained set of videos images and the interpolated frame images.