Multi-perspective stereoscopy from light fields

Number: 9843776
Published: 2017-12-12
Filed: 2015-07-15
Assignee: Disney Enterprises, Inc.
Inventors: Kim; Changil, Hornung; Alexander, Heinzle; Simon, Matusik; Wojciech, Gross; Markus
CPC: G02B27/0172; G06T15/06; G06T15/08; G06T3/18; G09G5/00; H04N13/128; H04N13/239; H04N13/271; H04N13/275; H04N13/282; H04N13/31; H04N13/32; H04N13/339; H04N13/395; H04N7/18
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Generates stereoscopic 3D content from light-field imaging with fine control over binocular disparity for comfortable depth perception.

Abstract

Methods and systems for generating stereoscopic content with granular control over binocular disparity based on multi-perspective imaging from representations of light fields are provided. The stereoscopic content is computed as piecewise continuous cuts through a representation of a light field, minimizingan energy reflecting prescribed parameters such as depth budget, maximum binocular disparity gradient, desired stereoscopic baseline. The methods and systems may be used for efficient and flexible stereoscopic post-processing, such as reducing excessive binocular disparity while preserving perceived depth or retargeting of already captured scenes to variousview settings. Moreover, such methods and systems are highly useful for content creation in the context of multi-view autostereoscopic displays and provide a novel conceptual approach to stereoscopic image processing and post-production.

Background

BRIEF DESCRIPTION OF THE DRAWINGS(1) The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.(2) In order to more fully understand the present invention, reference is made to the accompanying drawings. Understanding that these drawings are not to be considered limitationsin the scope of the invention, the presently described embodiments and the presently understood best mode of the invention are described with additional detail through use of the accompanying drawings.(3) FIG. 1 illustrates an operation of a system for generating multi-perspective stereoscopic output images from a 3D light field; FIG. 1 comprises FIGS. 1(a)-1(d).(4) FIG. 2 is a simple example that illustrates a light field parameterization; FIG.2 comprises FIGS. 2(a) and 2(b).(5) FIG. 3 illustrates examples of a planar single perspective surface cut of the light field and a non-planar multi-perspective surface cut of thelight field; FIG. 3 comprises FIGS. 3(a)-3(b).(6) FIG. 4 illustrates an example 2D EPI ofa light field with two planar u-t-cuts and the corresponding stereoscopic image pair generated from I.sub.s and I.sub.s′.(7) FIG. 5 illustrates an example 2D u-s-slice of a normalized disparity volume D and an example 2D u-s-slice of a true image disparity volume with respect to a reference view Is; FIG. 5 comprises FIGS. 5(a)

Claims

1. A method of deferred rendering of computer-generated content, the method comprising: generating a respective depth map for each of a set of computer-generated input images of a scene; generating a disparity volume from the generated depth maps, the disparity volume corresponding to a three-dimensional (3D) representation of a light field representing light rays from the scene; determining a non-planar surface cut through the disparity volume, wherein the non-planar surface cut is determined using a two-dimensional map; determining a plurality of two-dimensional (2D) subsets of the disparity volume corresponding to a plurality of 2D subsetsof the 3D representation of the light field, wherein at least one 2D subset of the disparity volume corresponds to the non-planar surface cut; determining a plurality of the computer-generated input images whose respective depth maps are included in the determined 2D subsets of the disparity volume; and rendering, by a computer system, the determined plurality of the computer-generated input images. 8. A system comprising: one or more processorsconfigured to: generate a respective depth map for each of a set of computer-generated input images of a scene; generate a disparity volume from the generated depth maps, the disparity volume corresponding to a three-dimensional (3D) representation of a light field representing light rays from the scene; determine a non-planar surface cut through the disparity volume, wherein the non-planar surface cut is determined using a two-dimensional map; determine a plurality of two-dimensional (2D) subsets of the disparity volume corresponding to a plurality of 2D subsets of the 3D representation of the light field, wherein at least one 2D subset of the disparity volume corresponds to the non-planar surface cut; determine a plurality of the computer-generated input images whose respective depth maps are included in the determined 2D subsets of the disparity volume; and render, by a computer system, the determined plurality of the computer-generated input images. 15. A non-transitory computer-readable medium containing program code that, when executed by a computer, causesthe computer to perform a method of deferred rendering of computer-generated content comprising: generating a respective depth map for each of a set of computer-generated input images of a scene; generating a disparity volume from the generated depth maps, the disparity volume corresponding to a three-dimensional (3D) representation of a light field representing light rays from the scene; determining a non-planar surface cut through the disparity volume, wherein the non-planar surface cut is determined using a two-dimensional map; determining a plurality of two-dimensional (2D) subsets of the disparity volume corresponding to a plurality of 2D subsets of the 3D representation of the light field, wherein at least one 2D subset of the disparity volume corresponds to the non-planar surface cut; determining a plurality of the computer-generated input images whose respective depth maps are included in the determined 2D subsets of the disparity volume; and rendering, by a computersystem, the determined plurality of the computer-generated input images.