Scene reconstruction from high spatio-angular resolution light fields

Number: 9786062
Published: 2017-10-10
Filed: 2013-07-17
Assignee: Disney Enterprises, Inc.
Inventors: Sorkine-Hornung; Alexander, Kim; Changil, Zimmer; Henning, Pritch; Yael, Gross; Markus
CPC: G06T7/50; G06T7/593; G06T7/557
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Estimates scene depth from light-field imagery by first identifying regions where epipolar-plane image analysis is expected to be reliable.

Abstract

The disclosure provides an approach for estimating depth in a scene. According to one aspect, regions where the depth estimation is expected to perform well may first be identified in full-resolution epipolar-plane images (EPIs) generated from a plurality of images of thescene. Depth estimates for EPI-pixels with high edge confidence are determined by testinga number of discrete depth hypotheses and picking depths that lead to highest color density of sampled EPI-pixels. The depth estimate may also be propagated throughout the EPIs. This process of depth estimation and propagation may be iterated until all EPI-pixels with high edge confidence have been processed, and all EPIs may also be processed in this manner. The EPIs are then iteratively downsampled to coarser resolutions, at which edge confidence for EPI-pixels not yet processed are determined, depth estimates of EPI-pixels with high edge confidence made, and depth estimates propagated throughout the EPIs.

Background

BRIEF DESCRIPTION OF THE DRAWINGS(1) The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.(2) So that the manner in which the above recited aspects are attained and can beunderstood in detail, a more particular description of aspects of the disclosure, brieflysummarized above, may be had by reference to the appended drawings.(3) It is to be noted,however, that the appended drawings illustrate only typical aspects of this disclosure and are therefore not to be considered limiting of its scope, for the disclosure may admit to other equally effective aspects.(4) FIG. 1 illustrates an approach for estimating depth from high spatio-angular resolution light fields, according to an aspect of the disclosure.(5) FIG. 2 illustrates a fine-to-coarse approach for depth estimation, according to an aspect of the disclosure.(6) FIG. 3 illustrates a method for estimating depth in a scene, according to an aspect of the disclosure.(7) FIG. 4 depicts a block diagram of a system in which an aspect may be implemented.DETAILED DESCRIPTION(8) Aspects disclosed herein providetechniques for estimating depth in a scene. Given a light field constructed from multipleimages, fine-to-coarse depth estimation is used to compute reliable depth estimates firstaround object boundaries, then at more homogeneous interior region

Claims

1.A computer-implemented method, comprising: receiving a plurality of images of a scene; and estimating, from the received images, depths of scene points at multiple image resolution levels beginning from a finest resolution level and iteratively proceeding to coarser resolution levels, wherein, during each iteration, a respective resolution level is processed by performing steps including: determining, based on at least local color variation, edge confidence scores for scene points for which depth estimates have not been assigned at any previous iteration, determining sufficiently detailed regions of the scene at the resolution level, the sufficiently detailed regions being regions in which the edge confidence scores of scene points exceed a threshold value, and determining depth estimates for scenepoints in the sufficiently detailed regions. 14. A non-transitory computer-readable storage medium storing instructions, which when executed by a computer system, perform operations comprising: receiving a plurality of images of a scene; and estimating, from the received images, depths of scene points at multiple resolution levels beginning from a finest resolution level and iteratively proceeding to coarser resolution levels, wherein, during each iteration, a respective resolution level is processed by performing steps including: determining, based on at least local color variation, edge confidence scores for scene points for which depth estimates have not been assigned at any previous iteration, determining sufficiently detailed regions of the scene at the resolution level, the sufficiently detailed regions being regions in which the edge confidence scores of scene points exceed a threshold value, and determining depth estimates for scene points in the sufficiently detailed regions. 27. A system, comprising: a processor; and a memory, wherein the memory includes an application program configured to perform operations, the operations comprising: receiving a plurality of images of a scene, and estimating, from the received images, depths of scene points at multiple resolution levels beginning from a finest resolution level and iteratively proceeding to coarser resolution levels, wherein, during each iteration, a respective resolution level is processed by performing steps including: determining, based onat least local color variation, edge confidence scores for scene points for which depth estimates have not been assigned at any previous iteration; determining sufficiently detailed regions of the scene at the resolution level, the sufficiently detailed regions being regions in which the edge confidence scores of scene points exceed a threshold value; and determining depth estimates for scene points in the sufficiently detailed regions.