Object reconstruction from dense light fields via depth from gradients

Number: 10122994
Published: 2018-11-06
Filed: 2017-10-31
Assignee: DISNEY ENTERPRISES, INC.; ETH ZÜRICH (EIDGENÖSSISCHE TECHNISCHE HOCHSCHULE ZÜRICH)
Inventors: Yücer; Kaan; Kim; Changil; Sorkine-Hornung; Alexander; Sorkine-Hornung; Olga
CPC: G06T7/564; H04N13/232; G06T7/557; H04N13/106; G06T7/593; G06T17/205; H04N13/271; G06T17/20; H04N13/15; G06T15/205
Verdict: Medium Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Light-field 3D object reconstruction (grant dup).

Abstract

The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.

Background

BACKGROUND(1) Reconstructing objects in three dimensions from a set of two-dimensional images is a long standing problem in computer vision. And despite significant research efforts, objects with thin features still pose problems for many reasons. First, the thin features occupy only a small number of pixels in the views that they are visible in, making locating them difficult. Moreover, many object reconstruction techniques miss the thin features because the techniques require patches on the objects to be several pixels wide, which is not always the case with thin features. The thin features are also usually only visible in a small number of views, making matching the thin features between different views difficult. Other reconstruction techniques face difficulties with texture-less thin features because it is hard for such techniques to localize the features using photoconsistency values inside a volumetric discretization, often resulting in elimination of these features in the reconstruction. Therefore, there is a need in the art to improve techniques for reconstructing objects in three dimensions from a set of two-dimensional images.SUMMARY(2) The present disclosure relates generally to object reconstruction. More particularly, techniques are described for reconstructing an object in three dimensions that is captured in a set of two-dimensional images.(3) In some embodiments, the object is reconstructed in three dimensions by computing depth values for edges of the object

Claims

1. A method for object reconstruction, the method comprising: receiving a light field represented by a plurality of images, wherein each image is at a different viewpoint, and wherein the plurality of images are associated with an object; determining a depth for a color edge of the object in an image of the plurality of images; propagating the depth for the color edge to one or more points around the color edge based on a type of the color edge, wherein a direction of propagation depends on whether the type of color edge is texture or silhouette; generating a depth map for the image using the depth for the color edge and the depth for the one or more points; and generating a mesh for the object based on the depth map. 8. A non-transitory computer-readable storage medium storing a plurality of instructions executable by one or more processors, the plurality of instructions when executed by the one or more processors cause the one or more processors to: receive a light field represented by a plurality of images, wherein each image is at a different viewpoint, and wherein the plurality of images are associated with an object; determine a depth for a color edge of the object in an image of the plurality of images; propagate the depth for the color edge to one or more points around the color edge based on a type of the color edge, wherein a direction of propagation depends on whether the type of color edge is texture or silhouette; generate a depth map for the image using the depth for the color edge and the depth for the one or more points; and generate a mesh for the object based on the depth map. 15. A system comprising: one or more processors; and a non-transitory computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to: receive a light field represented by a plurality of images, wherein each image is at a different viewpoint, and wherein the plurality of images are associated with an object; determine a depth for a color edge of the object in an image of the plurality of images; propagate the depth for the color edge to one or more points around the color edge based on a type of the color edge, wherein a direction of propagation depends on whether the type of color edge is texture or silhouette; generate a depth map for the image using the depth for the color edge and the depth for the one or more points; and generate a mesh for the object based on the depth map.