Light-field 3D object reconstruction.
The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.
BACKGROUND
Reconstructing objects in three dimensions from a set of two-dimensional images is a long standing problem in computer vision. And despite significant research efforts, objects with thin features still pose problems for many reasons. First, the thin features occupy only a small number of pixels in the views that they are visible in, making locating them difficult. Moreover, many object reconstruction techniques miss the thin features because the techniques require patches on the objects to be several pixels wide, which is not always the case with thin features. The thin features are also usually only visible in a small number of views, making matching the thin features between different views difficult. Other reconstruction techniques face difficulties with texture-less thin features because it is hard for such techniques to localize the features using photoconsistency values inside a volumetric discretization, often resulting in elimination of these features in the reconstruction. Therefore, there is a need in the art to improve techniques for reconstructing objects in three dimensions from a set of two-dimensional images.SUMMARY
The present disclosure relates generally to object reconstruction. More particularly, techniques are described for reconstructing an object in three dimensions that is captured in a set of two-dimensional images.
In some embodiments, the object is reconstructed in three dimensions by computing depth values for edges of t
1. A method for object reconstruction, the method comprising: receiving a light field represented by a plurality of images of an object, wherein the plurality of images include a first image and a second image, and wherein the second image is at a viewpoint different than the first image; identifying a first point of the object in the first image; identifying a second line in the second image based on the first point in the first image; identifying one or more second points along the second line; identifying one or more first points along a first line intersecting the first point, wherein each point of the one or more first points is projected from the one or more second points; determining that a second point of the one or more second points corresponds to the first point based on a color of each of the one or more first points and a color of each of the one or more second points; computing a depth for the first point based on the first point and the second point, wherein the depth is computed in response to determining that the second point corresponds to the first point; generating a depth map for the first image using the depth of the first point; and generating a mesh for the object based on the depth map.
8. A non-transitory computer-readable storage medium storing a plurality of instructions executable by one or more processors, the plurality of instructions when executed by the one or more processors cause the one or more processors to: receive a light field represented by a plurality of images of an object, wherein the plurality of images include a first image and a second image, and wherein the second image is at a viewpoint different than the first image; identify a first point of the object in the first image; identify a second line in the second image based on the first point in the first image; identify one or more second points along the second line; identify one or more first points along a first line intersecting the first point, wherein each point of the one or more first points is projected from the one or more second points; determine that a second point of the one or more second points corresponds to the first point based on a color of each of the one or more first points and a color of each of the one or more second points; compute a depth for the first point based on the first point and the second point, wherein the depth is computed in response to determining that the second point corresponds to the first point; generate a depth map for the first image using the depth of the first point; and generate a mesh for the object based on the depth map.
15. A system comprising: one or more processors; and a non-transitory computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to: receive a light field represented by a plurality of images of an object, wherein the plurality of images include a first image and a second image, and wherein the second image is at a viewpoint different than the first image; identify a first point of the object in the first image; identify a second line in the second image based on the first point in the first image; identify one or more second points along the second line; identify one or more first points along a first line intersecting the first point, wherein each point of the one or more first points is projected from the one or more second points; determine that a second point of the one or more second points corresponds to the first point based on a color of each of the one or more first points and a color of each of the one or more second points; compute a depth for the first point based on the first point and the second point, wherein the depth is computed in response to determining that the second point corresponds to the first point; generate a depth map for the first image using the depth of the first point; and generate a mesh for the object based on the depth map.