- Number
- 10887581
- Published
- 2021-01-05
- Filed
- 2017-10-31
- Assignee
- DISNEY ENTERPRISES, INC.
- Inventors
- Yücer; Kaan, Kim; Changil, Sorkine-Hornung; Alexander, Sorkine-Hornung; Olga
- CPC
- H04N13/232; G06T7/564; G06T7/593; H04N13/271; G06T7/557; G06T17/20; H04N13/15; H04N13/106; G06T15/205; G06T17/205
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
3D object reconstruction technique from light-field imaging.
Abstract
The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.
Background
BACKGROUND (1) Reconstructing objects in three dimensions from a set of two-dimensional images is a long standing problem in computer vision. And despite significant research efforts, objects with thin features still pose problems for many reasons. First, the thin features occupy only a small number of pixels in the views that they are visible in, making locating them difficult. Moreover, many object reconstruction techniques miss the thin features because the techniques require patches on the objects to be several pixels wide, which is not always the case with thin features. The thin features are also usually only visible in a small number of views, making matching the thin features between different views difficult. Other reconstruction techniques face difficulties with texture-less thin features because it is hard for such techniques to localize the features using photoconsistency values inside a volumetric discretization, often resulting in elimination of these features in the reconstruction. Therefore, there is a need in the art to improve techniques for reconstructing objects in three dimensions from a set of two-dimensional images. SUMMARY (2) The present disclosure relates generally to object reconstruction. More particularly, techniques are described for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. (3) In some embodiments, the object is reconstructed in three dimensions by computing depth values for edges of the ob
Claims
1. A method for 3D object reconstruction, the method comprising: receiving a light field represented by a plurality of images of an object, wherein the plurality of images include a first image and a second image, and wherein the second image is at a viewpoint different than the first image; identifying a first point of the object in the first image, the first point being on an edge of the object, the edge identified by a high-gradient region in the first image; identifying a second line in the second image based on the first point in the first image, the second line comprising an epipolar line based on the first point; identifying one or more second points along the second line, the one or more second points comprise estimates where the first point is in the second image; identifying one or more first points along a first line intersecting the first point, wherein each point of the one or more first points is projected from the one or more second points; determining that a second point of the one or more second points corresponds to the first point based on a color of each of the one or more first points and a color of each of the one or more second points; computing a depth for the first point based on the first point and the second point, wherein the depth is computed in response to determining that the second point corresponds to the first point, wherein a color gradient is computed using colors of the one or more first points and the one or more second points, and wherein the color gradient is used to compute the depth of the first point; propagating the depth to one or more points around the edge of the object based in part on a type of a color edge being a texture edge, wherein the texture edge is a boundary of the object that includes points on both sides of the boundary having similar depths; generating a depth map for the first image using the depth of the first point; and generating a mesh for the object based on the depth map. ||
6. A non-transitory computer-readable storage medium storing a plurality of instructions executable by one or more processors, the plurality of instructions when executed by the one or more processors cause the one or more processors to: receive a light field represented by a plurality of images of an object, wherein the plurality of images include a first image and a second image, and wherein the second image is at a viewpoint different than the first image; identify a first point of the object in the first image, the first point being on an edge of the object, the edge identified by a high-gradient region in the first image; identify a second line in the second image based on the first point in the first image, the second line comprising an epipolar line based on the first point; identify one or more second points along the second line, the one or more second points comprise estimates where the first point is in the second image; identify one or more first points along a first line intersecting the first point, wherein each point of the one or more first points is projected from the one or more second points; determine that a second point of the one or more second points corresponds to the first point based on a color of each of the one or more first points and a color of each of the one or more second points; compute a depth for the first point based on the first point and the second point, wherein the depth is computed in response to determining that the second point corresponds to the first point, wherein a color gradient is computed using colors of the one or more first points and the one or more second points, and wherein the color gradient is used to compute the depth of the first point; propagate the depth to one or more points around the edge of the object based in part on a type of a color edge being a texture edge, wherein the texture edge is a boundary of the object that includes points on both sides of the boundary having similar depths; generate a depth map for the first image using the depth of the first point; and generate a mesh for the object based on the depth map. ||
11. A system comprising: one or more processors; and a non-transitory computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to: receive a light field represented by a plurality of images of an object, wherein the plurality of images include a first image and a second image, and wherein the second image is at a viewpoint different than the first image; identify a first point of the object in the first image, the first point being on an edge of the object, the edge identified by a high-gradient region in the first image; identify a second line in the second image based on the first point in the first image, the second line comprising an epipolar line based on the first point; identify one or more second points along the second line, the one or more second points comprise estimates where the first point is in the second image; identify one or more first points along a first line intersecting the first point, wherein each point of the one or more first points is projected from the one or more second points; determine that a second point of the one or more second points corresponds to the first point based on a color of each of the one or more first points and a color of each of the one or more second points; compute a depth for the first point based on the first point and the second point, wherein the depth is computed in response to determining that the second point corresponds to the first point, wherein a color gradient is computed using colors of the one or more first points and the one or more second points, and wherein the color gradient is used to compute the depth of the first point; propagate the depth to one or more points around the edge of the object based in part on a type of a color edge being a texture edge, wherein the texture edge is a boundary of the object that includes points on both sides of the boundary having similar depths; generate a depth map for the first image using the depth of the first point; and generate a mesh for the object based on the depth map. ||
16. A method for 3D object reconstruction, the method comprising: receiving a light field represented by a plurality of images of an object, wherein the plurality of images include a first image and a second image, and wherein the second image is at a viewpoint different than the first image; identifying a first point of the object in the first image, the first point being on an edge of the object, the edge identified by a high-gradient region in the first image; identifying a second line in the second image based on the first point in the first image, the second line comprising an epipolar line based on the first point identifying one or more second points along the second line, the one or more second points comprise estimates where the first point is in the second image; identifying one or more first points along a first line intersecting the first point, wherein each point of the one or more first points is projected from the one or more second points; determining that a second point of the one or more second points corresponds to the first point based on a color of each of the one or more first points and a color of each of the one or more second points; computing a depth for the first point based on the first point and the second point, wherein the depth is computed in response to determining that the second point corresponds to the first point, wherein a color gradient is computed using colors of the one or more first points and the one or more second points, and wherein the color gradient is used to compute the depth of the first point; propagating the depth to one or more points around the edge of the object based in part on a type of a color edge being a silhouette, wherein the propagating comprises determining the depth for the color edge in a single direction when the type of the color edge being the silhouette; generating a depth map for the first image using the depth of the first point and generating a mesh for the object based on the depth map.