POINT CLOUD NOISE AND OUTLIER REMOVAL FOR IMAGE-BASED 3D RECONSTRUCTION

Number: 20180315168
Published: 2018-11-01
Filed: 2018-07-06
Assignee: Disney Enterprises, Inc.; ETH ZÜRICH (EIDGENÖSSISCHE TECHNISCHE HOCHSCHULE ZÜRICH)
Inventors: Kim; Changil; Sorkine-Hornung; Olga; Schroers; Christopher; Zimmer; Henning
CPC: G06T13/40; G06T15/06; G06T17/00; G06T17/20; G06T5/70
Verdict: Medium Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Point-cloud denoising for 3D reconstruction (PGPUB dup).

Abstract

Enhanced removing of noise and outliers from one or more point sets generated by image-based 3D reconstruction techniques is provided. In accordance with the disclosure, input images and corresponding depth maps can be used to remove pixels that are geometrically and/or photometrically inconsistent with the colored surface implied by the input images. This allows standard surface reconstruction methods (such as Poisson surface reconstruction) to perform less smoothing and thus achieve higher quality surfaces with more features. In some implementations, the enhanced point-cloud noise removal in accordance with the disclosure can include computing per-view depth maps, and detecting and removing noisy points and outliers from each per-view point cloud by checking if points are consistent with the surface implied by the other input views.

Background

BACKGROUND OF THE INVENTION

This disclosure relates to computer animation for creating a 3D model, in particular for creating a 3D model based on 2D images.

Acquiring the 3D geometry of real-world objects is generally known in the art. In computer vision, image-based scene reconstruction techniques are used to create a 3D model of a scene, given a set of 2D images of the scene. Many 3D reconstruction techniques are known in the art. For example, passive techniques that analyze a multitude of images of the scene and are referred to as, by those skilled in the art, multiview stereo or photogrammetry methods. These image-based methods can construct a 3D model relatively simply and cheaply by employing standard imaging hardware like consumer digital cameras. These image-based methods can provide color information of the scene and offer high resolution scanning thanks to the advances in image sensors. Most multi-view stereo methods filter, smooth, or denoise the reconstructed depth maps, and often these steps are integrated into the depth estimation stage and formulated as a (global) optimization problem.

One common approach employed by the multiview stereo methods for constructing a 3D model is to first compute camera poses and then estimate depth maps for all views by finding corresponding pixels between views and triangulating depth. Under that approach, all pixels are then projected into 3D space to obtain a point cloud from which a surface mesh can be extrac

Claims

1. A method for reconstructing a three-dimensional (3D) model of an object, the method being performed by a computer system and comprising: receiving images of the object, the images capturing the object from different viewpoints; obtaining a depth map for each of the images, the depth map including information relating to a surface of the object in the 3D space; projecting each pixel point in each depth map into a 3D space to obtain a point set; for each point in the point set, determining whether the point is consistent with the surface of the object in the 3D space as indicated by a set of depth maps for the images, wherein the consistency determination includes: for each depth map in the set: performing an intersection of a ray with the surface of the object indicated by the depth map to find an intersection point; and in response to the intersection point being found, determining a distance between the point and intersection point as a distance between the point and the surface of the object as seen from the camera viewpoint corresponding to the depth map; determining whether the point is consistent with the surface of the object based on one or more of the determined distances between the point and the surface of the object; and determining whether to discard the point from point set based on the determination whether the point is consistent with the surface of the object; and reconstructing the 3D model using the point set. 11. A system for reconstructing a three-dimensional (3D) model of an object, the system comprising one or more of a processor configured to execute machine-readable instructions such that when the machine-readable instructions are executed by the processor, the processor is caused to perform: receiving images of the object, the images capturing the object from different viewpoints; obtaining a depth map for each of the images, the depth map including information relating to a surface of the object in the 3D space; projecting each pixel point in each depth map into a 3D space to obtain a point set; for each point in the point set, determining whether the point is consistent with the surface of the object in the 3D space as indicated by a set of depth maps for the images, wherein the consistency determination includes: for each depth map in the set: performing an intersection of a ray with the surface of the object indicated by the depth map to find an intersection point; and in response to the intersection point being found, determining a distance between the point and intersection point as a distance between the point and the surface of the object as seen from the camera viewpoint corresponding to the depth map; determining whether the point is consistent with the surface of the object based on one or more of the determined distances between the point and the surface of the object; and determining whether to discard the point from point set based on the determination whether the point is consistent with the surface of the object; and reconstructing the 3D model using the point set.