Non-coherent point tracking and solving for ambiguous images and geometry

Number: 10109062
Published: 2018-10-23
Filed: 2016-09-30
Assignee: LUCASFILM ENTERTAINMENT COMPANY LTD.
Inventors: Moore; Douglas
CPC: G06T7/246; G06T7/33; G06T7/73
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

CV point tracking for ambiguous geometry.

Abstract

System and methods are provided for a non-coherent point tracking process that allows unknown camera motion to be estimated. One or more edges can be identified in images captured by a camera when shooting a scene. For each of the identified edge in the images, at least one tracking object can be placed arbitrarily on the edge. The positions of tracking objects in the images can then be used to estimate a camera motion. In some embodiments, two tracking objects can be placed arbitrarily on the edge to represent the edge and move along the edge arbitrarily from image to image where the edge appears. Multiple of such edges can be identified in the images and camera motions in multiple directions can be estimated based on the identified edges and combined to obtain a combined camera motion indicating the camera's movement in a 3D space when shooting the scene.

Background

BACKGROUND OF THE INVENTION(1) This disclosure relates to recovering 3D structure and/or unknown camera motions.(2) In computer vision, structure from motion refers to a process of estimating camera motion and 3D structure by exploring the motion in a 2D image plane caused by the moving camera. The theory that underpins such a process is that a feature in the 2D image plane seen at a particular point by the camera actually lies along a particular ray beginning at the camera and extending out to infinity. When the same feature is seen in two different images, the camera motion with respect to that feature can be resolved. Using this process, any point seen in at least two images may also be located in 3D using triangulation.(3) However, conventional feature-based camera motion estimation algorithms typically require at least one identifiable feature to exist in two images so that the feature can be tracked in the images. This is limited in that geometry information of a fixed feature needs to be known for those algorithms to work well. Some of those conventional algorithms also require the camera's information be known, such as aspect ratio or field of view.(4) For example, Blender® is a tool that can be used to estimate camera motion and reconstruct a scene in 3D virtual space. Specifically, Blender® can let the user or automatically specify one or more tracking points for certain identifiable features in a series of images extracted from a video footage by marking those poin

Claims

1. A method of for estimating camera motion, the method being performed by a computer and comprising: receiving a plurality of images for a scene, the image including a first image; determining there is not at least one trackable feature in the images; in response to the determination that there is not at least one trackable feature in the images, identifying a first edge in the first image; for each image of the images, placing a first tracking object arbitrarily on the first edge; obtaining positions of the first tracking object in the images; and estimating a first camera motion based on the positions of the first tracking object in the images, wherein the first camera motion include at least two of the following motions: panning, tilting, rolling, moving horizontal, moving vertically, moving back and forth, or moving diagonally. 11. A system of for estimating camera motion, the system comprising one or more processors configured to execute machine-readable instructions such that when the machine-readable instructions are executed, the one or more processors are caused to perform: receiving a plurality of images for a scene, the image including a first image; determining there is not at least one trackable feature in the images; in response to the determination that there is not at least one trackable feature in the images, identifying a first edge in the first image; for each image of the images, placing a first tracking object arbitrarily on the first edge; obtaining positions of the first tracking object in the images; and estimating a first camera motion based on the positions of the first tracking object in the images, wherein the first camera motion include at least two of the following motions: panning, tilting, rolling, moving horizontal, moving vertically, moving back and forth, or moving diagonally.