Reconstruction of articulated objects from a moving camera

Number: 9747668
Published: 2017-08-29
Filed: 2016-01-21
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Yucer; Kaan, Sorkine Hornung; Alexander, Wang; Oliver, Sorkine Hornung; Olga
CPC: G06T15/205; G06T7/579; G06T17/00
Verdict: Medium Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Reconstructs an articulated object (e.g. a robot or animatronic figure) from 2D feature points tracked across moving-camera footage over time (granted).

Abstract

Systems and method for the reconstruction of an articulated object are disclosed herein, The articulated object can be reconstructed from image data collected by a moving camera over a period of time. A plurality of 2D feature points can beidentified within the image data. These 2D feature points can be converted into three-dimensional space, which converted points are identified as 3D feature points. These 3D feature points can be used to identify one or several rigidity constrains and/or kinematic constraints. These rigidity and/or kinematic constraints can be applied to a model of the reconstructed articulated object.

Background

BRIEF DESCRIPTION OF THE DRAWINGS(1) FIG. 1 is a schematic illustration of one embodiment of a system for the virtual reconstruction of an articulated object.(2) FIG. 2 isa perspective view of one embodiment of an exemplary articulated object in a first position.(3) FIG. 3 is a perspective view of one embodiment of an exemplary articulated object in a second position.(4) FIG. 4 is a flowchart illustrating one embodiment of a high-level process for the virtual reconstruction of an articulated object.(5) FIG. 5 is a flowchart illustrating one embodiment of a process for the virtual reconstruction of an articulated object using image data.(6) FIG. 6 is a flowchart illustrating one embodiment of a processfor converting 2D points to 3D points.(7) FIG. 7 is a flowchart illustrating one embodiment of a process for generating rigidity constraints.(8) FIG. 8 is a flowchart illustratingone embodiment of a process for applying rigidity constraints.(9) FIG. 9 is a flowchart illustrating one embodiment of a process for generating and applying a kinematic constraint.(10) FIG. 10 includes a series of images showing the results of the application of methods disclosed herein.(11) FIG. 11 is a block diagram of a computer system or information processing device that may incorporate an embodiment, be incorporated into an embodiment, or be used to practice any of the innovations, embodiments, and/or examples found within thisdisclosure.DEFINITIONS(12) As used herein, an “articulated object” i

Claims

1. A method of reconstructing an articulated object from images captured with a moving camera, the method comprising: generating a plurality of 2D tracks, wherein at least one of the 2D tracks of the plurality of 2D tracks comprises a plurality of 2D feature points, wherein one of the 2D feature points identifies a two-dimensional location of a unique one of a series of points on an articulated object in aone of a sequenced series of captured images, wherein the sequenced series of captured images are captured by a moving camera, and wherein the moving camera is moving with respectto at least a fixed portion of a background of the images forming the sequenced series ofcaptured images; generating a manifold of rays comprising a plurality of rays, wherein the rays of the manifold of rays are defined in part as extending through a unique one of the data points of one of the 2D tracks; converting at least some of the plurality of 2D data points of the plurality of 2D tracks to 3D data points by identifying a position for at least one of the plurality of 2D data points along one of the plurality of rays forming the manifold of rays; and generating a first 3D representation of the articulated object foreach of the images of the sequenced series of captured images, wherein the first 3D representation comprises the 3D data points converted from 2D data points gathered from one of the images of the sequenced series of captured images. 12. A system for reconstructing an articulated object from images captured with a moving camera, the method comprising: memory comprising stored instructions; and a processor configured to: generate a plurality of 2D tracks, wherein at least one of the 2D tracks of the plurality of 2D tracks comprises a plurality of 2D data points identifying 2D locations of a unique one of a series of pointson an articulated object in a one of a sequenced series of captured images, wherein the sequenced series of captured images are captured by a moving camera, and wherein the movingcamera is moving with respect to at least a fixed portion of a background of the images forming the sequenced series of captured images; generate a manifold of rays comprising a plurality of rays, wherein the rays of the manifold of rays are defined in part as extending through a unique one of the data points of one of the 2D tracks; convert at least some of the plurality of 2D data points of the plurality of 2D tracks to 3D data points by identifying a position for at least some of the plurality of 2D data points along the each of the plurality of rays forming the manifold of rays; and generate a first 3D representation of the articulated object for each of the images of the sequenced series of captured images, wherein the first 3D representation comprises the 3D data points converted from 2D datapoints gathered from one of the images of the sequenced series of captured images.