Geometry tracking

Number: 9792479
Published: 2017-10-17
Filed: 2014-01-29
Assignee: LUCASFILM ENTERTAINMENT COMPANY LTD.
Inventors: Mallet; Ronald, Bhat; Kiran S., Wooley; Kevin
CPC: G06T19/00; G06T7/251
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

A motion capture technique that tracks 3D geometry changes by correlating a 3D subject model against its 2D representation across video frames.

Abstract

A method of motion capture may include accessing a 3D model of a subject, and associating the 3D model of the subject with a 2D representation of the subject in a plurality of frames. The method may also include identifying a change to the 2D representation of the subject between two or more of the plurality of frames, and deforming the 3Dmodel in a virtual 3D space. In some embodiments, the deforming may be based on the identified change to the 2D representation and at least one constraint restricting how the 3D model can be deformed.

Background

DESCRIPTION OF DRAWINGS(1) FIG. 1 includes an example system that can be used to track geometry in a video sequence.(2) FIG. 2 is a schematic diagram of an example video in which textures are tracked.(3) FIG. 3 is a schematic diagram of an example video in which the location, orientation, and shape of a model is calculated.(4) FIG. 4 is a flowchart of an example process of tracking geometry in a video.(5) FIG. 5 is a flowchart of an example process of initializing data for a video with geometry to be tracked.(6) FIG. 6 is a block diagram of a computing systemthat can be used in connection with computer-implemented methods described in this document.(7) Like reference symbols in the various drawings indicate like elements.DETAILED DESCRIPTION(8) A geometry tracker may use a constrained tracking framework to track the location, orientation, and shape of an object in a video sequence for matchmove, motion capture,or other uses. The geometry information may be used, for example, in a motion capture phase of a moviemaking project to capture the motion of a dynamic subject including, but not limited to a human actor. Some implementations use a hybrid 2D/3D approach for tracking anobject through a video sequence despite the object being deformed and/or undergoing appearance changes. For example, a three-dimensional model (3D) representing the subject can betracked from frame to frame in the video; a texture tracker can track the apparent movements of texture regions; and those m

Claims

1. A computer implemented method of motion capture, the method comprising: accessing a 3D model; associating the 3D model with a 2D representation of a subject in a plurality of frames, wherein the plurality of frames captures the subject in a real 3D space; identifying one or more motion vectors derived from a change to the position of the 2D representation of the subject between two or more of the plurality of frames; and deforming the 3D model in a virtual 3D space, wherein the deforming is based on: the identified change to the position of the 2D representation; and at least one constraint restricting how the 3D model can be deformed, wherein the at least one constraint comprises the one or more motion vectors. 13. A motion capture system, the system comprising: a geometry tracker configured to: access a 3D model; associate the 3D model with a 2D representation of a subject in a plurality of frames, wherein the plurality of frames capturesthe subject in a real 3D space; and deform the 3D model in a virtual 3D space, wherein the deforming is based on: an identified change to the position of the 2D representation; and at least one constraint restricting how the 3D model can be deformed, wherein the at least one constraint comprises one or more motion vectors; and a texture tracker configured to: identifying the one or more motion vectors derived from the change to the position of the 2D representation of the subject between two or more of the plurality of frames. 17. A non-transitory computer readable medium comprising instructions stored thereon which, whenexecuted by one or more processing devices, cause the one or more processing devices to perform operations comprising: accessing a 3D model; associating the 3D model with a 2D representation of a subject in a plurality of frames, wherein the plurality of frames captures the subject in a real 3D space; identifying one or more motion vectors derived from a change to the position of the 2D representation of the subject between two or more of the plurality of frames; and deforming the 3D model in a virtual 3D space, wherein the deformingis based on: the identified change to the position of the 2D representation; and at leastone constraint restricting how the 3D model can be deformed, wherein the at least one constraint comprises the one or more motion vectors.