Obtaining high resolution and dense reconstruction of face from sparse facial markers

Number: 11170571
Published: 2021-11-09
Filed: 2019-11-15
Assignee: LUCASFILM ENTERTAINMENT COMPANY LTD. LLC
Inventors: Cong; Matthew, Fedkiw; Ronald, Lan; Lana
CPC: G06F18/22; G06T17/20; G06T5/50; G06T7/246; G06T7/55; G06T7/75; G06V10/761; G06V10/774; G06V20/647; G06V40/171; G06V40/172
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Dense facial reconstruction from sparse facial markers (VFX, granted).

Abstract

Some implementations of the disclosure are directed to techniques for facial reconstruction from a sparse set of facial markers. In one implementation, a method comprises: obtaining data comprising a captured facial performance of a subject with a plurality of facial markers; determining a three-dimensional (3D) bundle corresponding to each of the plurality of facial markers of the captured facial performance; using at least the determined 3D bundles to retrieve, from a facial dataset comprising a plurality of facial shapes of the subject, a local geometric shape corresponding to each of the plurality of the facial markers; and merging the retrieved local geometric shapes to create a facial reconstruction of the subject for the captured facial performance.

Background

BRIEF SUMMARY OF THE DISCLOSURE (1) Implementations of the disclosure describe improved techniques for facial reconstruction from a sparse set of facial markers. In one embodiment, a method comprises: obtaining data comprising a captured facial performance of a subject with a plurality of facial markers; determining a three-dimensional (3D) bundle corresponding to each of the plurality of facial markers of the captured facial performance; using at least the determined 3D bundles to retrieve, from a facial dataset comprising a plurality of facial shapes of the subject, a local geometric shape corresponding to each of the plurality of the facial markers; and merging the retrieved local geometric shapes to create a facial reconstruction of the subject for the captured facial performance. (2) In some implementations, retrieving the local geometric shape corresponding to each of the plurality of the facial markers, comprises: evaluating a surface position of each of the 3D bundles on each of the plurality of facial shapes in the facial dataset to derive a point cloud corresponding to each of the 3D bundles; creating a tetrahedral mesh from each of the point clouds; and using each of the created tetrahedral meshes to retrieve, from the facial dataset, the local geometric shape corresponding to each of the facial markers. (3) In some implementations, the created tetrahedral mesh is a non-manifold tetrahedral mesh. (4) In some implementations, creating the tetrahedral mesh from each

Claims

1. A method, comprising: obtaining data comprising a captured facial performance of a subject with a plurality of facial markers; determining a three-dimensional (3D) bundle corresponding to each of the plurality of facial markers of the captured facial performance; using at least the determined 3D bundles to retrieve, from a facial dataset comprising a plurality of facial shapes of the subject, a local geometric shape corresponding to each of the plurality of the facial markers; and merging the retrieved local geometric shapes to create a facial reconstruction of the subject for the captured facial performance; wherein retrieving the local geometric shape corresponding to each of the plurality of the facial markers, comprises: evaluating a surface position of each of the 3D bundles on each of the plurality of facial shapes in the facial dataset to derive a point cloud corresponding to each of the 3D bundles; creating a tetrahedral mesh from each of the point clouds; and using each of the created tetrahedral meshes to retrieve, from the facial dataset, the local geometric shape corresponding to each of the facial markers. || 12. A non-transitory computer-readable medium having executable instructions stored thereon that, when executed by a processor, performs a method comprising: obtaining data comprising a captured facial performance of a subject with a plurality of facial markers; determining a three-dimensional (3D) bundle corresponding to each of the plurality of facial markers of the captured facial performance; using at least the determined 3D bundles to retrieve, from a facial dataset comprising a plurality of facial shapes of the subject, a local geometric shape corresponding to each of the plurality of the facial markers; and merging the retrieved local geometric shapes to create a facial reconstruction of the subject for the captured facial performance; wherein retrieving the local geometric shape corresponding to each of the plurality of the facial markers, comprises: evaluating a surface position of each of the 3D bundles on each of the plurality of shapes in the facial dataset to derive a point cloud corresponding to each of the 3D bundles; creating a tetrahedral mesh from each of the point clouds; and using each of the created tetrahedral meshes to retrieve, from the facial dataset, the local geometric shape corresponding to each of the facial markers.