SHAPE RECONSTRUCTION AND EDITING USING ANATOMICALLY CONSTRAINED IMPLICIT SHAPE MODELS

Number: 20250037375
Published: 2025-01-30
Filed: 2024-07-22
Assignee: DISNEY ENTERPRISES, INC.
Inventors: ZOSS; Gaspard et al.
CPC: G06T17/20; G06T19/20
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Shape reconstruction/editing using implicit shape models (companion patent).

Abstract

One embodiment of the present invention sets forth a technique for fitting a shape model for an object to a set of constraints associated with a target shape. The technique includes determining, based on the set of constraints, one or more ground truth positions of one or more points on the target shape. The technique also includes generating, via execution of a set of neural networks, a set of fitting parameters associated with the point(s) and computing, via the shape model, one or more predicted positions of the point(s) based on the set of fitting parameters. The technique further includes training the set of neural networks based on one or more losses associated with the predicted position(s) and the ground truth position(s) and generating, via execution of the trained set of neural networks, a three-dimensional (3D) model corresponding to the target shape.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to anatomically constrained implicit shape models. Description of the Related Art

Realistic digital representations of faces, hands, bodies, and other recognizable objects are required for various computer graphics and computer vision applications. For example, digital representations of real-world deformable objects may be used in virtual scenes of film or television productions, video games, virtual worlds, and/or other environments and/or settings.

One technique for representing a digital shape involves using a data-driven parametric shape model to characterize realistic variations in the appearance of the shape. The data-driven parametric shape model is typically built from a dataset of scans of the same type of shape and represents a new shape as a combination of existing shapes in the dataset.

One common parametric shape model includes a linear three-dimensional (3D) morphable model (3DMM) that expresses new faces, bodies, and/or other shapes as linear combinations of prototypical basis shapes from a dataset. However, the linear 3D morphable model is unable to represent continuous, nonlinear deformations that are common to faces and other recognizable shapes. At the same time, linear combinations of input shapes generated by the linear 3D morphable model can lead to unrealistic moti

Claims

1. A computer-implemented method for fitting a shape model for an object to a set of constraints associated with a target shape, the method comprising: determining, based on the set of constraints, one or more ground truth positions of one or more points on the target shape; generating, via execution of a set of neural networks, a set of fitting parameters associated with the one or more points; computing, via the shape model, one or more predicted positions of the one or more points based on the set of fitting parameters; training the set of neural networks based on one or more losses associated with the one or more predicted positions and the one or more ground truth positions; and generating, via execution of the trained set of neural networks, a three-dimensional (3D) model corresponding to the target shape. || 11. One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: determining one or more ground truth positions of one or more points on a target shape associated with an object; generating, via execution of a set of neural networks, a set of fitting parameters associated with the one or more points; computing, via a shape model, one or more predicted positions of the one or more points based on the set of fitting parameters; training the set of neural networks based on one or more losses associated with the one or more predicted positions and the one or more ground truth positions; and generating, via execution of the trained set of neural networks, a three-dimensional (3D) model corresponding to the target shape. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform operations comprising: determining one or more ground truth positions of one or more points on a target shape associated with an object; generating, via execution of a set of neural networks, a set of fitting parameters associated with the one or more points; computing, via a shape model, one or more predicted positions of the one or more points based on the set of fitting parameters; training the set of neural networks based on one or more losses associated with the one or more predicted positions and the one or more ground truth positions; and generating, via execution of the trained set of neural networks, a three-dimensional (3D) model corresponding to the target shape.