PARAMETRIC LANDMARK DETECTION

Number: 20260162369
Published: 2026-06-11
Filed: 2024-12-10
Assignee: DISNEY ENTERPRISES, INC.
Inventors: CHANDRAN; Prashanth, ZOSS; Gaspard, BRADLEY; Derek Edward
CPC: G06T7/75; G06T17/20; G06T13/40; G06T17/00; G06T19/20; G06T2200/04; G06T2207/20081; G06T2207/30201
Verdict: Low Notable software
First reported: 2026-W29 (2026-07-15)
Source: Google Patents · FreePatentsOnline

The keeper's note

One embodiment of the present invention sets forth a technique for performing landmark detection.

Abstract

One embodiment of the present invention sets forth a technique for performing landmark detection. The technique includes generating, via execution of a first machine learning model, a first set of morphable model coefficients associated with a first object depicted in a first image. The technique also includes determining one or more three-dimensional (3D) landmarks on the first object based on the first set of morphable model coefficients and projecting the first set of 3D landmarks onto the first image to generate one or more two-dimensional (2D) landmarks. The technique further includes training the first machine learning model based on one or more losses associated with the one or more 2D landmarks to generate a first trained machine learning model.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to techniques for performing parametric landmark detection. DESCRIPTION OF THE RELATED ART

Facial landmark detection refers to the detection of a set of specific key points, or landmarks, on a face that is depicted within an image and/or video. For example, a standard landmark detection technique may predict a set of 68 sparse landmarks that are spread across the face in a specific, predefined layout. The detected landmarks can then be used in various computer vision and computer graphics applications, such as (but not limited to) three-dimensional (3D) facial reconstruction, facial tracking, face swapping, segmentation, and/or facial re-enactment.

Deep learning approaches for predicting facial landmarks can generally be categorized into main types: direct prediction methods and heatmap prediction methods. In direct prediction methods, the x and y coordinates of the various landmarks are directly predicted by processing facial images. In heatmap prediction methods, the distribution of each landmark is first predicted, and the location of each landmark is subsequently extracted by maximizing that distribution function.

However, existing landmark detection techniques are associated with a number of drawbacks. First, most landmark detectors perform a face normalization pre-processing step th

Claims

1. A computer-implemented method for performing landmark detection, the method comprising: generating, via execution of a first machine learning model, a first set of morphable model coefficients associated with a first object depicted in a first image; determining one or more three-dimensional (3D) landmarks on the first object based on the first set of morphable model coefficients; projecting the one or more 3D landmarks onto the first image to generate one or more two-dimensional (2D) landmarks; and training the first machine learning model based on one or more losses associated with the one or more 2D landmarks to generate a first trained machine learning model. || 11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: generating, via execution of a first machine learning model, a first set of morphable model coefficients based on a first image depicting a first object; determining one or more three-dimensional (3D) landmarks on the first object based on the first set of morphable model coefficients; projecting the one or more 3D landmarks onto the first image to generate one or more two-dimensional (2D) landmarks; and training the first machine learning model based on one or more losses associated with the one or more 2D landmarks to generate a first trained machine learning model. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: determining a first machine learning model, wherein the first machine learning model is trained based on one or more losses associated with one or more landmarks corresponding to one or more sets of morphable model coefficients generated by the first machine learning model from a set of training images; generating, via execution of the first machine learning model, a second set of morphable model coefficients associated with an object depicted in an image; and generating a 3D shape associated with the object based on the second set of morphable model coefficients.