QUERY DEFORMATION FOR LANDMARK ANNOTATION CORRECTION

Number: 20250118102
Published: 2025-04-10
Filed: 2024-10-04
Assignee: DISNEY ENTERPRISES, INC.
Inventors: CHANDRAN; Prashanth et al.
CPC: G06T3/02; G06T7/70; G06T11/60; G06V10/82; G06V40/165; G06V40/171
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Landmark-annotation-correction facial technique (companion patent).

Abstract

One embodiment of the present invention sets forth a technique for performing landmark detection. The technique includes generating, via execution of a first machine learning model, a first set of displacements associated with a first set of query points on a canonical shape based on a first annotation style associated with the first set of query points. The technique also includes determining, via execution of a second machine learning model, a first set of landmarks on a first face depicted in a first image based on the first set of displacements. The technique further includes training the first machine learning model based on one or more losses associated with the first set of landmarks to generate a first trained machine learning model.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to techniques for performing query deformation for landmark annotation correction. DESCRIPTION OF THE RELATED ART

Facial landmark detection refers to the detection of a set of specific key points, or landmarks, on a face that is depicted within an image and/or video. For example, a standard landmark detection technique may predict a set of 68 sparse landmarks that are spread across the face in a specific, predefined layout. The detected landmarks can then be used in various computer vision and computer graphics applications, such as (but not limited to) three-dimensional (3D) facial reconstruction, facial tracking, face swapping, segmentation, and/or facial re-enactment.

Deep learning approaches for predicting facial landmarks can generally be categorized into main types: direct prediction methods and heatmap prediction methods. In direct prediction methods, the x and y coordinates of the various landmarks are directly predicted by processing facial images. In heatmap prediction methods, the distribution of each landmark is first predicted, and the location of each landmark is subsequently extracted by maximizing that distribution function.

However, existing landmark detection techniques are associated with a number of drawbacks. First, most landmark detectors perform a face normalization

Claims

1. A computer-implemented method for performing landmark detection, the method comprising: generating, via execution of a first machine learning model, a first set of displacements associated with a first set of query points on a canonical shape based on a first annotation style associated with the first set of query points; determining, via execution of a second machine learning model, a first set of landmarks on a first face depicted in a first image based on the first set of displacements; and training the first machine learning model based on one or more losses associated with the first set of landmarks to generate a first trained machine learning model. || 11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: generating, via execution of a first machine learning model, a first set of displacements associated with a first set of query points on a canonical shape based on a first annotation style associated with the first set of query points; determining, via execution of a second machine learning model, a first set of landmarks on a first face depicted in a first image based on the first set of displacements; and training the first machine learning model based on one or more losses associated with the first set of landmarks to generate a first trained machine learning model. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: generating, via execution of a first machine learning model, a first set of displacements associated with a first set of query points on a canonical shape based on a first annotation style associated with the first set of query points; determining, via execution of a second machine learning model, a first set of landmarks on a first face depicted in a first image based on the first set of displacements; and training the first machine learning model and the second machine learning model based on one or more losses associated with the first set of landmarks.