JOINT IMAGE NORMALIZATION AND LANDMARK DETECTION

Number: 20250118103
Published: 2025-04-10
Filed: 2024-10-04
Assignee: DISNEY ENTERPRISES, INC.
Inventors: CHANDRAN; Prashanth et al.
CPC: G06T3/02; G06T7/70; G06T11/60; G06V10/82; G06V40/165; G06V40/171
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Joint image normalization/landmark-detection facial technique.

Abstract

One embodiment of the present invention sets forth a technique for performing landmark detection. The technique includes applying, via execution of a first machine learning model, a first transformation to a first image depicting a first face to generate a second image. The technique also includes determining, via execution of a second machine learning model, a first set of landmarks on the first face based on the second image. The technique further includes training the first machine learning model based on one or more losses associated with the first set of landmarks to generate a first trained machine learning model.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to techniques for performing joint image normalization and landmark detection. Description of the Related Art

Facial landmark detection refers to the detection of a set of specific key points, or landmarks, on a face that is depicted within an image and/or video. For example, a standard landmark detection technique may predict a set of 68 sparse landmarks that are spread across the face in a specific, predefined layout. The detected landmarks can then be used in various computer vision and computer graphics applications, such as (but not limited to) three-dimensional (3D) facial reconstruction, facial tracking, face swapping, segmentation, and/or facial re-enactment.

Deep learning approaches for predicting facial landmarks can generally be categorized into main types: direct prediction methods and heatmap prediction methods. In direct prediction methods, the x and y coordinates of the various landmarks are directly predicted by processing facial images. In heatmap prediction methods, the distribution of each landmark is first predicted, and the location of each landmark is subsequently extracted by maximizing that distribution function.

However, existing landmark detection techniques are associated with a number of drawbacks. First, most landmark detectors perform a face normalization pre

Claims

1. A computer-implemented method for performing landmark detection, comprising: applying, via execution of a first machine learning model, a first transformation to a first image depicting a first face to generate a second image; determining, via execution of a second machine learning model, a first set of landmarks on the first face based on the second image; and training the first machine learning model based on one or more losses associated with the first set of landmarks to generate a first trained machine learning model. || 11. One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: applying, via execution of a first machine learning model, a first transformation to a first image depicting a first face to generate a second image; determining, via execution of a second machine learning model, a first set of landmarks on the first face based on the second image; and training the first machine learning model based on one or more losses associated with the first set of landmarks to generate a first trained machine learning model. || 20. A computer system, comprising: one or more memories that store instructions; and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: determining a first machine learning model and a second machine learning model, wherein the first machine learning model and the second machine learning model are trained based on one or more losses associated with a first set of landmarks generated by the second machine learning model from input that includes a transformed image generated via execution of the first machine learning model; applying, via execution of the first machine learning model, a transformation to a first image depicting a first face to generate a second image; and determining, via execution of the second machine learning model, a second set of landmarks on the first face based on the second image.