Application (pre-grant publication)
FLEXIBLE LANDMARK DETECTION
- Number
- 20240161540
- Published
- 2024-05-16
- Filed
- 2023-11-08
- Assignee
- DISNEY ENTERPRISES, INC.
- Inventors
- Bradley; Derek Edward et al.
- CPC
- G06T17/00; G06V40/165; G06V40/168; G06T7/11; G06V40/171; G06T7/20; G06V10/82
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Flexible facial-landmark detection technique.
Abstract
One or more embodiments comprise a computer-implemented method that includes receiving an input image including one or more facial representations and a set of points on a 3D canonical shape, wherein the set of points are selectable at runtime, extracting a set of features from the input image that represent at least one facial representation included in the one or more facial representations, and determining a set of landmarks on the at least one facial representation based on the set of features and the set of points, wherein each landmark in the set of landmarks is associated with at least one point in the set of points.
Background
BACKGROUND Field of the Various Embodiments
The various embodiments relate generally to landmark detection on images and, more specifically, to techniques for flexible landmark detection on images at runtime. DESCRIPTION OF THE RELATED ART
Many computer vision and computer graphics applications rely on landmark detection on images. Such applications include three-dimensional (3D) facial reconstruction, tracking, face swapping, segmentation, re-enactment, or the like. Landmarks, such as facial landmarks, can be used as anchoring points for models, such as, 3D face appearance or autoencoders. Locations of landmarks are used, for instance, to spatially align faces. In some applications, facial landmarks are important for enabling visual effects on faces, for tracking eye gaze, or the like.
Some approaches for facial landmark detection involve deep learning techniques. These techniques can generally be categorized into main types: direct prediction methods and heatmap prediction methods. In direct prediction methods, the x and y coordinates of the various landmarks are directly predicted by processing facial images. In heatmap prediction methods, the distribution of each landmark is first predicted and then the location of each landmark is extracted by maximizing that distribution function.
One drawback to these approaches is that the predicted landmarks are fixed and follow a pre-determined layout. For example, facial landmarks are often predicted as