Techniques for re-aging faces in images and video frames

Number: 12307616
Published: 2025-05-20
Filed: 2021-09-13
Assignee: Disney Enterprises, INC.
Inventors: Zoss; Gaspard et al.
CPC: G06T17/00; G06T19/20; G06N3/0475; G06N3/08; G06N3/094; G06T5/60; G06T5/77; G06T7/149; G06T17/20
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Face re-aging VFX technique.

Abstract

Techniques are disclosed for re-aging images of faces and three-dimensional (3D) geometry representing faces. In some embodiments, an image of a face, an input age, and a target age, are input into a re-aging model, which outputs a re-aging delta image that can be combined with the input image to generate a re-aged image of the face. In some embodiments, 3D geometry representing a face is re-aged using local 3D re-aging models that each include a blendshape model for finding a linear combination of sample patches from geometries of different facial identities and generating a new shape for the patch at a target age based on the linear combination. In some embodiments, 3D geometry representing a face is re-aged by performing a shape-from-shading technique using re-aged images of the face captured from different viewpoints, which can optionally be constrained to linear combinations of sample patches from local blendshape models.

Background

BACKGROUND Technical Field (1) Embodiments of the present disclosure relate generally to computer science and computer graphics and, more specifically, to techniques for re-aging faces in images and video frames. Description of the Related Art (2) Oftentimes, faces need to be digitally aged or de-aged in film productions and advertising, among other things. For example, the face of an actor could be aged or de-aged when the actor is playing a character whose age differs from an age of the actor, or for a particular scene that takes place when a character is at an older or younger age. As used herein, “re-aging” a face refers to either aging the face from a younger age to an older age or de-aging the face from an older age to a younger age. (3) One approach for re-aging faces in images or video frames is to manually re-age the faces by editing the images or video frames by hand. Alternatively, a three-dimensional (3D) re-aged face rig can be created by manually sculpting 3D geometry of the face that is captured using performance capture or keyframe animation techniques. The re-aged face rig can then be rendered from any desired viewpoint. However, when faces are manually re-aged in images or video frames, or using face rigs, the overall process of re-aging can be error-prone, tedious, and very time consuming. (4) Another approach for re-aging faces in images or video frames is to train an artificial neural network to automatically re-age faces in images or video frames that ar

Claims

1. A computer-implemented method for re-aging a face included in a first image, the method comprising: generating an input age image corresponding to an input age associated with the face in the first image and a target age image associated with a target age of the face in the first image, wherein the input age image includes a first plurality of pixels having age values indicating the input age and the target age image includes a second plurality of pixels having age values indicating the target age, and wherein the first image, the input age image, and the target age image have the same spatial resolution; generating, via a machine learning model, a second image based on a multi-channel tensor provided to the machine learning model at execution, wherein the multi-channel tensor comprises (i) the first image that includes the face, (ii) an input age image, and (iii) a target age image, wherein the second image includes one or more differences from the first image; and combining the first image and the second image into a third image. || 12. One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processing units, cause the one or more processing units to perform steps for re-aging a face included in a first image, the steps comprising: generating an input age image corresponding to an input age associated with the face in the first image and a target age image associated with a target age of the face in the first image, wherein the input age image includes a first plurality of pixels having age values indicating the input age and the target age image includes a second plurality of pixels having age values indicating the target age, and wherein the first image, the input age image, and the target age image have the same spatial resolution; generating, via a machine learning model, a second image based on a multi-channel tensor provided to the machine learning model at execution, wherein the multi-channel tensor comprises (i) the first image that includes the face, (ii) an input age image, and (iii) a target age image, wherein the second image includes one or more differences from the first image; and combining the first image and the second image into a third image.