TECHNIQUES FOR RE-AGING FACES IN IMAGES AND VIDEO FRAMES

Number: 20230080639
Published: 2023-03-16
Filed: 2021-09-13
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Zoss; Gaspard et al.
CPC: G06T17/00; G06T19/20; G06N3/0475; G06N3/08; G06N3/094; G06T5/60; G06T5/77; G06T7/149; G06T17/20
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Facial re-aging VFX technique.

Abstract

Techniques are disclosed for re-aging images of faces and three-dimensional (3D) geometry representing faces. In some embodiments, an image of a face, an input age, and a target age, are input into a re-aging model, which outputs a re-aging delta image that can be combined with the input image to generate a re-aged image of the face. In some embodiments, 3D geometry representing a face is re-aged using local 3D re-aging models that each include a blendshape model for finding a linear combination of sample patches from geometries of different facial identities and generating a new shape for the patch at a target age based on the linear combination. In some embodiments, 3D geometry representing a face is re-aged by performing a shape-from-shading technique using re-aged images of the face captured from different viewpoints, which can optionally be constrained to linear combinations of sample patches from local blendshape models.

Background

BACKGROUND Technical Field

Embodiments of the present disclosure relate generally to computer science and computer graphics and, more specifically, to techniques for re-aging faces in images and video frames. Description of the Related Art

Oftentimes, faces need to be digitally aged or de-aged in film productions and advertising, among other things. For example, the face of an actor could be aged or de-aged when the actor is playing a character whose age differs from an age of the actor, or for a particular scene that takes place when a character is at an older or younger age. As used herein, “re-aging” a face refers to either aging the face from a younger age to an older age or de-aging the face from an older age to a younger age.

One approach for re-aging faces in images or video frames is to manually re-age the faces by editing the images or video frames by hand. Alternatively, a three-dimensional (3D) re-aged face rig can be created by manually sculpting 3D geometry of the face that is captured using performance capture or keyframe animation techniques. The re-aged face rig can then be rendered from any desired viewpoint. However, when faces are manually re-aged in images or video frames, or using face rigs, the overall process of re-aging can be error-prone, tedious, and very time consuming.

Another approach for re-aging faces in images or video frames is to train an artificial neural network to automatically re-age faces in images or video fr

Claims

1. A computer-implemented method for re-aging a face included in a first image, the method comprising: generating, via a machine learning model, a second image based on (i) the first image that includes the face, (ii) an input age, and (iii) a target age, wherein the second image includes one or more differences from the first image; and combining the first image and the second image into a third image. || 11. A computer-implemented method for re-aging three-dimensional (3D) geometry representing a face, the method comprising: generating, via a machine learning model, a first image that includes a face at a target age based on a second image that includes the face at an input age; and deforming a 3D geometry representing the face at the input age based on the first image to generate a re-aged 3D geometry. || 19. One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processing units, cause the one or more processing units to perform steps for re-aging a face included in a first image, the steps comprising: generating, via a machine learning model, a second image based on (i) the first image that includes the face, (ii) an input age, and (iii) a target age, wherein the second image includes one or more differences from the first image; and combining the first image and the second image into a third image.