FACE SWAPPING WITH NEURAL NETWORK-BASED GEOMETRY REFINING

Number: 20220374649
Published: 2022-11-24
Filed: 2021-09-24
Assignee: DISNEY ENTERPRISES, INC.
Inventors: NARUNIEC; Jacek Krzysztof, BRADLEY; Derek Edward, GOTARDO; Paulo Fabiano Urnau, HELMINGER; Leonhard Markus, OTTO; Christopher Andreas, SCHROERS; Christopher Richard, WEBER; Romann Matthew
CPC: G06F18/21; G06N3/045; G06N3/0455; G06N3/0464; G06N3/08; G06N3/088; G06N3/09; G06T11/00; G06T11/10; G06T17/20; G06T19/20; G06V10/774; G06V10/82; G06V20/647; G06V40/161; G06V40/172
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Neural face-swap VFX technique with geometry refinement.

Abstract

Various embodiments set forth systems and techniques for changing a face within an image. The techniques include receiving a first image including a face associated with a first facial identity; generating, via a machine learning model, at least a first texture map and a first position map based on the first image; rendering a second image including a face associated with a second facial identity based on the first texture map and the first position map, wherein the second facial identity is different from the first facial identity.

Background

BACKGROUND Technical Field

Embodiments of the present disclosure relate generally to computer science and computer graphics and, more specifically, to face swapping with neural network-based geometry refining. Description of the Related Art

Oftentimes, the facial identity of an individual needs to be changed in the frames of a video or in a standalone image while maintaining the performance of the individual within the video or image. As used herein, a “facial identity” refers to aspects of a facial appearance that arise from differences in personal identities, ages, lighting conditions, and the like. Thus, two different facial identities may be attributed to different individuals or to the same individual under different conditions, such as the same individual at different ages or under different lighting conditions. As used herein, the “performance” of an individual, which also is referred to as the dynamic “behavior” of an individual, includes the facial expressions and poses with which the individual appears in the frames of a video or in a standalone image.

One example scenario that requires the facial identity of an individual to be changed while maintaining the way the individual is performing is when the individual needs to be portrayed at a younger age in a particular scene within video content (e.g., a film, a show, etc.). As another example, an individual may be unavailable for a given video content production, and the face of that individual m

Claims

1. A computer-implemented method for changing a face within an image, the method comprising: receiving a first image including a face associated with a first facial identity; generating, via a machine learning model, at least a first texture map and a first position map based on the first image; and rendering a second image including a face associated with a second facial identity based on the first texture map and the first position map, wherein the second facial identity is different from the first facial identity. || 11. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: receiving a first image including a face associated with a first facial identity; generating, via a machine learning model, at least a first texture map and a first position map based on the first image; and rendering a second image including a face associated with a second facial identity based on the first texture map and the first position map, wherein the second facial identity is different from the first facial identity. || 20. A system comprising: one or more memories storing instructions; one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to: receive a first image including a face associated with a first facial identity; generate, via a machine learning model, at least a first texture map and a first position map based on the first image; and render a second image including a face associated with a second facial identity based on the first texture map and the first position map, wherein the second facial identity is different from the first facial identity.