GENERATING AN IMAGE INCLUDING A SOURCE INDIVIDUAL

Number: 20230377213
Published: 2023-11-23
Filed: 2023-05-18
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Naruniec; Jacek Krzysztof et al.
CPC: G06T11/00; G06V10/77; G06V10/774; G06V10/776; G06V10/82; G06V40/168; G06V40/172; G06V40/178
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Face/identity image-generation VFX technique.

Abstract

One embodiment of the present invention sets forth a technique for performing face swapping. The technique includes generating a latent representation of a first facial identity included in an input image. The technique further includes identifying a first identity-specific neural network layer associated with a second facial identity from a plurality of identity-specific neural network layers, wherein each neural network layer included in the plurality of identity-specific neural network layers is associated with a different facial identity. The technique further includes executing the first identity-specific neural network layer and one or more other neural network layers to generate one or more decoder input values corresponding to the latent representation. The technique further includes executing a decoder neural network that converts the one or more decoder input values into an output image depicting the second facial identity.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to techniques for generating an image including a source individual. Description of the Related Art

Face swapping refers to the changing of the facial identity of an individual in standalone images or video frames while maintaining the performance of the individual within the standalone images or video frames. This facial identity includes aspects of a facial appearance that arise from differences in personal identities, ages, eye colors, and/or other factors. For example, two different facial identities may be attributed to two different individuals, the same individual under different lighting conditions, and/or the same individual at different ages. Further, the performance of an individual, which also is referred to as the dynamic behavior of an individual, includes the facial expressions and poses of the individual, as depicted in the video frames or in standalone images.

Face swapping can be conducted under various types of scenarios. For example, the facial identity of an actor within a given scene of video content (e.g., a film, a show, etc.) could be changed to a different facial identify of the same actor at a younger age or at an older age. In another example, a first actor could be unavailable for a video shoot because of scheduling conflicts, because the first actor is deceased, and

Claims

1. A computer-implemented method for performing face swapping, the computer-implemented method comprising: generating a latent representation of a first facial identity included in an input image; identifying a first identity-specific neural network layer associated with a second facial identity from a plurality of identity-specific neural network layers, wherein each neural network layer included in the plurality of identity-specific neural network layers is associated with a different facial identity; executing the first identity-specific neural network layer and one or more other neural network layers to generate one or more decoder input values corresponding to the latent representation; and executing a decoder neural network that converts the one or more decoder input values into an output image depicting the second facial identity. || 14. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: generating a latent representation of a first facial identity included in an input image; executing one or more identity-independent neural network layers to generate one or more decoder input values corresponding to the latent representation; executing one or more layers associated with a second facial identity to generate one or more identity-specific decoder input values; and executing a decoder neural network that converts the one or more decoder input values and one or more identity-specific decoder input values into an output image depicting the second facial identity. || 19. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: generating a latent representation of a first facial identity included in an input image; identifying a first identity-specific neural network layer associated with a second facial identity from a plurality of identity-specific neural network layers, wherein each neural network layer included in the plurality of identity-specific neural network layers is associated with a different facial identity; executing the first identity-specific neural network layer and one or more other neural network layers to generate one or more decoder input values corresponding to the latent representation; and executing a decoder neural network that converts the one or more decoder input values into an output image depicting the second facial identity.