- Number
- 12111880
- Published
- 2024-10-08
- Filed
- 2021-09-24
- Assignee
- DISNEY ENTERPRISES, INC.
- Inventors
- Naruniec; Jacek Krzysztof et al.
- CPC
- G06F18/21; G06N3/045; G06N3/0455; G06N3/0464; G06N3/08; G06N3/088; G06N3/09; G06T11/00; G06T11/10; G06T17/20; G06T19/20; G06V10/774; G06V10/82; G06V20/647; G06V40/161; G06V40/172
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Neural face-swap VFX technique with geometry refinement (granted).
Abstract
Various embodiments set forth systems and techniques for changing a face within an image. The techniques include receiving a first image including a face associated with a first facial identity; generating, via a machine learning model, at least a first texture map and a first position map based on the first image; rendering a second image including a face associated with a second facial identity based on the first texture map and the first position map, wherein the second facial identity is different from the first facial identity.
Background
BACKGROUND Technical Field (1) Embodiments of the present disclosure relate generally to computer science and computer graphics and, more specifically, to face swapping with neural network-based geometry refining. Description of the Related Art (2) Oftentimes, the facial identity of an individual needs to be changed in the frames of a video or in a standalone image while maintaining the performance of the individual within the video or image. As used herein, a “facial identity” refers to aspects of a facial appearance that arise from differences in personal identities, ages, lighting conditions, and the like. Thus, two different facial identities may be attributed to different individuals or to the same individual under different conditions, such as the same individual at different ages or under different lighting conditions. As used herein, the “performance” of an individual, which also is referred to as the dynamic “behavior” of an individual, includes the facial expressions and poses with which the individual appears in the frames of a video or in a standalone image. (3) One example scenario that requires the facial identity of an individual to be changed while maintaining the way the individual is performing is when the individual needs to be portrayed at a younger age in a particular scene within video content (e.g., a film, a show, etc.). As another example, an individual may be unavailable for a given video content production, and the face of that individual may need t
Claims
1. A computer-implemented method for changing a face within an image, the method comprising: receiving a first image including a face associated with a first facial identity; generating, via a machine learning model, at least a first texture map, a second texture map, and a first position map based on the first image, wherein the second texture map represents one or more adjustments to the first texture map; and rendering a second image including a face associated with a second facial identity based on the first texture map, the first position map, and the second texture map, wherein the second facial identity is different from the first facial identity. ||
3. A computer-implemented method for changing a face within an image, the method comprising: receiving a first image including a face associated with a first facial identity; generating, via a machine learning model, wherein the machine learning model comprises a decoder including a plurality of sets of layers, wherein each set of layers included in the plurality of sets of layers is associated with a different facial identity, and wherein generating the first texture map and the first position map uses a first set of layers included in the plurality of sets of layers that is associated with the second facial identity; and rendering a second image including a face associated with a second facial identity based on the first texture map and the first position map, wherein the second facial identity is different from the first facial identity. ||
11. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: receiving a first image including a face associated with a first facial identity; generating, via a machine learning model, at least a first texture map, a second texture map, and a first position map based on the first image, wherein the second texture map comprises one or more adjustments to the first texture map; and rendering a second image including a face associated with a second facial identity based on the first texture map, the first position map, and the second texture map, wherein the second facial identity is different from the first facial identity. ||
20. A system comprising: one or more memories storing instructions; one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to: receive a first image including a face associated with a first facial identity; generate, via a machine learning model, at least a first texture map, a second texture map, and a first position map based on the first image, wherein the second texture map comprises one or more adjustments to the first texture map; and render a second image including a face associated with a second facial identity based on the first texture map, the first position map, and the second texture map, wherein the second facial identity is different from the first facial identity.