TUNABLE MODELS FOR CHANGING FACES IN IMAGES

Number: 20210327038
Published: 2021-10-21
Filed: 2020-04-16
Assignee: DISNEY ENTERPRISES, INC.
Inventors: HELMINGER; Leonard Markus, NARUNIEC; Jacek Krzysztof, WEBER; Romann Matthew, SCHROERS; Christopher Richard
CPC: G06V40/169; G06T5/20; G06T9/00; G06T9/002; G06T11/60; G06V10/32; G06V10/454; G06V10/82; G06V40/168
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Tunable ML model for face synthesis/editing.

Abstract

Techniques are disclosed for changing the identities of faces in images. In embodiments, a tunable model for changing facial identities in images includes an encoder, a decoder, and dense layers that generate either adaptive instance normalization (AdaIN) coefficients that control the operation of convolution layers in the decoder or the values of weights within such convolution layers, allowing the model to change the identity of a face in an image based on a user selection. A separate set of dense layers may be trained to generate AdaIN coefficients for each of a number of facial identities, and the AdaIN coefficients output by different sets of dense layers can be combined to interpolate between facial identities. Alternatively, a single set of dense layers may be trained to take as input an identity vector and output AdaIN coefficients or values of weighs within convolution layers of the decoder.

Background

BACKGROUND Technical Field

Embodiments of the present disclosure relate generally to computer science and computer graphics and, more specifically, to tunable models for changing faces in images. Description of the Related Art

Oftentimes, the facial identity of an individual needs to be changed in the frames of a video or in a standalone image while maintaining a performance of the individual.

Claims

1. A computer-implemented method for changing a face within an image or video frame, the method comprising: encoding an input image that includes a face to generate a latent representation of the input image; and decoding the latent representation based on one or more parameters that are determined via at least one dense layer of a machine learning model to generate an output image. || 12. A non-transitory computer-readable storage medium including instructions that, when executed by a processing unit, cause the processing unit to perform steps for changing a face within an image or video frame, the steps comprising: encoding an input image that includes a face to generate a latent representation of the input image; and decoding the latent representation based on one or more parameters that are determined via at least one dense layer of a machine learning model to generate an output image. || 20. A computing device comprising: a memory storing an application; and a processor coupled to the memory, wherein when executed by the processor, the application causes the processor to: encode an input image that includes a face to generate a latent representation of the input image, and decode the latent representation based on one or more parameters that are determined via at least one dense layer of a machine learning model to generate an output image.