IDENTITY-PRESERVING IMAGE GENERATION USING DIFFUSION MODELS

Number: 20260330707
Published: 2026-10-22
Filed: 2026-07-02
Assignee: DISNEY ENTERPRISES, INC.
Inventors: KANSY; Manuel Jakob, RAËL; Anton Julien, NARUNIEC; Jacek Krzysztof, SCHROERS; Christopher Richard, WEBER; Romann Matthew
CPC: G06T11/00; G06T5/70; G06V10/82; G06V40/171; G06T2207/20081; G06T2207/30201; G06T2210/32; G06V2201/10
Verdict: Set aside dropped in weekly review
In edition: 2026-W43
Source: Google Patents · FreePatentsOnline

The keeper's note

One embodiment of the present invention sets forth a technique for performing identity-preserving image generation.

Abstract

One embodiment of the present invention sets forth a technique for performing identity-preserving image generation. The technique includes converting an identity image depicting a facial identity into an identity embedding. The technique further includes generating a combined embedding based on the identity embedding and a diffusion iteration identifier. The technique further includes converting, using a neural network and based on the combined embedding, a first input image that includes first noise into a first predicted image depicting one or more facial features that include one or more first facial identity features, wherein the one or more first facial identity features correspond to one or more respective second facial identity features of the identity image and are based at least on the identity embedding.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to techniques for generating images of individuals. Description of the Related Art

As used herein, a facial identity of an individual refers to an appearance of a face that is considered distinct from facial appearances of other entities due to differences in personal identity, age, or the like. Two facial identities may be of different individuals or the same individual under different conditions, such as the same individual at different ages.

A face of an individual depicted in an image, such as a digital photograph or a frame of digital video, can be converted to a value referred to as an identity embedding in a latent space. The identity embedding includes a representation of identity features of the individual and can, in practice, also include other features that do not represent the individual. The identity embedding can be generated by a machine-learning model such as a face recognition model. The identity embedding does not have a readily-discernible correspondence to the image, since the identity embedding depends on the particular configuration of the network in addition to the content of the image. Although the identity embedding does not necessarily encode sufficient information to reconstruct the exact original image from the embedding, reconstruction of an approximation of the ori

Claims

1. A computer-implemented method for training a machine learning model, the computer-implemented method comprising: generating a combined embedding based on (i) a ground-truth identity vector associated with a facial identity depicted in a clean image and (ii) a diffusion iteration identifier; converting, using a neural network and based on the combined embedding, an input image that includes noise into a predicted clean image; generating a predicted identity vector from the predicted clean image; determining an identity loss based on a difference between the predicted identity vector and the ground-truth identity vector; and updating one or more parameters of the neural network based at least on the identity loss. || 11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: generating a combined embedding based on (i) a ground-truth identity vector associated with a facial identity depicted in a clean image and (ii) a diffusion iteration identifier; converting, using a neural network and based on the combined embedding, an input image that includes noise into a predicted clean image; generating a predicted identity vector from the predicted clean image; determining an identity loss based on a difference between the predicted identity vector and the ground-truth identity vector; and updating one or more parameters of the neural network based at least on the identity loss. || 18. A system comprising: one or more memories storing instructions; and one or more processors for executing the instructions to: generate a combined embedding based on (i) a ground-truth identity vector associated with a facial identity depicted in a clean image and (ii) a diffusion iteration identifier; convert, using a neural network and based on the combined embedding, an input image that includes noise into a predicted clean image; generate a predicted identity vector from the predicted clean image; determine an identity loss based on a difference between the predicted identity vector and the ground-truth identity vector; and update one or more parameters of the neural network based at least on the identity loss.