WAVELET-BASED AUTOENCODERS FOR LATENT DIFFUSION MODELS

Number: 20250252533
Published: 2025-08-07
Filed: 2025-01-31
Assignee: Disney Enterprises, Inc.
Inventors: SADAT; Seyedmorteza et al.
CPC: G06T5/10; G06V10/44; G06T5/60; G06V10/52; G06V10/806
Verdict: Set aside generic diffusion-model research
Source: Google Patents · FreePatentsOnline

Abstract

The computational requirements of an encoder of an autoencoder can be reduced by pre-processing the images using a discrete wavelet transform (DWT). In one embodiment, the encoder uses a multi-level DWT to extract multiscale information from the input images. If using a learned encoder, performing the multi-level DWT enables the encoder to have less complex feature extraction and aggregation networks (e.g., convolution neural networks (CNNs)) than a standard encoder for an autoencoder. This means the VAE can execute faster, use less computational resources (such as GPU memory), and use less power than traditional VAEs. If using a non-learned encoder, the result of the multi-level DWT can be used as the latent code without using feature extraction and aggregation networks.

Background

BACKGROUND

Autoencoders are a type of neural network primarily designed to compress input data into a lower-dimensional representation (latent space) and then reconstruct the original data from that compressed version, extracting key features from the data while learning important patterns within it. Autoencoders are commonly used for tasks like dimensionality reduction, data denoising, anomaly detection, and feature extraction in images and other data types.

Autoencoders are often used with latent diffusion models (LDMs) which are a type of generative artificial intelligence (AI) model used primarily for image synthesis. LDMs have assumed dominance in the field of high-resolution image generation, primarily due to their scalability and training stability over pixel-space diffusion. SUMMARY

One embodiment described herein is a method that includes transforming an input image into a latent code using an encoder of an autoencoder by performing a multi-level discrete wavelet transform (DWT) on the input image; generating, based on the latent code, a reconstructed latent code using a LDM; and training the LDM using a denoising loss based on comparing the reconstructed latent code and the latent code.

Another embodiment described herein is a non-transitory computer readable medium containing computer program code that, when executed by operation of one or more computer processors performs operations. The operations includes transforming an input image i

Claims

1. A method, comprising: transforming an input image into a latent code using an encoder of an autoencoder by performing a multi-level discrete wavelet transform (DWT) on the input image; generating, based on the latent code, a reconstructed latent code using a latent diffusion model (LDM); and training the LDM using a denoising loss based on comparing the reconstructed latent code and the latent code. || 9. A non-transitory computer readable medium containing computer program code that, when executed by operation of one or more computer processors, performs operations comprising: transforming an input image into a latent code using an encoder of an autoencoder by performing a multi-level discrete wavelet transform (DWT) on the input image; generating, based on the latent code, a reconstructed latent code using a latent diffusion model (LDM); and training the LDM using a denoising loss based on comparing the reconstructed latent code and the latent code. || 16. A system, comprising: a processor; and a memory having instructions stored thereon which, when executed on the processor, performs operations comprising: transforming an input image into a latent code using an encoder of an autoencoder by performing a multi-level discrete wavelet transform (DWT) on the input image; generating, based on the latent code, a reconstructed latent code using a latent diffusion model (LDM); and training the LDM using a reconstruction loss based on comparing the reconstructed latent code and the latent code.