LOSSY IMAGE COMPRESSION WITH DIFFUSION MODELS

Number: 20250157087
Published: 2025-05-15
Filed: 2024-10-18
Assignee: Disney Enterprises, Inc.
Inventors: Relic; Lucas et al.
CPC: H04N19/124; G06T9/002; H04N19/86; G06T5/70; G06T5/60; H04N19/192; H04N19/117
Verdict: Set aside image compression, codec plumbing
Source: Google Patents · FreePatentsOnline

Abstract

In some embodiments, a method receives a quantized latent representation of an image in a latent space. The image is encoded into a representation in the latent space and quantized to generate the quantized latent representation. A time step parameter is received that is generated based on the representation. The method performs an inverse quantization process to generate a reconstructed representation. A diffusion model performs a denoising process for a number of iterations based on the time step parameter to remove noise from the reconstructed representation to generate a denoised reconstructed representation. The denoised reconstructed representation is decoded into a reconstructed image.

Background

BACKGROUND

Multimedia content is delivered through networks globally, and makes up a large portion of the traffic. The development of efficient compression algorithms is important to efficiently deliver the multimedia content throughout the networks.

Traditional encoder-decoders (CODECS), which use handcrafted transformations by users, may be outperformed by data-driven neural image compression (NIC) methods that optimize for both rate and distortion. Nevertheless, neural image compression methods may still produce blurry and unrealistic images, such as in low bitrate settings. This is because the methods may be optimized for rate distortion, where distortion is measured with pixel-wise metrics like mean squared error. The optimizing for low distortion, such as pixel-wise error, may result in unrealistic images. This may be because emphasizing pixel-wise accuracy or similarity to the original image may lead to overly smoothed or blurry outputs.

Claims

1. A method comprising: receiving a quantized latent representation of an image in a latent space, wherein the image is encoded into a latent representation in the latent space and quantized to generate the quantized latent representation; receiving a time step parameter that is generated based on the latent representation; performing an inverse quantization process to generate a reconstructed latent representation; performing, using a diffusion model, a denoising process for a number of iterations based on the time step parameter to remove noise from the reconstructed latent representation to generate a denoised reconstructed latent representation; and decoding the denoised reconstructed latent representation into a reconstructed image. || 12. A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for: receiving a quantized latent representation of an image in a latent space, wherein the image is encoded into a latent representation in the latent space and quantized to generate the quantized latent representation; receiving a time step parameter that is generated based on the latent representation; performing an inverse quantization process to generate a reconstructed latent representation; performing, using a diffusion model, a denoising process for a number of iterations based on the time step parameter to remove noise from the reconstructed latent representation to generate a denoised reconstructed latent representation; and decoding the denoised reconstructed latent representation into a reconstructed image. || 13. A method comprising: receiving an image; encoding the image into a latent representation in a latent space; estimating a time step parameter based on the latent representation; performing a quantization process on the latent representation to generate a quantized latent representation; and transmitting the quantized latent representation to a receiver, wherein an inverse quantization process is performed to generate a reconstructed latent representation and a diffusion model performs a denoising process for a number of iterations based on the time step parameter to remove noise from the reconstructed latent representation.