SYSTEMS AND METHODS FOR GENERATING A LATENT SPACE RESIDUAL

Number: 20210067808
Published: 2021-03-04
Filed: 2019-08-30
Assignee: Disney Enterprises, Inc.
Inventors: Schroers; Christopher, Campos; Joaquim, Djelouah; Abdelaziz, Xue; Yuanyi, Doggett; Erika Varis, McPhillen; Jared, Labrozzi; Scott
CPC: G06V10/82; G06N3/045; H04N19/50; H04N19/12; H04N19/91; G06N3/08; G06N3/0464; G06N3/0455; G06N3/088; H04N19/124; G06V20/46; G06V20/49; G06V10/454; G06N3/0495; H04N19/90
Verdict: Set aside generic video compression ML, codec plumbing
Source: Google Patents · FreePatentsOnline

Abstract

Systems and methods are disclosed for generating a latent space residual. A computer-implemented method may use a computer system that includes non-transient electronic storage, a graphical user interface, and one or more physical computer processors. The computer-implemented method may include: obtaining a target frame, obtaining a reconstructed frame, encoding the target frame into a latent space to generate a latent space target frame, encoding the reconstructed frame into the latent space to generate a latent space reconstructed frame, and generating a latent space residual based on the latent space target frame and the latent space reconstructed frame.

Background

TECHNICAL FIELD

The present disclosure relates generally to encoders and decoders. BRIEF SUMMARY OF THE DISCLOSURE

Embodiments of the disclosure are directed to systems and methods for generating a latent space residual. In one embodiment, a computer-implemented method may include obtaining, from a non-transient electronic storage, a target frame. The computer-implemented method may also include obtaining, from the non-transient electronic storage, a reconstructed frame. The reconstructed frame may be based on surrounding reference frames. The computer-implemented method may further include encoding, with a physical computer processor, the target frame into a latent space to generate a latent space target frame. The computer-implemented method may also include encoding, with the physical computer processor, the reconstructed frame into the latent space to generate a latent space reconstructed frame. The computer-implemented method may further include generating, with the physical computer processor, a latent space residual based on the latent space target frame and the latent space reconstructed frame.

In embodiments, the computer-implemented method may include decoding, with the physical computer processor, the latent space residual and the latent space reconstructed frame to generate a decoded target frame.

In embodiments, the reconstructed frames may be generated by a number of steps. One step may include obtaining, from the non-transient electr

Claims

1. A computer-implemented method comprising: obtaining, from a non-transient electronic storage, a target frame; obtaining, from the non-transient electronic storage, a reconstructed frame, wherein the reconstructed frame is based on surrounding reference frames; encoding, with a physical computer processor, the target frame into a latent space to generate a latent space target frame; encoding, with the physical computer processor, the reconstructed frame into the latent space to generate a latent space reconstructed frame; and generating, with the physical computer processor, a latent space residual based on the latent space target frame and the latent space reconstructed frame. || 7. A computer-implemented method comprising: obtaining, from a non-transient electronic storage, a target frame; obtaining, from the non-transient electronic storage, one or more reference frames surrounding the target frame; obtaining, from the non-transient electronic storage, an encoder and a decoder; applying, with the physical computer processor, the one or more reference frames to the decoder to generate one or more decoded reference frames; predicting, with the physical computer processor, a reconstructed frame corresponding to the target frame based on the one or more decoded reference frames, applying, with the physical computer processor, the target frame to the encoder to generate a latent space target frame; applying, with the physical computer processor, the reconstructed frame to the encoder to generate a latent space reconstructed frame; and generating, with the physical computer processor, a latent space residual based on the latent space target frame and the latent space reconstructed frame. || 14. A system for generating a latent space residual, the system comprising: non-transient electronic storage; a physical computer processor configured by machine-readable instructions to: obtain a target frame; obtain a reconstructed frame, wherein the reconstructed frame is based on surrounding reference frames; encode the target frame into a latent space to generate a latent space target frame; encode the reconstructed frame into the latent space to generate a latent space reconstructed frame; and generate a latent space residual based on the latent space target frame and the latent space reconstructed frame.