Microdosing for low bitrate video compression

Number: 12010335
Published: 2024-06-11
Filed: 2022-03-25
Assignee: Disney Enterprises, Inc.
Inventors: Djelouah; Abdelaziz et al.
CPC: G06N3/045; G06N3/0455; G06N3/0464; G06N3/047; G06N3/0475; G06N3/0495; G06N3/088; G06N3/09; G06N3/094; G06N3/096; H04N19/42; H04N19/44
Verdict: Set aside video compression codec, plumbing
Source: Google Patents · FreePatentsOnline

Abstract

A system includes a machine learning (ML) model-based video encoder configured to receive an uncompressed video sequence including multiple video frames, determine, from among the multiple video frames, a first video frame subset and a second video frame subset, encode the first video frame subset to produce a first compressed video frame subset, and identify a first decompression data for the first compressed video frame subset. The ML model-based video encoder is further configured to encode the second video frame subset to produce a second compressed video frame subset, and identify a second decompression data for the second compressed video frame subset. The first decompression data is specific to decoding the first compressed video frame subset but not the second compressed video frame subset, and the second decompression data is specific to decoding the second compressed video frame subset but not the first compressed video frame subset.

Background

BACKGROUND (1) Video content represents the majority of total Internet traffic and is expected to increase even more as spatial resolution frame rate, and color depth of videos increase and more users adopt earning services. Although existing codecs have achieved impressive performance, they have been engineered to the point where adding further small improvements is unlikely to meet future demands. Consequently, exploring fundamentally different ways to perform video coding may advantageously lead to a new class of video codecs with improved performance and flexibility. (2) For example, one advantage of using a trained machine learning (ML) model, such as a neural network (NN), in the form of a generative adversarial network (GAN) for example, to perform video compression is that it enables the ML model to infer visual details that it would otherwise be costly in terms of data transmission, to obtain. However, the model size remains an important issue in current state-of-the-art proposals and existing solutions require significant computation effort on the decoding side. That is to say, one significant drawback of existing GAN-based compression frameworks is that they typically require large decoder models that are sometimes trained on private datasets. Therefore, retraining these models to their original performance is not generally possible, and even when the training data is available, retraining the model would be complicated and time consuming. Moreover, the memory requ

Claims

1. A system comprising: a machine learning (ML) model-based video encoder; and an ML model-based video decoder comprising a degradation-aware block based Micro-Residual-Network (MicroRN) defined by a number of hidden channels and a number of degradation-aware blocks of the MicroRN, the MicroRN configured to decode a first compressed video frame subset using a first decompression data, and decode a second compressed video frame subset using a second decompression data, without utilizing a residual network of a generative adversarial network (GAN) trained decoder; the ML model-based video encoder configured to: receive an uncompressed video sequence including a plurality of video frames; determine, from among the plurality of video frames, a first video frame subset and a second video frame subset; encode the first video frame subset to produce the first compressed video frame subset; identify the first decompression data for the first compressed video frame subset; encode the second video frame subset to produce the second compressed video frame subset; and identify the second decompression data for the second compressed video frame subset. || 9. A method for use by a system including a machine learning (ML) model-based video encoder and an ML model-based video decoder comprising a degradation-aware block based Micro-Residual-Network (MicroRN) defined by a number of hidden channels and a number of degradation-aware blocks of the MicroRN, the MicroRN configured to decode a first compressed video frame subset using a first decompression data, and decode a second compressed video frame subset using a second decompression data, without utilizing a residual network of a generative adversarial network (GAN) trained decoder, the method comprising: receiving, by the ML model-based video encoder, an uncompressed video sequence including a plurality of video frames; determining, by the ML model-based video encoder from among the plurality of video frames, a first video frame subset and a second video frame subset; encoding, by the ML model-based video encoder, the first video frame subset to produce the first compressed video frame subset; identifying, by the ML model-based video encoder, the first decompression data for the first compressed video frame subset; encoding, by the ML model-based video encoder, the second video frame subset to produce the second compressed video frame subset; and identifying, by the ML model-based video encoder, the second decompression data for the second compressed video frame subset.