OFFLINE REINFORCEMENT LEARNING FOR VIDEO ENCODER RATE CONTROL

Number: 20260292200
Published: 2026-09-24
Filed: 2026-02-25
Assignee: Disney Enterprises, Inc.
Inventors: Cosier; Lucas-Claudiu, Serifi; Agon, De Albuquerque Azevedo; Roberto Gerson, Schroers; Christopher Richard
CPC: H04N19/149; G06N3/0455; G06N3/08; H04N19/172
Verdict: Set aside streaming, codec, cdn/infra
In edition: 2026-W39
Source: Google Patents · FreePatentsOnline

The keeper's note

Abstract

A system and method for video encoder rate control that leverages offline-trained sequence modeling neural networks to dynamically select quantization parameters for video encoding, ensuring adherence to target bitrate constraints while optimizing video quality. Training data comprising sequences of encoder states, quantization parameter decisions, and bitrate usage is augmented using sparse Gaussian Processes and Fourier transform-based techniques to create diverse encoding trajectories. The neural network, preferably a transformer-based architecture, is trained to predict optimal quantization parameter sequences, incorporating a trajectory stitching mechanism for enhanced policy generalization. During deployment, the trained model adaptively predicts quantization parameters for each frame or coding unit, enabling efficient video compression and streaming across diverse content types and network conditions. Embodiments provide robust, practical solutions for media processing pipelines, content delivery networks, and streaming platforms, and methods for both training and deploying the model are disclosed.

Background

FIELD OF THE INVENTION

The present invention relates generally to the field of digital video processing and compression, and more specifically to methods and systems for rate control in video encoders. In particular, the invention concerns the application of offline reinforcement learning and sequence modeling techniques, such as transformer-based neural networks, to dynamically adjust quantization parameters in video encoding processes in order to achieve specified bitrate constraints while optimizing perceived video quality. BACKGROUND OF THE INVENTION

Video encoders need to dynamically adjust the amount of quantization (lossy compression) on each frame or coding unit to achieve a specified target bitrate. Such a process, named rate control, is a complex problem given the different network/user requirements and the content diversity an encoder needs to support.

Traditional approaches for rate control rely on hand-crafted heuristics that often fail to adapt to diverse content types and dynamic scene changes, leading to sub-optimal compression efficiency and bitrate overshooting. Current deep-learning solutions require complex online training or explicit constraint optimization, making them impractical for some real-world deployment.

While modern video codecs employ rate-distortion optimization through Lagrangian methods, these approaches often struggle to adapt to diverse content types, such as High Dynamic Range (HDR) content, and dynamic scene c

Claims

1. A method for controlling quantization parameter selection in video encoding, the method comprising: collecting training data comprising a plurality of trajectories, wherein each trajectory comprises a sequence of encoder states, quantization parameter (QP) decisions, returns-to-go indicative of future video quality, and costs-to-go indicative of a future bitrate budget; training, in an offline and return-conditioned manner, a sequence modeling neural network using the training data to predict a sequence of QP decisions that satisfies a target bitrate constraint while maximizing video quality; and deploying the trained sequence modeling neural network to encode a video sequence by, for each of one or more frames of the video sequence: inputting a sequence of prior encoder states, QP decisions, returns-to-go, and costs-to-go into the trained sequence modeling neural network; obtaining, from the trained sequence modeling neural network, a predicted QP decision for a current frame; and encoding the current frame using the predicted QP decision. || 8. A method for training a neural network for video encoder rate control, the method comprising: collecting training data comprising a plurality of trajectories, wherein each trajectory comprises a sequence of encoder states, quantization parameter (QP) decisions, returns-to-go indicative of future video quality, and costs-to-go indicative of a future bitrate budget; augmenting the training data by perturbing QP decision sequences within the plurality of trajectories to generate additional, valid trajectories; and training, in an offline and return-conditioned manner, a sequence modeling neural network using the augmented training data to predict a sequence of QP decisions that optimizes a trade-off between video quality and bitrate usage based on the returns-to-go and costs-to-go. || 14. A method for controlling quantization parameter selection in video encoding using a trained sequence modeling neural network, the method comprising: receiving a video sequence for encoding; and for each of one or more frames or coding units of the video sequence: constructing an input sequence for the trained sequence modeling neural network, the input sequence comprising a history of prior encoder states, QP decisions, target returns-to-go indicative of desired future video quality, and target costs-to-go indicative of a desired future bitrate budget; applying a trajectory composition mechanism to dynamically adjust a history length of the input sequence based on learned in-support return and cost values to generate an adjusted input sequence; obtaining, from the trained sequence modeling neural network, a predicted QP decision for a current frame or coding unit based on the adjusted input sequence; and encoding the current frame or coding unit using the predicted QP decision.