Outer Rim Archives
Archives · 2025 · 12470720

Granted patent

Content adaptive boundary placement for distributed encodes

Number
12470720
Published
2025-11-11
Filed
2020-03-30
Assignee
Disney Enterprises, Inc.
Inventors
Xue; Yuanyi et al.
CPC
H04N19/115; H04N19/119; H04N19/14; H04N19/177; H04N19/40; H04N19/436
Verdict
Set aside distributed video encoding, codec plumbing
Source
Google Patents · FreePatentsOnline

Abstract

Chunk based video encoding involves dividing a video into chunks with varying lengths based on the content within those frames. In contrast, dividing the video at a fix interval is prone to generating chunks starting at the middle of hard to encode areas, which can lead to a loss of encoder rate-control efficiency and produce visual quality gaps at the beginning of such chunks. The system can identify a set of boundaries for dividing the video into chunks having similar lengths and with little to no impact on visual quality. The boundaries of the chunks are placed at locations (or frames) that are far from the complex (or hard to encode) areas of the video. To do so, the system evaluates the video using various complexity metrics to identify the complex areas that require more bits to encode relative to less complex areas.

Background

BACKGROUND (1) A video stream includes a series of video frames. A set of frames may be referred to as a segment. Segments are generally of equal length, but there may be exceptions. Segments denote, for example, places where the video can be interrupted (e.g., to insert a commercial). (2) Chunk-based video encoding involves dividing the frames in a video into groups called “chunks” that may be encoded (i.e., compressed) by a single transcoder, or separately by multiple transcoders (i.e. “separate nodes”) in parallel. A basic approach is to divide a video into chunks having equal length and then encode the chunks in parallel using different transcoders.

Claims

1. A method of content-adaptive boundary placement for distributed encoding, the method comprising: evaluating frames in a received video to identify a complexity metric, wherein the complexity metric represents the complexity of encoding the frames in the received video, and wherein the complexity metric is derived using a machine learning model, the complexity metric comprising at least two of a luminance derived from the frames, a difference derived from the frames, or a statistic derived from the frames, each of the at least two having a different weighting, the complexity metric including the statistic, the statistic including three or more of a quantization statistic, a bitrate statistic, a buffer statistic, an encoding statistic, or a frame cost statistic, wherein: the quantization statistic comprises a quantization parameter fluctuation, the bitrate statistic comprises a bitrate factor, the buffer statistic comprises a buffer state, the encoding statistic comprises a macroblock encode time, and the frame cost statistic comprises a frame cost ratio; generating, based on the complexity metric, a complexity heat map specifying one or more hot spots in the received video, each hot spot comprising at least one frame predicted by the complexity metric to satisfy a complexity criterion comprising a count of bits to encode the at least one frame, wherein each of the three or more statistics is used in generating the complexity heat map; increasing bitrate-control efficiency in encoding a plurality of chunks, by reducing incidence of the one or more hot spots being encoded across chunk boundaries, wherein the incidence is reduced by permitting chunk lengths of the plurality of chunks to vary when the plurality of chunks is formed; sacrificing, in favor of increased parallelism when multiple encoders encode the chunks in parallel, a further increase in the bitrate-control efficiency, by restricting the chunk lengths from varying beyond a tolerance, wherein the plurality of chunks is formed by identifying boundaries in the received video based on the complexity heat map, a desired chunk length, and the tolerance, comprising, for each frame in a range of frames identified in the received video using the desired chunk length and the tolerance, identifying a minimum distance to a closest frame of a neighboring hot spot in a backward or forward direction in a timeline of the received video, wherein an end boundary of a first chunk is set at the frame in the range of frames that has the largest minimum distance; encoding the plurality of chunks in parallel by a plurality of encoders of the multiple encoders and with a higher measure of parallelism than absent the varying being restricted, the chunk lengths varying within the tolerance, the encoding comprising distributed encoding, wherein each of the plurality of encoders is configured to perform an analysis pass and an encoding pass as part of a multi-pass encoder pipeline; and combining the encoded chunks to form an encoded video. || 6. A non-transitory computer readable medium containing computer program code that, when executed by operation of one or more computer processors, performs an operation for content-adaptive boundary placement for distributed encoding, the operation comprising: evaluating frames in a received video to identify a complexity metric, wherein the complexity metric represents the complexity of encoding the frames in the received video, and wherein the complexity metric is derived using a machine learning model, the complexity metric comprising at least three of a luminance derived from the frames, a difference derived from the frames, or a statistic derived from the frames, each of the at least three having a different weighting, the complexity metric including the statistic, the statistic including two or more of a quantization statistic, a bitrate statistic, a buffer statistic, an encoding statistic, or a frame cost statistic, wherein: the quantization statistic comprises a quantization parameter fluctuation, the bitrate statistic comprises a bitrate factor the buffer statistic comprises a buffer state, the encoding statistic comprises a macroblock encode time, and the frame cost statistic comprises a frame cost ratio; generating, based on the complexity metric, a complexity heat map specifying one or more hot spots in the received video, each hot spot comprising at least one frame predicted by the complexity metric to satisfy a complexity criterion comprising a count of bits to encode the at least one frame, wherein each of the two or more statistics is used in generating the complexity heat map; increasing bitrate-control efficiency in encoding a plurality of chunks, by reducing incidence of the one or more hot spots being encoded across chunk boundaries, wherein the incidence is reduced by permitting chunk lengths of the plurality of chunks to vary when the plurality of chunks is formed; sacrificing, in favor of increased parallelism when multiple encoders encode the chunks in parallel, a further increase in the bitrate-control efficiency, by restricting the chunk lengths from varying beyond a tolerance, wherein the plurality of chunks is formed by identifying boundaries in the received video based on the complexity heat map, a desired chunk length, and the tolerance, comprising:, for each frame in a range of frames identified in the received video using the desired chunk length and the tolerance, identifying a minimum distance to a closest frame of a neighboring hot spot in a backward or forward direction in a timeline of the received video, wherein an end boundary of a first chunk is set at the frame in the range of frames that has the largest minimum distance; encoding the plurality of chunks in parallel by the multiple encoders and with a higher measure of parallelism than absent the varying being restricted, the chunk lengths varying within the tolerance, the encoding comprising distributed encoding, wherein each of the plurality of encoders is configured to perform an analysis pass and an encoding pass as part of a multi-pass encoder pipeline; and combining the encoded chunks to form an encoded video. || 8. A system for content-adaptive boundary placement for distributed encoding, the system comprising: a processor; and memory configured to store an application, wherein, when executed by the processor, the application performs an operation comprising: evaluating frames in a received video to identify a complexity metric, wherein the complexity metric represents the complexity of encoding the frames in the received video, and wherein the complexity metric is derived using a machine learning model, the complexity metric comprising at least three of a luminance derived from the frames, a difference derived from the frames, or a statistic derived from the frames, each of the at least two having a different weighting, the complexity metric including the statistic, the statistic including three or more of a quantization statistic, a bitrate statistic, a buffer statistic, an encoding statistic, or a frame cost statistic, wherein: the quantization statistic comprises a quantization parameter fluctuation, the bitrate statistic comprises a bitrate factor the buffer statistic comprises a buffer state, the encoding statistic comprises a macroblock encode time, and the frame cost statistic comprises a frame cost ratio; generating, based on the complexity metric, a complexity heat map specifying one or more hot spots in the received video, each hot spot comprising at least one frame predicted by the complexity metric to satisfy a complexity criterion comprising a count of bits to encode the at least one frame, wherein each of the two or more statistics is used in generating the complexity heat map; increasing bitrate-control efficiency in encoding a plurality of chunks, by reducing incidence of the one or more hot spots being encoded across chunk boundaries, wherein the incidence is reduced by permitting chunk lengths of the plurality of chunks to vary when the plurality of chunks is formed; sacrificing, in favor of increased parallelism when multiple encoders encode the chunks in parallel, a further increase in the bitrate-control efficiency, by restricting the chunk lengths from varying beyond a tolerance, wherein the plurality of chunks is formed by identifying boundaries in the received video based on the complexity heat map, a desired chunk length, and the tolerance, comprising, for each frame in a range of frames identified in the received video using the desired chunk length and the tolerance, identifying a minimum distance to a closest frame of a neighboring hot spot in a backward or forward direction in a timeline of the received video, wherein an end boundary of a first chunk is set at the frame in the range of frames that has the largest minimum distance; encoding the plurality of chunks in parallel by a plurality of encoders of the multiple encoders and with a higher measure of parallelism than absent the varying being restricted, the chunk lengths varying within the tolerance, the encoding comprising distributed encoding, wherein each of the plurality of encoders is configured to perform an analysis pass and an encoding pass as part of a multi-pass encoder pipeline; and combining the encoded chunks to form an encoded video.