Outer Rim Archives
Archives · 2026 · 20260331116

Application (pre-grant publication)

MULTIMODAL AUDIO CHANNEL CLASSIFICATION

Number
20260331116
Published
2026-10-22
Filed
2025-04-21
Assignee
Disney Enterprises, Inc.
Inventors
GAVIN; Patrick
CPC
G10L19/04; G06V20/46; G06V20/63; G06V30/10
Verdict
Set aside streaming, codec
In edition
2026-W43
Source
Google Patents · FreePatentsOnline

The keeper's note

Embodiments provide for multimodal audio channel classification.

Abstract

Embodiments provide for multimodal audio channel classification. Audio data comprising a set of channels is accessed, and a set of spectrograms comprising a respective spectrogram for each respective channel of the set of channels is generated. An initial set of channel classifications is generated based on, for each respective channel of the set of channels, processing the respective spectrogram using a channel classification machine learning model. A revised set of channel classifications is generated based on evaluating the initial set of channel classifications using a set of channel rules. The revised set of channel classifications is output.

Background

BACKGROUND

There are a wide variety of encoding and formatting configurations in common use today for media files (including audio files, video files, or multimedia files including both audio and video). For example, audio is often stored or provided as a set of channels, where each channel may have distinct portions of the desired audio output. However, there are many different common channel configurations. That is, the particular audio that is assigned to any particular channel may vary widely. For example, channel configurations may include mono (e.g., a single channel, or the same audio data on each channel), stereo (e.g., with a left and a right channel), 2.1 (e.g., stereo plus a subwoofer channel), 5.1 (e.g., five full-bandwidth channels and a subwoofer channel), and the like.

Often, video and audio files may be received from various sources (including international versions), and it may be unclear whether any the files follow the same specification requirements for audio formatting. However, a wide variety of downstream processes (e.g., encoding or re-encoding the file for storage, streaming, and the like) rely on dependable audio formatting. When the audio format of a given file is unknown or cannot be trusted, these downstream operations can encounter numerous errors. However, given the variety of possible configurations, there are no adequate techniques available to automatically perform channel classification.

Claims

1. A method, comprising: accessing audio data comprising a set of channels; generating a set of spectrograms comprising a respective spectrogram for each respective channel of the set of channels; generating an initial set of channel classifications based on, for each respective channel of the set of channels, processing the respective spectrogram using a channel classification machine learning model; generating a revised set of channel classifications based on evaluating the initial set of channel classifications using a set of channel rules; and outputting the revised set of channel classifications. || 13. One or more non-transitory computer readable media containing, in any combination, computer program code that, when executed by operation of any combination of one or more processors, performs an operation comprising: accessing audio data comprising a set of channels; generating a set of spectrograms comprising a respective spectrogram for each respective channel of the set of channels; generating an initial set of channel classifications based on, for each respective channel of the set of channels, processing the respective spectrogram using a channel classification machine learning model; generating a revised set of channel classifications based on evaluating the initial set of channel classifications using a set of channel rules; and outputting the revised set of channel classifications. || 17. A system, comprising: one or more processors; and one or more memories storing a program, which, when executed on any combination of the one or more processors, performs operations, the operations comprising: accessing audio data comprising a set of channels; generating a set of spectrograms comprising a respective spectrogram for each respective channel of the set of channels; generating an initial set of channel classifications based on, for each respective channel of the set of channels, processing the respective spectrogram using a channel classification machine learning model; generating a revised set of channel classifications based on evaluating the initial set of channel classifications using a set of channel rules; and outputting the revised set of channel classifications.