SYSTEM AND METHOD FOR SYNCHRONIZING LIVE VIDEO WITH EXTERNAL DATA SOURCES

Number: 20250088685
Published: 2025-03-13
Filed: 2024-11-27
Assignee: Disney Enterprises, Inc.
Inventors: Hyde; Andrew et al.
CPC: H04L65/75; H04N21/26258; H04N21/4305; H04N21/8456; H04N21/242
Verdict: Set aside broadcast synchronization, plumbing
Source: Google Patents · FreePatentsOnline

Abstract

Methods and systems for synchronizing video with external data sources by a user device where the video is grouped into segment video files having video frames, includes establishing a persistent connection between the user device and a synchronization server, receiving a sync message associated with the beginning of a video segment to be played by the user device, the sync message having an initial recording wall clock time stamp, calculating by the user device a recording wall clock time for each frame in the segment using the initial wall clock time and a frame rate, downloading the video segment file for playing on the user device, receiving by the user device, external data having a wall clock time stamp, and displaying on the user device the external data having the same time stamp as the recording time stamp of the video frame being displayed.

Background

BACKGROUND

It is common practice for computer-based servers to digitally stream media content (e.g., video, music, text, graphics, and the like), live or on-demand, to electronic devices, such as mobile smartphones, tablets and the like, for viewing by a user. Such media content is often provided to the user devices (or clients) via the internet using a distributed network of computer-based data file servers deployed in one or more data centers, also referred to as a content delivery network or content distribution network (collectively, CDN).

HTTP-based streaming methods, such as HLS (HTTP-based Live Media Streaming communication protocol), supported or implemented by products made by companies such as Apple, Inc., HDS (HTTP Dynamic Streaming), supported or implemented by products made by companies such as Adobe, Inc., and DASH (Dynamic Adaptive bit-rate Streaming over HTTP) or MPEG-DASH, are commonly used for streaming media content and operate using “HTTP-based streaming,” where media content is stored in “segments” or “chunks” of a predetermined length (e.g., 1 to 10 seconds) and a playlist (or manifest or index) file is provided that lists, or points to, each of the segments or chunks of the media stream for display or playback purposes by the user devices.

HLS typically works by generating a manifest file which is continually polled by the video player and contains the location (or address or URL) where the segments of video should be downloaded fro

Claims

1. A method for synchronizing streaming media with external data sources received by a user device, the media being grouped into segment media files having media frames, comprising: establishing a connection between the user device and a server; receiving by the user device from the server via the connection, a sync message associated with the beginning of a media segment having media frames to be played by the user device, the sync message having an initial recording wall clock time stamp corresponding to a first frame of the media segment; and calculating, by the user device, a recording wall clock time for each frame in the segment using the initial recording wall clock time stamp and a frame rate for the segment. || 23. A method for synchronizing media with external data sources on a user device, the media grouped into segment media files having media frames, comprising: establishing a connection between the user device and a server; receiving a sync message associated with the beginning of a media segment to be played by the user device, the sync message having at least an initial recording wall clock time stamp corresponding to a first frame of the media segment; determining by the user device, using the initial recording wall clock time stamp and a frame rate, a recording wall clock time for each media frame in the media segment; downloading the media segment file for playing on the user device; receiving by the user device, external data having an event wall clock time stamp; and synchronizing in realtime by the user device, the external data with the media segment file having the same time stamp. || 37. A method for synchronizing media with external data sources, comprising: receiving by a user device streaming encoded media content grouped into segment media files, each segment media file having a plurality of media frames and having an initial recording wall clock time stamp corresponding to a first frame of the media segment; downloading to the user device, via a segment delivery connection between the user device and a segment server, a media segment file from the segment server, for playing on the user device; extracting from the mdeia segment file the initial recording wall clock time stamp of the media segment; calculating, by the user device, a recording wall clock time for each frame in the media segment using the initial recording wall clock time stamp and a frame rate for the media segment; receiving by the user device external data having an event wall clock time, the external data for a given event wall clock time stamp being received by the user device before the media segment having a corresponding time stamp; and displaying on the user device, the external data having the same wall clock time stamp as the wall clock time stamp of the media frame being displayed, thereby providing realtime synchronized media and external data displayed on the user device.