VIDEO REMASTERING VIA DEEP LEARNING

Number: 20230267706
Published: 2023-08-24
Filed: 2023-02-21
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Djelouah; Abdelaziz et al.
CPC: G06T3/02; G06T3/40; G06T3/4046; G06T3/4092; G06T5/10; G06V10/24; G06V10/62; G06V10/771; G06V10/82; G06V20/46
Verdict: Set aside generic video remastering
Source: Google Patents · FreePatentsOnline

Abstract

One embodiment of the present invention sets forth a technique for performing remastering of video content. The technique includes determining a first input frame corresponding to a first frame included in a first video and a first target frame corresponding to a second frame included in a second video based on one or more alignments between the first frame and the second frame. The technique also includes executing a machine learning model to convert the first input frame into a first output frame. The technique further includes training the machine learning model based on one or more losses associated with the first output frame and the first target frame.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to video remastering and, more specifically, to video remastering via deep learning. Description of the Related Art

Video remastering is the process of changing or improving the quality of a video. For example, video remastering can involve adjusting the color, brightness, contrast, and saturation of a video; reducing noise and graininess in the video; improving the sharpness or resolution of the video; and/or repairing damaged or degraded video.

Video remastering techniques are commonly used to convert older “legacy” content that is captured on film into high-resolution digital formats that are suitable for streaming or playback on a laptop computer, smart television, tablet computer, or other type of electronic device. For example, a master copy of an episode of a television show could be stored on a film reel. The episode could be remastered into a digital copy by scanning the film reel using a modern scanning device. The resulting scanned version of the episode could have a higher resolution, higher visual quality, and better color range than an analog broadcast format, such as National Television System Committee (NTSC) or Phase Alternating Line (PAL), used to air the episode on broadcast television.

However, original film copies of some legacy video content can be missing, damaged, or otherwise unavailable. For example, a film master could be un

Claims

1. A computer-implemented method for performing remastering of video content, the computer-implemented method comprising: determining a first input frame corresponding to a first frame included in a first video and a first target frame corresponding to a second frame included in a second video based on one or more alignments between the first frame and the second frame; executing a machine learning model to convert the first input frame into a first output frame; and training the machine learning model based on one or more losses associated with the first output frame and the first target frame. || 11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: determining a first input frame corresponding to a first frame included in a first video and a first target frame corresponding to a second frame included in a second video based on one or more alignments between the first frame and the second frame; executing a machine learning model to convert the first input frame into a first output frame; and training the machine learning model based on one or more losses associated with the first output frame and the first target frame. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: determining a first input frame corresponding to a first frame included in a first video and a first target frame corresponding to a second frame included in a second video based a temporal alignment between the first frame and the second frame and a geometric alignment between the first frame and the second frame; executing a machine learning model to convert the first input frame into a first output frame; and training the machine learning model based on one or more losses associated with the first output frame and the first target frame. || 21. A computer-implemented method for performing remastering of video content, the method comprising: determining a first input frame corresponding to a first frame included in a first video, wherein the first input frame is associated with a first level of quality; executing a machine learning model to convert the first input frame into a first output frame, wherein the first output frame is associated with a second level of quality that is higher than the first level of quality, and wherein the machine learning model is trained using a set of input frames that is associated with the first level of quality and a set of target frames that is temporally aligned with the set of input frames and associated with the second level of quality; and generating a second video that includes the first output frame.