Deinterlacing via deep learning

Number: 11570397
Published: 2023-01-31
Filed: 2020-07-10
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Bernasconi; Michael et al.
CPC: G06N3/084; G06N3/09; G06N3/0464; H04N7/012; G06N3/045; G06N3/0495; G06T3/4046; H04N5/145
Verdict: Set aside generic video deinterlacing ML
Source: Google Patents · FreePatentsOnline

Abstract

One embodiment of the present invention sets forth a technique for performing deinterlacing. The technique includes separating a first interlaced video frame into a first sequence of fields ordered by time, the first sequence of fields including a first field. The technique also includes generating, by applying a deinterlacing network to a first field in the first sequence, a second field that is missing from the first sequence of fields and is complementary to the first field. The technique further includes constructing a progressive video frame based on the first field and the second field.

Background

BACKGROUND Field of the Various Embodiments (1) Embodiments of the present disclosure relate generally to deinterlacing of video frames, and more specifically, to deinterlacing of video frames via deep learning. Description of the Related Art (2) Interlaced video is commonly used to double the perceived frame rate of a video display without consuming extra bandwidth or, conversely, to maintain the same frame rate while reducing data transfer or increasing the spatial resolution of the video. Each frame of interlaced video contains two fields captured at consecutive points in time; one field contains odd-numbered scan lines in the frame, and the other field contains even-numbered scan lines in the frame. (3) While certain types of displays (e.g., cathode ray tube (CRT)) can display interlaced video without modification (e.g., by scanning each frame of interlaced video in two passes, one for each field), modern digital displays typically support only progressive scanning, where all the lines of a frame are drawn in sequence (e.g., from top to bottom). As a result, interlaced video that is shown on these digital displays may exhibit combing artifacts when objects in the video move quickly enough to be in different positions when consecutive fields are captured. (4) To mitigate combing effects or other artifacts associated with displaying interlaced video on a progressive scan display, a deinterlacing process is commonly used to convert the interlaced video into a progressive for

Claims

1. A method for performing deinterlacing, comprising: separating a first interlaced video frame into a first sequence of fields ordered by time, the first sequence of fields including a first field representing a first point in time and a second field representing a second point in time that is different from the first point in time; generating, by applying a machine learning model comprising a deinterlacing network that includes one or more residual links to the first field in the first sequence of fields, a third field that is missing from the first sequence of fields and is complementary to the first field; and constructing a progressive video frame based on the first field and the third field. || 13. A non-transitory computer readable medium storing instructions that, when executed by a processor, cause the processor to perform the steps of: separating a first interlaced video frame into a first sequence of fields ordered by time, the first sequence of fields including a first field representing a first point in time and a second field representing a second point in time that is different from the first point in time; adding, to the first sequence of fields, one or more fields from one or more frames adjacent to the first interlaced video frame within an interlaced video; generating, by applying a machine learning model that comprises a deinterlacing network to the first sequence of fields, a third field that is missing from the first sequence of fields and is complementary to the first field in a progressive video frame; and constructing a progressive video frame based on the first field and the third field. || 20. A system, comprising: a memory that stores instructions, and a processor that is coupled to the memory and, when executing the instructions, is configured to: separate a first interlaced video frame into a first sequence of fields ordered by time, the first sequence of fields including a first field representing a first point in time and a second field representing a second point in time that is different from the first point in time; generate, by applying a machine learning model comprising a deinterlacing network that includes one or more residual links to the first field in the first sequence of fields, a third field that is missing from the first sequence of fields and is complementary to the first field; and construct a progressive video frame based on the first field and the third field.