Video super-resolution using an artificial neural network

Number: 10621695
Published: 2020-04-14
Filed: 2018-02-01
Assignee: Disney Enterprises, Inc.
Inventors: Schroers; Christopher, Wang; Yifan, Perazzi; Federico, McWilliams; Brian, Hornung; Alexander Sorkine
CPC: G06T3/4007; G06T3/4046; G06T3/4053
Verdict: Set aside generic video super-resolution, no creative hook
Source: Google Patents · FreePatentsOnline

Abstract

According to one implementation, a video processing system includes a computing platform having a hardware processor and a system memory storing a software code including an artificial neural network (ANN). The hardware processor is configured to execute the software code to receive a first video sequence having a first display resolution, and to produce a second video sequence based on the first video sequence using the ANN. The second video sequence has a second display resolution higher than the first display resolution. The ANN is configured to provide sequential frames of the second video sequence that are temporally stable and consistent in color to reduce visual flicker and color shifting in the second video sequence.

Background

BACKGROUND(1) Spatial up-sampling of discretely sampled visual data, often referred to as super-resolution, has applications that are in considerable demand at present. For example, super-resolution may be desirable for use in converting high-definition (HD) video content, e.g., 1K or 2K resolution video, for viewing on the increasingly popular and commercially available Ultra HD 4K video displays, as well as the next generation of 8K video displays.(2) Conventional methods for performing super-resolution typically rely on redundancy and explicit motion estimation between video frames to effectively reconstruct a higher resolution signal from many lower resolution measurements. Although such conventional approaches can in principle result in a correct reconstruction of missing detail, their reliance on the quality of estimated motion between frames limits their ability to up-sample unconstrained real-world video with rapid motion, blur, occlusions, drastic appearance changes, and/or presenting other common video processing challenges.SUMMARY(3) There are provided systems and methods for performing video super-resolution using an artificial neural network, substantially as shown in arid/or described in connection with at least one of the figures, and as set forth more completely in the claims.

Claims

1. A video processing system comprising: a computing platform including a hardware processor and a system memory; a software code including an artificial neural network (ANN) stored in the system memory; the hardware processor configured to execute the software code to: receive a first video sequence having a first display resolution; produce a second video sequence based on the first video sequence using the ANN, the second video sequence having a second display resolution higher than the first display resolution; wherein the ANN is configured to provide sequential frames of the second video sequence that are temporally stable and consistent in color to reduce visual flicker and color shifting in the second video sequence. 9. A method for use by a video processing system including a computing platform having a hardware processor and a system memory storing a software code including an artificial neural network (ANN), the method comprising: receiving, using the hardware processor, a first video sequence having a first display resolution; producing, using the hardware processor and the ANN, a second video to sequence based on the first video sequence, the second video sequence having a second display resolution higher than the first display resolution; wherein the ANN is configured to provide sequential frames of the second video sequence that are temporally stable and consistent in color to reduce visual flicker and color shifting in the second video sequence. 17. A computer-readable non-transitory medium having stored thereon instructions and an artificial neural network (ANN), which when executed by a hardware processor, instantiate a method comprising: receiving a first video sequence having a first display resolution; producing a second video sequence based on the first video sequence using the ANN, the second video sequence having a second display resolution higher than the first display resolution; wherein the ANN is configured to provide sequential frames of the second video sequence that are temporally stable and consistent in color to reduce visual flicker and color shifting in the second video sequence.