Sample-based video sharpening

Number: 10489891
Published: 2019-11-26
Filed: 2017-06-22
Assignee: Disney Enterprises, Inc.
Inventors: Schroers; Christopher et al.
CPC: G06T1/20; G06T5/20; G06T5/70; G06T5/73; H04N5/21; H04N5/213
Verdict: Set aside sample-based video sharpening, generic video enhancement
Source: Google Patents · FreePatentsOnline

Abstract

According to one implementation, a video processing system includes a computing platform having a hardware processor and a system memory storing a sample-based video sharpening software code. The sample-based video sharpening software code receives a video sequence, and classifies frames of the video sequence as sharp or unsharp. For each pixel of an unsharp frame, the sample-based video sharpening software code determines a mapping of the pixel to another pixel in some or all of the sharp frames, determines a reverse mapping of each of the other pixels to the pixel, identifies a first confidence value corresponding to each of the other pixels based on the mapping, identifies a second confidence value corresponding to each of the other pixels based on the mapping and the reverse mapping, and sharpens the pixel based on a weighted combination of the other pixels determined using the first and second confidence values.

Background

BACKGROUND(1) Due to the popularity of video as an entertainment medium, ever more video content, including high definition (HD) and Ultra HD video content is being produced and made available to consumers. One fundamental challenge encountered in video processing is video sharpening, which is the de-blurring of video images. An important objective of video sharpening is to de-blur video images without destroying small scale features of those images.(2) One technique for video sharpening is performed in three-dimensional (3D) “scene space,” in which video pixels are processed according to their 3D positions. Scene space based video sharpening relies on depth reconstruction, which limits that approach to static scenes with significant camera motion. However, in practice, most video sequences feature many dynamic moving objects, and often little or no camera motion, making depth reconstruction impossible.SUMMARY(3) There are provided sample-based video sharpening systems and methods, substantially as shown in and/or described in connection with at least one of the figures, and as set forth more completely in the claims.

Claims

1. A video processing system comprising: a computing platform including a hardware processor and a system memory; a sample-based video sharpening software code stored in the system memory; the hardware processor configured to execute the sample-based video sharpening software code to: receive a video sequence; classify a plurality of frames of the video sequence as sharp frames and at least one frame of the video sequence as an unsharp frame; for each pixel of the unsharp frame: determine a mapping of the pixel to an other pixel in each of the sharp frames; determine a reverse mapping of the other pixel in each of the sharp frames to the pixel in the unsharp frame; identify a first confidence value corresponding to the other pixel in each of the sharp frames, based on the mapping; identify a second confidence value corresponding to the other pixel in each of the sharp frames, based on the mapping and the reverse mapping; and sharpen the pixel as a combination of the pixel and the other pixel in each of the sharp frames, wherein the other pixel in each of the sharp frames is weighted using the first confidence value and the second confidence value. 2. The video processing system of claim 1, wherein for each pixel of the unsharp frame, the hardware processor is further configured to execute the sample-based video sharpening software code to: identify a third confidence value corresponding to the other pixel in each of the sharp frames, based on a sharpness of the other pixel; and sharpen the pixel as a combination of the pixel and the other pixel in each of the sharp frames, wherein the other pixel in each of the sharp frames is weighted using the first confidence value, the second confidence value, and the third confidence value. 3. The video processing system of claim 1, wherein the pixel has a first location in the unsharp frame, and wherein the other pixel in each of the sharp frames has a respective second location in the sharp frames. 4. The video processing system of claim 1, wherein the first confidence value corresponding to the other pixel in each of the sharp frames is based on a color match between the pixel of the unsharp frame and the other pixel. 5. The video processing system of claim 1, wherein the first confidence value is proportional to a color match between the pixel of the unsharp frame and the other pixel. 6. The video processing system of claim 1, wherein the second confidence value corresponding to the other pixel in each of the sharp frames is based on a comparison of the reverse mapping to the mapping. 7. The video processing system of claim 1, wherein the second confidence value corresponding to the other pixel in each of the sharp frames is proportional to an extent to which the reverse mapping is inverse to the mapping. 8. A method for use by a video processing system including a computing platform having a hardware processor and a system memory storing a sample-based video sharpening software code, the method comprising: receiving, using the hardware processor, a video sequence; classifying, using the hardware processor, a first plurality of frames of the video sequence as sharp frames and at least one frame of the video sequence as an unsharp frame; for each pixel of the unsharp frame: determining, using the hardware processor, a mapping of the pixel to an other pixel in each of the sharp frames; determining, using the hardware processor, a reverse mapping of the other pixel in each of the sharp frames to the pixel in the unsharp frame; identifying, using the hardware processor, a first confidence value corresponding to the other pixel in each of the sharp frames, based on the mapping; identifying, using the hardware processor, a second confidence value corresponding to the other pixel in each of the sharp frames, based on the mapping and the reverse mapping; and sharpening, using the hardware processor, the pixel as a combination of the pixel and the other pixel in each of the sharp frames, wherein the other pixel in each of the sharp frames is weighted using the first confidence value and the second confidence value. 9. The method of claim 8, further comprising, for each pixel of the unsharp frame: identifying, using the hardware processor, a third confidence value corresponding to the other pixel in each of the sharp frames, based on a sharpness of the other pixel; and sharpening, using the hardware processor, the pixel as a combination of the pixel and the other pixel in each of the sharp frames, wherein the other pixel in each of the sharp frames is weighted using the first confidence value, the second confidence value, and the third confidence value. 10. The method of claim 8, wherein the pixel has a first location in the unsharp frame, and wherein the other pixel in each of the sharp frames has a respective second location in the sharp frames. 11. The method of claim 8, wherein the first confidence value corresponding to the other pixel in each of the sharp frames is based on a color match between the pixel of the unsharp frame and the other pixel. 12. The method of claim 8, wherein the first confidence value is proportional to a color match between the pixel of the unsharp frame and the other pixel. 13. The method of claim 8, wherein the second confidence value corresponding to the other pixel in each of the sharp frames is based on a comparison of the reverse mapping to the mapping. 14. The method of claim 8, wherein the second confidence value corresponding to the other pixel in each of the sharp frames is proportional to an extent to which the reverse mapping is inverse to the mapping. 15. A computer-readable non-transitory medium having stored thereon instructions, which when executed by a hardware processor, instantiate a method comprising: receiving a video sequence; classifying a plurality of frames of the video sequence as sharp frames and at least one frame of the video sequence as an unsharp frame; for each pixel of the unsharp frame: determining a mapping of the pixel to an other pixel in each of the sharp frames; determining a reverse mapping of the other pixel in each of the sharp frames to the pixel in the unsharp frame; identifying a first confidence value corresponding to the other pixel in each of the sharp frames, based on the mapping; identifying a second confidence value corresponding to the other pixel in each of the sharp frames, based on the mapping and the reverse mapping; and sharpening the pixel as a combination of the pixel and the other pixel in each of the sharp frames, wherein the other pixel in each of the sharp frames is weighted using the first confidence value and the second confidence value. 16. The computer-readable non-transitory medium of claim 15, wherein the method further comprises, for each pixel of the unsharp frame: identifying a third confidence value corresponding to the other pixel in each of the sharp frames, based on a sharpness of the other pixel; and sharpening the pixel as a combination of the pixel and the other pixel in each of the sharp frames, wherein the other pixel in each of the sharp frames is weighted using the first confidence value, the second confidence value, and the third confidence value. 17. The computer-readable non-transitory medium of claim 15, wherein the pixel has a first location in the unsharp frame, and wherein the other pixel in each of the sharp frames has a respective second location in the sharp frames. 18. The computer-readable non-transitory medium of claim 15, wherein the first confidence value corresponding to the other pixel in each of the sharp frames is based on a color match between the pixel of the unsharp frame and the other pixel. 19. The computer-readable non-transitory medium of claim 15, wherein the first confidence value is proportional to a color match between the pixel of the unsharp frame and the other pixel. 20. The computer-readable non-transitory medium of claim 15, wherein the second confidence value corresponding to the other pixel in each of the sharp frames is based on a comparison of the reverse mapping to the mapping.