TECHNIQUES FOR UPSCALING IMAGES GENERATED WITH UNDETERMINED DOWNSCALING KERNELS

Number: 20210049739
Published: 2021-02-18
Filed: 2019-08-15
Assignee: DISNEY ENTERPRISES, INC.
Inventors: SCHROERS; Christopher, WANG; Yifan, CORNILLERE; Victor, SORKINE-HORNUNG; Olga, DJELOUAH; Abdelaziz
CPC: G06T3/4046
Verdict: Set aside generic image upscaling technique
Source: Google Patents · FreePatentsOnline

Abstract

A scaling application estimates a downscaling kernel used to generate a downscaled image. The scaling application upscales the downscaled image based on the estimated downscaling kernel, thereby generating a higher resolution version of the downscaled image with minimal visual artifacts. The scaling application includes various networks that perform the above operations. A kernel mapping network generates a degradation map based on the estimated downscaling kernel. A degradation-aware generator network generates a reconstructed image based on the downscaled image and the degradation map. A kernel discriminator network generates an image delta that reflects visual artifacts present in the reconstructed image. The scaling application includes a parameter optimizer that iteratively modifies the estimated downscaling kernel to reduce visual artifacts indicated in the image delta. Via one or more iterations, the scaling application generates a reasonably accurate estimation of the downscaling kernel and can then upscale the downscaled image with reduced visual artifacts.

Background

BACKGROUND Field of the Various Embodiments

Various embodiments relate generally to video processing and, more specifically, to techniques for upscaling images generated with undetermined downscaling kernels. Description of the Related Art

A typical media production pipeline includes one or more media processing stages where the resolution of the media content being processed is increased and/or decreased for various reasons. For example, the media content being processed could have been captured initially at a relatively higher resolution, such as an 8K resolution. This higher-resolution version of the media content could then be downscaled, using a known downscaling kernel, to a relatively lower resolution, such as an HD (high-definition) resolution, that is more suitable for downstream production processing operations. In the downstream portion of the media production pipeline, this lower-resolution version of the media content could be composited with additional rendered content to generate a lower-resolution composite version of the media content. Subsequently, the lower-resolution composite version of the media content could be upscaled, using the known downscaling kernel, to a relatively higher resolution that is more suitable for viewing, such as a 4K resolution.

One drawback of the above approach is that media content usually can be downscaled to a relatively lower resolution and then upscaled to a relatively high resolution without introducing v

Claims

1. A computer-implemented method for scaling image content, the method comprising: generating a first reconstructed image based on a scaled image and a scaling kernel, wherein the first reconstructed image has a first resolution, and the scaled image has a second resolution; generating an image difference based on the scaled image and the scaling kernel, wherein the image difference indicates that at least one visual artifact resides within the first reconstructed image; modifying the scaling kernel to produce a modified scaling kernel; and generating a second reconstructed image based on the scaled image and the modified scaling kernel, wherein the at least one visual artifact has been reduced within the second reconstructed image or eliminated. || 11. A non-transitory computer-readable medium storing program instructions that, when executed by a processor, cause the processor to scale image content by performing the steps of: generating a first reconstructed image based on a scaled image and a scaling kernel, wherein the first reconstructed image has a first resolution, and the scaled image has a second resolution; generating an image difference based on the scaled image and the scaling kernel, wherein the image difference indicates that at least one visual artifact resides within the first reconstructed image; and generating a second reconstructed image based on the scaled image and a modified scaling kernel that is generated via an optimization process, wherein the at least one visual artifact has been reduced within the second reconstructed image or eliminated. || 20. A system, comprising: a memory storing a software application; and a processor that, when executing the software application, is configure to perform the steps of: generating a first reconstructed image based on a scaled image and a scaling kernel, wherein the first reconstructed image has a first resolution, and the scaled image has a second resolution, generating an image difference based on the scaled image and the scaling kernel, wherein the image difference indicates that at least one visual artifact resides within the first reconstructed image, modifying the scaling kernel to produce a modified scaling kernel, and generating a second reconstructed image based on the scaled image and the modified scaling kernel, wherein the at least one visual artifact has been reduced within the second reconstructed image or eliminated.