Granted patent
Techniques for upscaling images generated with undetermined downscaling kernels
- Number
- 11080824
- Published
- 2021-08-03
- Filed
- 2019-08-15
- Assignee
- DISNEY ENTERPRISES, INC.
- Inventors
- Schroers; Christopher, Wang; Yifan, Cornillere; Victor, Sorkine-Hornung; Olga, Djelouah; Abdelaziz
- CPC
- G06T3/4046
- Verdict
- Set aside generic image upscaling technique
- Source
- Google Patents · FreePatentsOnline
Abstract
A scaling application estimates a downscaling kernel used to generate a downscaled image. The scaling application upscales the downscaled image based on the estimated downscaling kernel, thereby generating a higher resolution version of the downscaled image with minimal visual artifacts. The scaling application includes various networks that perform the above operations. A kernel mapping network generates a degradation map based on the estimated downscaling kernel. A degradation-aware generator network generates a reconstructed image based on the downscaled image and the degradation map. A kernel discriminator network generates an image delta that reflects visual artifacts present in the reconstructed image. The scaling application includes a parameter optimizer that iteratively modifies the estimated downscaling kernel to reduce visual artifacts indicated in the image delta. Via one or more iterations, the scaling application generates a reasonably accurate estimation of the downscaling kernel and can then upscale the downscaled image with reduced visual artifacts.
Background
BACKGROUND Field of the Various Embodiments (1) Various embodiments relate generally to video processing and, more specifically, to techniques for upscaling images generated with undetermined downscaling kernels. Description of the Related Art (2) A typical media production pipeline includes one or more media processing stages where the resolution of the media content being processed is increased and/or decreased for various reasons. For example, the media content being processed could have been captured initially at a relatively higher resolution, such as an 8K resolution. This higher-resolution version of the media content could then be downscaled, using a known downscaling kernel, to a relatively lower resolution, such as an HD (high-definition) resolution, that is more suitable for downstream production processing operations. In the downstream portion of the media production pipeline, this lower-resolution version of the media content could be composited with additional rendered content to generate a lower-resolution composite version of the media content. Subsequently, the lower-resolution composite version of the media content could be upscaled, using the known downscaling kernel, to a relatively higher resolution that is more suitable for viewing, such as a 4K resolution. (3) One drawback of the above approach is that media content usually can be downscaled to a relatively lower resolution and then upscaled to a relatively high resolution without introducing visual art