- Number
- 12141945
- Published
- 2024-11-12
- Filed
- 2020-02-19
- Assignee
- Disney Enterprises, INC.
- Inventors
- Cai; Zhilin et al.
- CPC
- G06T5/70; G06T5/50; G06T5/60
- Verdict
- Set aside generic image denoising, no creative hook
- Source
- Google Patents · FreePatentsOnline
Abstract
Techniques are disclosed for training and applying a denoising model. The denoising model includes multiple specialized denoisers and a generalizer, each of which is a machine learning model. The specialized denoisers are trained to denoise images associated with specific ranges of noise parameters. The generalizer is trained to generate per-pixel denoising kernels for denoising images associated with arbitrary noise parameters using outputs of the specialized denoisers. Subsequent to training, a noisy image, such as a live-action image or a rendered image, can be denoised by inputting the noisy image into the specialized denoisers to obtain intermediate denoised images that are then input, along with the noisy image, into the generalizer to obtain per-pixel denoising kernels, which can be normalized and applied to denoise the noisy image.
Background
BACKGROUND Technical Field (1) Embodiments of the present disclosure relate generally to image processing and denoising and, more specifically, to techniques for robust image denoising. Description of the Related Art (2) Image noise refers to random variation in the color and/or brightness within images. Noise is a typical byproduct in images that are rendered using Monte Carlo rendering techniques. Images (i.e., photographs) captured by cameras can also contain noise due to the mechanics of camera sensors. For example, the quality of images captured in low-lighting conditions can be adversely affected by noise. (3) Image denoising is the process of removing noise from an image. Conventional approaches for image denoising utilize either specialized denoisers or “blind” denoisers. Specialized denoisers, such as deep denoising networks, can be trained to remove specific types of noise from images, where the noise is typically within narrow ranges of one or more noise parameters, such as additive white Gaussian noise having a certain magnitude. However, the performance of specialized denoisers drops rapidly for images having noise with noise parameters that are different than the training noise parameters. (4) In contrast to specialized denoisers, blind denoisers are typically trained using images having noise with diverse noise parameters. Although blind denoisers can be robust to different noise parameters, such robustness comes at the cost of overall denoising quality. In par
Claims
1. A computer-implemented method for denoising an image, the method comprising: processing the image using a plurality of first denoisers to generate a plurality of first denoised images, wherein each first denoiser included in the plurality of first denoisers is trained to denoise a set of images associated with a different set of noise parameters; processing the image and the plurality of first denoised images using a second denoiser, into which the image and the plurality of first denoised images are input, to generate a second denoised image, wherein the second denoiser is trained to denoise an additional set of images that includes one or more images from each set of images used to train a corresponding first denoiser included in the plurality of first denoisers; downsampling the image to generate a downsampled image; processing the downsampled image using the plurality of first denoisers to generate a plurality of third denoised images; processing the downsampled image and the plurality of third denoised images using the second denoiser, into which the downsampled image and the plurality of third denoised images are input, to generate a fourth denoised image; and combining the second denoised image and the fourth denoised image to produce a first blended image, wherein the second denoised image and the fourth denoised image are combined using a scale compositor that receives the second denoised image and the fourth denoised image as input and outputs the first blended image. ||
8. A computer-implemented method for training denoisers, the method comprising: training each first denoiser included in a plurality of first denoisers using a respective set of images associated with a different set of noise parameters and ground truth images corresponding to images included in the respective set of images; training a second denoiser using an additional set of images, ground truth images corresponding to images included in the additional set of images, and a plurality of denoised images, wherein the plurality of denoised images is generated by processing the additional set of images using the plurality of first denoisers, and wherein the additional set of images includes one or more images from each set of images used to train a corresponding first denoiser included in the plurality of first denoisers; processing an image using the plurality of first denoisers to generate a plurality of first denoised images; processing the image and the plurality of first denoised images using the second denoiser, into which the image and the plurality of first denoised images are input, to generate a second denoised image; downsampling the image to generate a downsampled image; processing the downsampled image using the plurality of first denoisers to generate a plurality of third denoised images; processing the downsampled image and the plurality of third denoised images using the second denoiser, into which the downsampled image and the plurality of third denoised images are input, to generate a fourth denoised image; and combining the second denoised image and the fourth denoised image to produce a first blended image, wherein the second denoised image and the fourth denoised image are combined using a scale compositor that receives the second denoised image and the fourth denoised image as input and outputs the first blended image. ||
16. A non-transitory computer-readable storage medium including instructions that, when executed by a processing unit, cause the processing unit to perform steps for denoising an image, the steps comprising: processing the image using a plurality of first denoisers to generate a plurality of first denoised images, wherein each first denoiser is trained to denoise a set of images associated with a different set of noise parameters; processing the image and the plurality of first denoised images using a second denoiser, into which the image and the plurality of first denoised images are input, to generate a second denoised image, wherein the second denoiser is trained to denoise an additional set of images that includes one or more images from each set of images used to train a corresponding first denoiser included in the plurality of first denoisers; downsampling the image to generate a downsampled image; processing the downsampled image using the plurality of first denoisers to generate a plurality of third denoised images; processing the downsampled image and the plurality of third denoised images using the second denoiser, into which the downsampled image and the plurality of third denoised images are input, to generate a fourth denoised image; and combining the second denoised image and the fourth denoised image to produce a first blended image, wherein the second denoised image and the fourth denoised image are combined using a scale compositor that receives the second denoised image and the fourth denoised image as input and outputs the first blended image.