TECHNIQUES FOR DENOISING VIDEOS

Number: 20230334626
Published: 2023-10-19
Filed: 2022-04-14
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Zhang; Yang et al.
CPC: G06N3/084; G06T5/70; G06N3/0455; G06N3/096; G06N20/00; G06T5/60; G06T5/73
Verdict: Set aside generic video denoising
Source: Google Patents · FreePatentsOnline

Abstract

Techniques are disclosed for denoising videos. In some embodiments, video frames are denoised using a denoising model that includes an encoder-decoder architecture and attention modules. During training of the denoising model, the attention modules learn weightings to upweight certain dimensions of input features to help pixel registration, remove ghosting artifacts, and improve temporal consistency when the frames of a video are being denoised. The denoising model can also be used to train a student denoising model that has a same architecture as, but is smaller and faster than, the denoising model. After training, noisy video frames can be input into the denoising model and/or the student denoising model to generate corresponding denoised video frames.

Background

BACKGROUND Technical Field

Embodiments of the present disclosure relate generally to computer science and video processing and, more specifically, to techniques for denoising videos. Description of the Related Art

Image noise refers to random variation in the color and/or brightness within images. Examples of images include standalone images and the frames of a video. Images captured by cameras can contain noise due to the mechanics of camera sensors. For example, the quality of images captured in low-lighting conditions can be adversely affected by noise. Noise can also be a byproduct in images that are rendered using Monte Carlo rendering techniques.

Image denoising is the process of removing noise from an image. Conventional approaches for image denoising can introduce unwanted artifacts into denoised images. For example, when the frames of a video are denoised according to conventional approaches, the denoised frames can have temporally inconsistent denoised regions that vary from frame to frame, even when no such variations should occur. As another example, noise and texture can “drag” from one denoised frame to another denoised frame, producing a ghost-like effect. Such artifacts that are introduced during the denoising process greatly reduce the quality of the generated denoised images.

In addition, conventional approaches for image denoising require a user to have extensive knowledge of the noise characteristics within each noisy image in o

Claims

1. A computer-implemented method for denoising videos, the method comprising: receiving a video that includes a plurality of frames; and processing one or more frames included in the plurality of frames using a denoising model to generate one or more denoised frames, wherein the denoising model includes one or more attention modules, and each attention module weights one or more dimensions of a plurality of features that are input into the attention module. || 10. One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processing units, cause the one or more processing units to perform steps for denoising videos, the steps comprising: receiving a video that includes a plurality of frames; and processing one or more frames included in the plurality of frames using a denoising model to generate one or more denoised frames, wherein the denoising model includes one or more attention modules, and each attention module weights one or more dimensions of a plurality of features that are input into the attention module. || 19. A system, comprising: one or more memories storing instructions; and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to: receive a video that includes a plurality of frames, and process one or more frames included in the plurality of frames using a denoising model to generate one or more denoised frames, wherein denoising model includes one or more attention modules, and each attention module weights one or more dimensions of a plurality of features that are input into the attention module.