Deep SDR-HDR Conversion

Number: 20220084170
Published: 2022-03-17
Filed: 2020-09-14
Assignee: Disney Enterprises, Inc.
Inventors: AYDIN; Tunc Ozan, ZHANG; Yang, SLOAN; Blake, SCHULTE; Jeroen, MALONEY; Michael
CPC: G06N3/006; G06N3/045; G06N3/0455; G06N3/0464; G06N3/08; G06N3/09; G06N3/092; G06N7/01; G06T5/50; G06T5/60; G06T5/90
Verdict: Set aside generic HDR image conversion, no creative hook
Source: Google Patents · FreePatentsOnline

Abstract

The exemplary embodiments relate to converting Standard Dynamic Range (SDR) content to High Dynamic Range (HDR) content using a machine learning system. In some embodiments, the neural network is trained to convert an input SDR image into an HDR image using the encoded representation of a training SDR image and a training HDR image. In other embodiments, the neural network is trained to convert an input SDR image into an HDR image using a predefined set of color grading actions and the training images.

Background

BACKGROUND INFORMATION

A display device may support Standard Dynamic Range (SDR) content and High Dynamic Range (HDR) content. Compared to SDR content, HDR content may support a greater dynamic range of luminosity, more contrast and a wider range of colors. Therefore, HDR content may provide a better viewing experience.

Workflows for HDR color grading and HDR remastering require a significant amount of manual input. These processes may include multiple iterations and quality control checks that may take highly experienced technicians anywhere from a few days to a few weeks to finish. As a result, HDR color grading and HDR remastering may require a significant amount of resources. Due to this cost, a large amount of available content is not configured in HDR format. Accordingly, there is a need to reduce the complexity, time, money and other resources involved in HDR color grading and HDR remastering.

Techniques for converting SDR content to HDR content (e.g., HDR color grading, HDR remastering, etc.) often encounter two fundamental issues. One issue is that quantization effects and banding artifacts may become visible due to the dynamic range extension from SDR format to HDR format. Under most circumstances, it may take an experienced technician several iterations to adequately blend the different shades of similar colors. Another issue is the hallucination of clipped details that may appear in the under-exposed or over-exposed regions in the SDR image. F

Claims

1. A method, comprising: collecting multiple training images, wherein the training images include a training standard dynamic range (SDR) image and a training high dynamic range (HDR) image; generating a low dimensional latent representation of the training SDR image; generating an HDR image by decoding the low dimensional latent representation of the training SDR image; and generating an SDR to HDR conversion model using the HDR image, the training SDR image and the training HDR image, wherein the SDR to HDR conversion model is configured to convert an input SDR image into an output HDR image and trained to reconstruct a rolled off highlight in the output HDR image that was not visible in the input SDR image. || 6. A method, comprising: collecting multiple training images, wherein the training images include a training standard dynamic range (SDR) image and a training high dynamic range (HDR) image; defining a set of color grading actions that may be performed by a neural network; and training a neural network to convert an input SDR image into an HDR image using the defined set of color grading actions and the training images, wherein converting the SDR image includes applying one or more color grading actions from the set of color grading actions and generating output that includes an indication of the one or more color grading actions. || 15. A method, comprising: receiving a standard dynamic range (SDR) image; converting the SDR image into a high dynamic range (HDR) image using a neural network trained to reconstruct a rolled off highlight in the HDR image that was not visible in the SDR image; generating a set of tonal curves based on performing a regression on the HDR image; receiving user input to modify the set of tonal curves; and generating a modified HDR image based on the user input.