Deep SDR-HDR Conversion

Number: 20240013354
Published: 2024-01-11
Filed: 2023-09-25
Assignee: Disney Enterprises, Inc.
Inventors: Aydin; Tunc Ozan et al.
CPC: G06N3/006; G06N3/045; G06N3/0455; G06N3/0464; G06N3/08; G06N3/09; G06N3/092; G06N7/01; G06T5/50; G06T5/60; G06T5/90
Verdict: Set aside video/HDR conversion, plumbing
Source: Google Patents · FreePatentsOnline

Abstract

The exemplary embodiments relate to converting Standard Dynamic Range (SDR) content to High Dynamic Range (HDR) content using a machine learning system. In some embodiments, the neural network is trained to convert an input SDR image into an HDR image using the encoded representation of a training SDR image and a training HDR image. In other embodiments, the neural network is trained to convert an input SDR image into an HDR image using a predefined set of color grading actions and the training images.

Background

BACKGROUND INFORMATION

A display device may support Standard Dynamic Range (SDR) content and High Dynamic Range (HDR) content. Compared to SDR content, HDR content may support a greater dynamic range of luminosity, more contrast and a wider range of colors. Therefore, HDR content may provide a better viewing experience.

Workflows for HDR color grading and HDR remastering require a significant amount of manual input. These processes may include multiple iterations and quality control checks that may take highly experienced technicians anywhere from a few days to a few weeks to finish. As a result, HDR color grading and HDR remastering may require a significant amount of resources. Due to this cost, a large amount of available content is not configured in HDR format. Accordingly, there is a need to reduce the complexity, time, money and other resources involved in HDR color grading and HDR remastering.

Techniques for converting SDR content to HDR content (e.g., HDR color grading, HDR remastering, etc.) often encounter two fundamental issues. One issue is that quantization effects and banding artifacts may become visible due to the dynamic range extension from SDR format to HDR format. Under most circumstances, it may take an experienced technician several iterations to adequately blend the different shades of similar colors. Another issue is the hallucination of clipped details that may appear in the under-exposed or over-exposed regions in the SDR image. F

Claims

21: A method comprising: obtaining multiple training images, wherein the training images include a training standard dynamic range (SDR) image and a training high dynamic range (HDR) image; training a neural network, using the training images and a set of color grading actions, for converting SDR images into HDR images; receiving an input SDR image; and converting the input SDR image into the HDR image using the training images and one or more color grading actions from the set of color grading actions. || 26: A system comprising: one or more processors configured to: obtain multiple training images, wherein the training images include a training standard dynamic range (SDR) image and a training high dynamic range (HDR) image; train a neural network, using the training images and a set of color grading actions, for converting SDR images into HDR images; receive an input SDR image; and convert the input SDR image into the HDR image using the training images and one or more color grading actions from the set of color grading actions. || 31: A method comprising: receiving a standard dynamic range (SDR) image; converting the SDR image into a high dynamic range (HDR) image using a neural network trained to reconstruct a rolled off highlight in the HDR image that is not visible in the SDR image; generating a set of tonal curves based on performing a regression on the HDR image; receiving a user input to modify the set of tonal curves; and modifying the HDR image based on the modified set of tonal curves.