Deep SDR-HDR conversion

Number: 11803946
Published: 2023-10-31
Filed: 2020-09-14
Assignee: Disney Enterprises, Inc.
Inventors: Aydin; Tunc Ozan et al.
CPC: G06N3/006; G06N3/045; G06N3/0455; G06N3/0464; G06N3/08; G06N3/09; G06N3/092; G06N7/01; G06T5/50; G06T5/60; G06T5/90
Verdict: Set aside generic HDR image conversion, no creative hook
Source: Google Patents · FreePatentsOnline

Abstract

The exemplary embodiments relate to converting Standard Dynamic Range (SDR) content to High Dynamic Range (HDR) content using a machine learning system. In some embodiments, the neural network is trained to convert an input SDR image into an HDR image using the encoded representation of a training SDR image and a training HDR image. In other embodiments, the neural network is trained to convert an input SDR image into an HDR image using a predefined set of color grading actions and the training images.

Background

BACKGROUND INFORMATION (1) A display device may support Standard Dynamic Range (SDR) content and High Dynamic Range (HDR) content. Compared to SDR content, HDR content may support a greater dynamic range of luminosity, more contrast and a wider range of colors. Therefore, HDR content may provide a better viewing experience. (2) Workflows for HDR color grading and HDR remastering require a significant amount of manual input. These processes may include multiple iterations and quality control checks that may take highly experienced technicians anywhere from a few days to a few weeks to finish. As a result, HDR color grading and HDR remastering may require a significant amount of resources. Due to this cost, a large amount of available content is not configured in HDR format. Accordingly, there is a need to reduce the complexity, time, money and other resources involved in HDR color grading and HDR remastering. (3) Techniques for converting SDR content to HDR content (e.g., HDR color grading, HDR remastering, etc.) often encounter two fundamental issues. One issue is that quantization effects and banding artifacts may become visible due to the dynamic range extension from SDR format to HDR format. Under most circumstances, it may take an experienced technician several iterations to adequately blend the different shades of similar colors. Another issue is the hallucination of clipped details that may appear in the under-exposed or over-exposed regions in the SDR image. For exampl

Claims

1. A method, comprising: collecting multiple training images, wherein the training images include a training standard dynamic range (SDR) image and a training high dynamic range (HDR) image; defining a set of color grading actions for use by a neural network; and training the neural network to convert an input SDR image into an HDR image using the defined set of color grading actions and the training images, wherein the trained neural network is configured to convert the SDR image into the HDR image by applying one or more color grading actions from the set of color grading actions and generating an output that includes an indication of the one or more color grading actions; wherein training the neural network includes generating an intermediate HDR image using the input SDR image, and measuring how close the intermediate HDR image is to the training HDR image. || 12. A method, comprising: collecting multiple training images, wherein the training images include a training standard dynamic range (SDR) image and a training high dynamic range (HDR) image; defining a set of color grading actions for use by a neural network; and training the neural network to convert an input SDR image into an HDR image using the defined set of color grading actions and the training images, wherein the trained neural network is configured to convert the SDR image into the HDR image by applying one or more color grading actions from the set of color grading actions and generating an output that includes an indication of the one or more color grading actions; wherein training the neural network to convert the input SDR image into the HDR image includes training the neural network to maximize a pixel level distance between the training SDR image and the training HDR image. || 20. A method, comprising: collecting multiple training images, wherein the training images include a training standard dynamic range (SDR) image and a training high dynamic range (HDR) image; defining a set of color grading actions for use by a neural network; and training the neural network to convert an input SDR image into an HDR image using the defined set of color grading actions and the training images, wherein the trained neural network is configured to convert the SDR image into the HDR image by applying one or more color grading actions from the set of color grading actions and generating an output that includes an indication of the one or more color grading actions; wherein the neural network includes a fully-convolutional asynchronous advantage actor-critic (A3C) network architecture.