MULTI-LAYER REPRESENTATIONS FOR ARTISTICALLY CONTROLLABLE STEREO CONVERSION

Number: 20260311353
Published: 2026-10-08
Filed: 2025-04-08
Assignee: DISNEY ENTERPRISES, INC.
Inventors: MEHL; Lukas Francesco, AYDIN; Tunc Ozan, WEBB; Emma L., SCHROERS; Christopher Richard, HUTCHINSON; Jacqueline P., ORTIZ; Raphael Francois
CPC: G06T7/50; G06T3/18; G06T5/50; G06T5/77; G06T2207/20221
Verdict: Low Notable software
First reported: 2026-W41 (2026-10-08)
Source: Google Patents · FreePatentsOnline

The keeper's note

Uses machine learning to predict depth and split a 2D image into layers so artists can convert flat footage into stereo 3D with manual control over the depth effect.

Abstract

The present invention sets forth a technique for performing stereo conversion. This technique includes generating, using a first set of machine learning models, a predicted depth map for an input image based on one or more depth values associated with pixels within the input image, determining one or more uncertainty regions associated with the depth map, wherein each uncertainty region identifies a different area of the input image having one or more pixels that can be in multiple depth layers on the input image, processing, using a second set of machine learning models, the depth map and the one or more uncertainty regions to determine, for each pixel in one of the one or more uncertainty regions, a foreground context and a background context for that pixel based on the depth map, and generating an output image based on the foreground context and background context.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to techniques for performing artistically controllable stereo conversion using artificial intelligence. Description of the Related Art

Stereo conversion refers to the generation of stereoscopic video content from a traditional monocular (non-stereo) video sequence. For example, live-action feature films are often not filmed in stereo, but it may be desirable for a movie studio to produce a stereo version of a film for distribution alongside a non-stereo version of the film. This is achieved through stereo conversion implemented as a post-production process. Further, movie studios often possess large amounts of legacy content, and stereo conversion allows the studio to make this legacy content available to audiences in stereo.

Stereo conversion during the post-production process is a costly and often manual process. There are several approaches that offer automated or partially automated stereo conversion techniques. One subset of approaches operates with a single depth value per pixel and, as a result, struggle in several situations. For example, intricate object boundaries, such as hair where multiple colors may be mixed, cannot be represented with a single depth. In another example, reflections and partially transparent objects also require multiple depth layers. Finally, volumetric phenomena,

Claims

1. A computer-implemented method for performing stereo conversion, the computer-implemented method comprising: generating, using a first set of machine learning models, a depth map for an input image based on one or more depth values associated with pixels within the input image; determining one or more uncertainty regions associated with the depth map, wherein each uncertainty region identifies a different area of the input image having one or more pixels that can be in multiple depth layers on the input image; processing, using a second set of machine learning models, the depth map and the one or more uncertainty regions to determine, for each pixel in one of the one or more uncertainty regions, a foreground context and a background context for that pixel based on the depth map; and generating an output image based on the foreground context and background context for each pixel in the one or more uncertainty regions, wherein the output image forms a stereo pair with the input image. || 8. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: generating, using a first set of machine learning models, a depth map for an input image based on one or more depth values associated with pixels within the input image; determining one or more uncertainty regions associated with the depth map, wherein each uncertainty region identifies a different area of the input image having one or more pixels that can be in multiple depth layers on the input image; processing, using a second set of machine learning models, the depth map and the one or more uncertainty regions to determine, for each pixel in one of the one or more uncertainty regions, a foreground context and a background context for that pixel based on the depth map; and generating an output image based on the foreground context and background context for each pixel in the one or more uncertainty regions, wherein the output image forms a stereo pair with the input image. || 15. A system, comprising: one or more memories storing instructions; and one or more processors that execute the instructions to: generate, using a first set of machine learning models, a depth map for an input image based on one or more depth values associated with pixels within the input image; determine one or more uncertainty regions associated with the depth map, wherein each uncertainty region identifies a different area of the input image having one or more pixels that can be in multiple depth layers on the input image; process, using a second set of machine learning models, the depth map and the one or more uncertainty regions to determine, for each pixel in one of the one or more uncertainty regions, a foreground context and a background context for that pixel based on the depth map; and generate an output image based on the foreground context and background context for each pixel in the one or more uncertainty regions, wherein the output image forms a stereo pair with the input image.