SCREEN OUTPAINTING FOR IMMERSIVE EXPERIENCES

Number: 20260134504
Published: 2026-05-14
Filed: 2024-11-13
Assignee: DISNEY ENTERPRISES, INC.
Inventors: DJELOUAH; Abdelaziz, MEGARO-BOLDINI; Martina N., YU; Zhongrui
CPC: G06T3/40; G06T5/60; G06T5/77; G06T15/20; G06T15/205; G06T17/00; G06T2210/32
Verdict: Low Notable software
First reported: 2026-W29 (2026-07-15)
Source: Google Patents · FreePatentsOnline

The keeper's note

The present invention sets forth a technique for performing screen outpainting.

Abstract

The present invention sets forth a technique for performing screen outpainting. The technique includes receiving an input video sequence including one or more frames depicting a scene, identifying one or more static scene elements included in the scene, and generating a three-dimensional (3D) reconstruction of the scene based on the input video sequence and the one or more static scene elements. The technique also includes rendering one or more novel views of the scene based on the 3D reconstruction of the scene and generating one or more expanded frames, wherein each expanded frame includes a first region including visual content included in a frame included in the input video sequence, and one or more additional regions that each include visual content based on the one or more novel views of the scene. The technique further includes generating an output video sequence based on the one or more expanded frames.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to computer vision, video processing and, more specifically, to automated techniques for performing screen outpainting when displaying visual content. Description of the Related Art

Head Mounted Displays (HMDs) provide an immersive experience when viewing video content, as video content designed for viewing via an HMD may fill the entire field of view of the HMD, often extending horizontally up to 180 degrees or more in front of a viewer. Not all video content is designed for viewing via an HMD, including large quantities of existing legacy video content (e.g., movies or television shows) that were designed to be viewed via a conventional screen, such as a cinema or television screen. Such legacy video content, when viewed via an HMD, may not provide an immersive viewing experience, as the legacy video content may not fill the entire field of view of the HMD. In particular, if the legacy video content is scaled or otherwise resized to fill the vertical field of view of the HMD, the legacy video content may not fill the horizontal field of view of the HMD, leaving blank rectangular regions located on the left and right sides of the centered legacy video content. It is desirable to adapt legacy video content for viewing via an HMD, such that the presentation of the legacy video content fills the field of view of the HMD in both the vertical and horizontal dimensions.

Claims

1. A computer-implemented method for performing screen outpainting, the method comprising: receiving an input video sequence including one or more frames depicting a scene; identifying one or more static scene elements included in the scene; generating a three-dimensional (3D) reconstruction of the scene based on the input video sequence and the one or more static scene elements; rendering one or more novel views of the scene based on the 3D reconstruction of the scene; generating one or more expanded frames, wherein each expanded frame includes a first region including visual content included in a frame included in the input video sequence, and one or more additional regions that each include visual content based on the one or more novel views of the scene; and generating an output video sequence based on the one or more expanded frames. || 11. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: receiving an input video sequence including one or more frames depicting a scene; identifying one or more static scene elements included in the scene; generating a three-dimensional (3D) reconstruction of the scene based on the input video sequence and the one or more static scene elements; rendering one or more novel views of the scene based on the 3D reconstruction of the scene; generating one or more expanded frames, wherein each expanded frame includes a first region including visual content included in a frame included in the input video sequence, and one or more additional regions that each include visual content based on the one or more novel views of the scene; and generating an output video sequence based on the one or more expanded frames. || 18. A system comprising: one or more memories storing instructions; and one or more processors for executing the instructions to: receive an input video sequence including one or more frames depicting a scene; identify one or more static scene elements included in the scene; generate a three-dimensional (3D) reconstruction of the scene based on the input video sequence and the one or more static scene elements; render one or more novel views of the scene based on the 3D reconstruction of the scene; generate one or more expanded frames, wherein each expanded frame includes a first region including visual content included in a frame included in the input video sequence, and one or more additional regions that each include visual content based on the one or more novel views of the scene; and generate an output video sequence based on the one or more expanded frames.