Depth codec for real-time, high-quality light field reconstruction

Number: 11153550
Published: 2021-10-19
Filed: 2018-09-21
Assignee: Disney Enterprises, Inc.
Inventors: Mitchell; Kenneth J., Koniaris; Charalampos, Kosek; Malgorzata E., Sinclair; David A.
CPC: H04N19/96; H04N19/12; H04N19/103; H04N19/19; H03M13/6572; G06T9/40; H04N19/11; H04N19/597; H04N13/161; G06T17/005; H03M13/6513; H04N19/1883; H04N13/232
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Depth codec enabling real-time light-field display reconstruction.

Abstract

Systems, methods, and articles of manufacture are disclosed that enable the compression of depth data and real-time reconstruction of high-quality light fields. In one aspect, spatial compression and decompression of depth images is divided into the following stages: generating a quadtree data structure for each depth image captured by a light field probe and difference mask associated with the depth image, with each node of the quadtree approximating a corresponding portion of the depth image data using an approximating function; generating, from the quadtree for each depth image, a runtime packed form that is more lightweight and has a desired maximum error; and assembling multiple such runtime packed forms into per-probe stream(s); and decoding at runtime the assembled per-probe stream(s). Further, a block compression format is disclosed for approximating depth data by augmenting the block compression format 3DC+(BC4) with a line and two pairs of endpoints.

Background

BACKGROUND Field of the Disclosure (1) Aspects of the disclosure presented herein relate to image rendering and, more specifically, to a depth codec for real-time, high-quality light field reconstruction. Description of the Related Art (2) Recent advances in virtual and augmented reality software and hardware have sparked interest in high-quality mixed reality experiences that seamlessly blend together characters and environments.

Claims

1. A computer-implemented method, comprising: determining a mask for a received depth image frame based, at least in part, on differences between the received depth image frame and one or more previous depth image frames; and generating a quadtree data structure that includes nodes associated with pixels of the depth image frame masked by the determined mask, wherein each node of the generated quadtree data structure indicates a respective function, of a plurality of functions, used to approximate a corresponding portion of the received depth image frame and coefficient values used in the approximation. || 11. A non-transitory computer-readable storage medium storing a program, which, when executed by a processor performs operations comprising: determining a mask for a received depth image frame; and generating a quadtree data structure that includes nodes associated with pixels of the depth image frame masked by the determined mask. || 20. A computer-implemented method, comprising: determining a mask for a depth image frame based, at least in part, on differences between the depth image frame and one or more previous depth image frames; and generating a quadtree data structure that includes nodes associated with pixels of the depth image frame masked by the mask, wherein each node of the quadtree data structure indicates a respective function of a plurality of functions and coefficient values associated with the respective function.