REAL-TIME RENDERING WITH COMPRESSED ANIMATED LIGHT FIELDS

Number: 20180322691
Published: 2018-11-08
Filed: 2018-05-04
Assignee: Disney Enterprises, Inc.
Inventors: MITCHELL; Kenneth J.; KONIARIS; Charalampos; KOSEK; Malgorzata E.; SINCLAIR; David A.
CPC: G06T15/06; G06F3/013; G06T7/586; G06T15/08; G06T15/005; G06T15/80
Verdict: High Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Novel real-time rendering (compressed light fields).

Abstract

Systems, methods, and articles of manufacture for real-time rendering using compressed animated light fields are disclosed. One embodiment provides a pipeline, from offline rendering of an animated scene from sparse optimized viewpoints to real-time rendering of the scene with freedom of movement, that includes three stages: offline preparation and rendering, stream compression, and real-time decompression and reconstruction. During offline rendering, optimal placements for cameras in the scene are determined, and color and depth images are rendered using such cameras. Color and depth data is then compressed using an integrated spatial and temporal scheme permitting high performance on graphics processing units for virtual reality applications. The compressed content may be decoded and reconstructed in real-time by selecting, using heuristics, cameras that provide useful data for a viewer, selecting grid cells from those cameras that are visible to the viewer, and using a ray marching technique to reconstruct the scene.

Background

BACKGROUNDField

Aspects of the disclosure presented herein relate to image rendering and, more specifically, to rendering images using compressed animated light fields.Description of the Related Art

Recently, virtual reality (VR) has become practical for widespread consumer adoption with a variety of hardware releases of head-mounted displays (HMDs). However, the real-time rendering performance requirements for VR content are typically much higher than for non-VR rendering. As a result, VR content has traditionally been rendered with lower-complexity visuals than non-VR rendering. For example, cinematic-quality VR graphics could not be rendered in real time.

Immersive 360-degree videos (monoscopic or stereoscopic) have also emerged as a popular form of content. However, such videos are typically captured assuming specific eye locations. As a result, traditional immersive 360-degree videos lack motion parallax and can result in immersion breaking and the feeling that the content is flat, or even discomfort, when viewers' eyes diverge from the specific eye locations from which the videos were captured. To prevent such immersion breaking and discomfort, the content being displayed to the user must be modified to appear correct from any given eye location and head angle. However, this typically requires capturing and rendering the content from a very large number of eye locations, thereby increasing authoring, storage, processing, and bandwidth costs, among othe

Claims

1. A computer-implemented method, comprising: determining placements for a plurality of virtual cameras in a virtual scene, wherein the placements for the plurality of virtual cameras provide full motion light field visibility; rendering the virtual scene using the virtual cameras at the determined placements, wherein the rendering produces color data and depth data; compressing the color data and the depth data; decompressing at least a portion of the compressed data; and reconstructing one or more video frames using the decompressed portion of the compressed data. 11. A non-transitory computer-readable storage medium storing a program, which, when executed by a processor performs operations comprising: determining placements for a plurality of virtual cameras in a virtual scene, wherein the placements for the plurality of virtual cameras provide full motion light field visibility; rendering the virtual scene using the virtual cameras at the determined placements, wherein the rendering produces color data and depth data; and compressing the color data and the depth data. 20. A system, comprising: a processor; and a memory, wherein the memory includes an application program configured to perform operations comprising: determining placements for a plurality of virtual cameras in a virtual scene, wherein the placements for the plurality of virtual cameras provide full motion light field visibility, rendering the virtual scene using the virtual cameras at the determined placements, wherein the rendering produces color data and depth data, and compressing the color data and the depth data.