Systems and methods for estimating sky light probes for outdoor images

Number: 9860453
Published: 2018-01-02
Filed: 2015-02-04
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Matthews; Iain; Lalonde; Jean-Francois
CPC: H04N23/71; H04N23/741; H04N23/951; H04N5/265
Verdict: Medium Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Sky-light-probe estimation for outdoor images (rendering).

Abstract

Methods and systems for estimating HDR sky light probes for outdoor images are disclosed. A precaptured sky light probe database is leveraged. The database includes a plurality of HDR sky light probes captured under a plurality of different illumination conditions. A HDR sky light probe is estimated from an outdoor image by fitting a three dimensional model to an object of interest in the image and solving an inverse optimization lighting problem for the 3D model where the space of possible HDR sky light probes is constrained by the HDR sky light probes of the database.

Background

TECHNICAL FIELD(1) The present disclosure relates generally to sky light probes, and more particularly, some embodiments relate to systems and methods for estimating sky light probes for outdoor images.DESCRIPTION OF THE RELATED ART(2) Image-based lighting technology is a key component of creating photo-realistic computer-generated content for insertion in images and video. Movie special effects shots, for example, often contain several virtual objects that are inserted and relighted into the movie after the movie was filmed. Current methods of realistically illuminating these special effects objects require direct capture of the illumination conditions of the scene in which the virtual object is inserted.(3) In general, existing techniques for capturing illumination conditions of a scene require some combination of taking multiple images of the scene, capturing a specially designed calibration object (e.g. a chrome sphere) at the scene, and using expensive hardware equipment such as custom DSLR cameras. The expense, computational demand, and limitations of such techniques make image-based lighting technology inaccessible to the casual user. Additionally, conventional techniques cannot be applied to images whose illumination conditions were not recorded at the time of image capture.BRIEF SUMMARY OF THE DISCLOSURE(4) Systems and methods are provided in various embodiments for estimating sky light probes for outdoor images. In one embodiment of the technology disclosed herein,

Claims

1. A system, comprising: a camera for capturing an outdoor image; one or more processors; and one or more non-transitory computer-readable mediums operatively coupled to at least one of the one or more processors and having instructions stored thereon that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to: fit a three-dimensional (3D) model to an object of interest in the outdoor image; and estimate a sky light probe for the outdoor image by applying an inverse optimization lighting algorithm based on the fitted 3D model, wherein applying an inverse optimization lighting algorithm based on the fitted 3D model comprises constraining lighting of the optimization light algorithm to a set of a plurality of precaptured sky light probes captured under a plurality of sky illumination conditions. 11. A method, comprising: capturing an outdoor image with a camera; fitting a three-dimensional (3D) model to an object of interest in the outdoor image; and estimating a sky light probe for the outdoor image by applying an inverse optimization lighting algorithm based on the fitted 3D model, wherein applying an inverse optimization lighting algorithm based on the fitted 3D model comprises constraining lighting of the optimization light algorithm to a set of a plurality of precaptured sky light probes captured under a plurality of sky illumination conditions.