Predicts outdoor lighting probes for an image by learning a mapping from image features to illumination parameters using a light probe database.
Methods and systems for predicting light probes for outdoor images are disclosed. A light probe database is created to learn a mapping from the outdoor image's features to predicted outdoor light probe illumination parameters. The database includes a plurality of images, image features for each of the plurality of images, and a captured light probe for each of the plurality of images. A light probe illumination model based on a sun model andsky model is fitted to the captured light probes. The light probe for the outdoor image may be predicted based on the database dataset and fitted light probe models.
BRIEF DESCRIPTION OF THE DRAWINGS(1) The present disclosure, in accordance with one or more various embodiments, is described in detail with reference to the following figures. The figures are provided for purposes of illustration only and merely depict typical or example embodiments of the disclosure.(2) FIG. 1A illustrates a communications environment in accordance with the present disclosure.(3) FIG. 1B illustrates a high-level block diagram of a light probe prediction device.(4) FIG. 2A is an operational flow diagram illustrating an example process for creating fitting models for light probe illumination parameters based on a created light probe database. The fitted models may be used in a light probe prediction process.(5) FIG. 2B is an operationalflow diagram illustrating an example process for creating a light probe database that maybe used in the process of FIG. 2A.(6) FIG. 3 illustrates an example set of light probe data for one scene that may be used in a light probe database.(7) FIG. 4 illustrates two example sun and sky model combinations that may be used as a model for light probe illumination conditions for an outdoor image.(8) FIG. 5 is an operational flow diagram illustrating an example process for predicting a light probe for an outdoor image based on a learned mapping from the outdoor image's features to light probe illumination parameters.(9) FIG. 6 illustrates an example image that uses a predicted light probe to light an inserted virtual object.(10) FIG
1.A method, comprising: predicting a light probe for an outdoor image based on a learned mapping from the outdoor image's features to light probe illumination parameters; wherein the light probe illumination parameters comprise sun parameters and sky parameters; wherein the learned mapping is based on a light probe database and a double exponential sun model or von-Mises Fisher sun model for the sun parameters and a sky model for the sky parameters, and wherein the light probe database comprises: a plurality of images comprising a plurality of objects at a plurality of different locations captured under a plurality of illumination conditions; image features for each of the plurality of images; and a plurality ofcaptured light probes associated with the plurality of images.
9. A method, comprising: predicting a light probe for an outdoor image based on a learned mapping from the outdoor image's features to light probe illumination parameters; wherein the light probe illumination parameters comprise sun parameters and sky parameters, wherein the sun parameters comprise sun position, sun angular variation, and sun color, wherein the sky parameters comprise sky color and sky angular variation; wherein the learned mapping is based on a sun modelfor the sun parameters and a sky model for the sky parameters; wherein the learned mapping is based on a light probe database comprising: a plurality of images comprising a plurality of locations captured under a plurality of illumination conditions; image features foreach of the plurality of images; and a plurality of captured light probes associated withthe plurality of images; and wherein predicting a light probe for an outdoor image comprises: predicting the sun position based on a probabilistic model; and predicting the sun angular variation, sun colors, sky colors, and sky angular variation based on a regression technique.
12. A system, comprising: a camera configured to take an outdoor image; and a computer configured to predict a light probe for the outdoor image based on a learned mapping from the outdoor image's features to light probe illumination parameters: wherein the light probe illumination parameters comprise sun parameters and sky parameters; wherein the learned mapping is based on a light probe database and a double exponential sun model or von-Mises Fisher sun model for the sun parameters and a sky model from the sky parameters, and wherein the light probe database comprises: a plurality of images comprising a plurality of objects at a plurality of different locations captured under a plurality of illumination conditions; image features for each of the plurality of images; and a plurality of captured light probes associated with the plurality of images.
23. A method of creating a light probe database, comprising: capturing, with a camera, a plurality of images of a plurality of objects at plurality of different locations under a plurality of illumination conditions; capturing, with a camera, a light probe for each of the plurality of images; recovering image features for each of the plurality of captured images; wherein recovering image features for each of the plurality of captured images comprises recovering a normal map for each of the plurality of captured images; and calibrating and aligning the captured images and captured light probes.