- Number
- 9854176
- Published
- 2017-12-26
- Filed
- 2014-01-24
- Assignee
- Lucasfilm Entertainment Company Ltd.
- Inventors
- Sanders; Michael, Bhat; Kiran, Miyashiro; Curt Isamu, Snell; Jason, Grabli; Stephane
- CPC
- G06T15/50; H04N23/73; H04N23/71
- Verdict
- Medium Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Captures real-world lighting on set during filming and reconstructs it computationally to realistically illuminate CG characters composited into the live-action scene.
Abstract
Systems and techniques for dynamically capturing and reconstructinglighting are provided. The systems and techniques may be based on a stream of images capturing the lighting within an environment as a scene is shot. Reconstructed lighting data may be used to illuminate a character in a computer-generated environment as the scene is shot. For example, a method may include receiving a stream of images representing lighting of a physical environment. The method may further include compressing the stream of imagesto reduce an amount of data used in reconstructing the lighting of the physical environment and may further include outputting the compressed stream of images for reconstructing the lighting of the physical environment using the compressed stream, the reconstructed lighting being used to render a computer-generated environment.
Background
BRIEF DESCRIPTION OF THE DRAWINGS(1) Illustrative embodimentsof the present invention are described in detail below with reference to the following drawing figures:(2) FIG. 1 illustrates an example of a system for dynamic lighting capture in a physical environment in accordance with an embodiment of the present invention.(3) FIG. 2 illustrates an example of a computer-generated environment rendered using dynamically captured and reconstructed lighting in accordance with an embodiment of the present invention.(4) FIG. 3 illustrates an example of a dynamic lighting capture and reconstruction system in accordance with one embodiment of the present invention.(5) FIG. 4 illustrates an example of a dynamic lighting capture system in accordance with one embodiment of the present invention.(6) FIG. 5 illustrates an example of a process for dynamically capturing and reconstructing lighting in accordance with one embodiment of the present invention.(7) FIG. 6 illustrates an example of a process for changing the lighting of a computer-generated environment based on a stream of images.(8) FIG. 7 shows an example of a computer system that may be used in various embodiments of the present invention.(9) FIG. 8 shows an example of an equirectangular image.DETAILED DESCRIPTION(10) In the following description, for the purposes of explanation, specific details are set forth in order to provide a thorough understanding of embodiments of the invention. However, it will be apparent that various
Claims
1. A computer-implemented method comprising: receiving a stream of one or more images of a physical environment; generating one or more light maps based on the one or more images, wherein the one ormore light maps include a plurality of light intensity values indicating lighting of the physical environment; determining a plurality of coefficients representing the plurality of light intensity values of the one or more light maps; generating one or more compressed light maps, the one or more compressed light maps including the plurality of coefficients representing the plurality of light intensity values of the one or more light maps; comparing the plurality of coefficients with stored coefficients generated for one or more previous light maps to identify one or more coefficients of the plurality of coefficients that have changed as compared with the stored coefficients of the one or more previous light maps, wherein a number of coefficients in the identified one or more coefficients is less than a number of coefficients in the plurality of coefficients; and transmitting, to a rendering device, only the identified one or more coefficients of the plurality of coefficientsfor reconstructing the lighting of the physical environment, the reconstructed lighting being usable by the rendering device to render a computer-generated environment with the lighting of the physical environment.
12. A computer-implemented method, comprising: receiving a stream of one or more images of a physical environment; generating one or more light maps based on the one or more images; compressing the one or more light maps to produce one or more compressed light maps, the one or more compressed light maps including a plurality of coefficients associated with the one or more light maps; comparing the plurality of coefficients with stored coefficients generated for one or more previous light maps to identify one or more coefficients of the plurality of coefficients that have changed as compared with the stored coefficients of the one or more previous light maps, wherein a number of coefficients in the identified one or more coefficients is less than a number of coefficients in the plurality of coefficients; and transmitting only the identified one or more coefficients of the plurality of coefficients for reconstructing the lighting of the physical environment.
13. A system, comprising: a memory storing a plurality of instructions; and one or more processors to: receive a stream of one or more images of a physical environment; generate one or more light maps based on the one or more images, wherein the one or more light maps include a plurality of light intensity values indicating lighting of the physical environment; determine a plurality of coefficients representing the plurality of light intensity values of the one or more light maps; generate one or more compressed lightmaps, the one or more compressed light maps including the plurality of coefficients representing the plurality of light intensity values of the one or more light maps; compare theplurality of coefficients with stored coefficients generated for one or more previous light maps to identify one or more coefficients of the plurality of coefficients that have changed as compared with the stored coefficients of the one or more previous light maps, wherein a number of coefficients in the identified one or more coefficients is less than a number of coefficients in the plurality of coefficients; and transmit, to a rendering device, only the identified one or more coefficients of the plurality of coefficient for reconstructing the lighting of the physical environment, the reconstructed lighting being usable by the rendering device to render a computer-generated environment with the lighting of the physical environment.
15. A computer-readable memory storing a plurality of instructionsexecutable by one or more processors, the plurality of instructions comprising: instructions that cause the one or more processors to receive a stream of one or more images of a physical environment; instructions that cause the one or more processors to generate one ormore light maps based on the one or more images, wherein the one or more light maps include a plurality of light intensity values indicating lighting of the physical environment; instructions that cause the one or more processors to determine a plurality of coefficients representing the plurality of light intensity values of the one or more light maps; instructions that cause the one or more processors to generate one or more compressed light maps including the plurality of coefficients representing the plurality of light intensity values of the one or more light maps; instructions that cause the one or more processors tocompare the plurality of coefficients with stored coefficients generated for one or more previous light maps to identify one or more coefficients of the plurality of coefficients that have changed as compared with the stored coefficients of the one or more previous light maps, wherein a number of coefficients in the identified one or more coefficients is less than a number of coefficients in the plurality of coefficients; and instructions that cause the one or more processors to transmit, to a rendering device, only the identified one or more coefficients of the plurality of coefficients have changed for reconstructing the lighting of the physical environment, the reconstructed lighting being usable by the rendering device to render a computer-generated environment with the lighting of the physicalenvironment.