Path-space motion estimation, VFX rendering technique (duplicate grant).
Particular embodiments perform a light path analysis of an image comprising a scene, wherein the scene comprises at least one refractive or reflective object. The image may be decomposed based on the light path analysis into a plurality of components, each of the components representing a contribution to lighting in the scene by a different type of light interaction. For each of the components, one or more motion vectors are extracted for each of the components in order to capture motion in the scene. Finally, a final contribution of each of the components to the image is computed based on the motion vectors.
TECHNICAL FIELD(1) This disclosure generally relates to three-dimensional (“3D”) computer graphics.BACKGROUND(2) Pixar is well known for producing award-winning three-dimensional (“3D”) computer-animated films, such as “Toy Story” (1995), “Monsters, Inc.” (2001), “Finding Nemo” (2003), “The Incredibles” (2004), “Ratatouille” (2007), “WALL-E” (2008), “Up” (2009), and “Brave” (2012). In order to produce films such as these, Pixar developed its own platform for network-distributed rendering of complex 3D graphics, including ray-traced 3D views. The RenderMan® platform includes the RenderMan® Interface Specification (an API to establish an interface between modeling programs, e.g., AUTODESK MAYA, and rendering programs in order to describe 3D scenes), RenderMan® Shading Language (a language to define various types of shaders: surface, light, volume, imager, and displacement), and PhotoRealistic RenderMan® (a rendering software system).(3) Modern computer-animated movies have reached an impressive level of visual complexity and fidelity, driven in part by the industry adoption of physically based rendering and production path tracing. Unfortunately, these gains come at tremendous computational effort. Given that hundreds of thousands of frames are needed for a feature length film, the computational costs are a critical factor that will become even more important with the proliferation of stereoscopic, high-resolution, and high-frame rate cinema and home displays.(4) These computat
1. A method comprising, by one or more computing systems: performing a light path analysis of an image comprising a scene, wherein the scene comprises at least one refractive or reflective object; for each of a plurality of pixels in the image: decomposing a color of the pixel based on the light path analysis into a plurality of components, each of the components representing a contribution to lighting in the scene by a different type of light interaction, wherein the decomposing the color of the pixel comprises, for each type of light interaction: extracting a color component representing a contribution by the respective type of light interaction to color for the scene; extracting a reflectance component representing a contribution by the respective type of light interaction to texture for the scene; computing an irradiance component representing a contribution by the respective type of light interaction to lighting for the scene; and assessing a residual component representing a contribution by all unmatched paths to lighting for the scene; for each of the components, extracting one or more motion vectors for each of the components in order to capture motion in the scene; and computing, based on the motion vectors, a final contribution of each of the components to the color of the pixel.
10. One or more computer-readable non-transitory storage media embodying software comprising instructions operable when executed to: perform a light path analysis of an image comprising a scene, wherein the scene comprises at least one refractive or reflective object; for each of a plurality of pixels in the image: decompose a color of the pixel based on the light path analysis into a plurality of components, each of the components representing a contribution to lighting in the scene by a different type of light interaction, wherein the decomposing the color of the pixel comprises, for each type of light interaction: extracting a color component representing a contribution by the respective type of light interaction to color for the scene; extracting a reflectance component representing a contribution by the respective type of light interaction to texture for the scene; computing an irradiance component representing a contribution by the respective type of light interaction to lighting for the scene; and assessing a residual component representing a contribution by all unmatched paths to lighting for the scene; for each of the components, extract one or more motion vectors for each of the components in order to capture motion in the scene; and compute, based on the motion vectors, a final contribution of each of the components to the color of the pixel.
19. A system comprising one or more processors and a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to: perform a light path analysis of an image comprising a scene, wherein the scene comprises at least one refractive or reflective object; for each of a plurality of pixels in the image: decompose a color of the pixel based on the light path analysis into a plurality of components, each of the components representing a contribution to lighting in the scene by a different type of light interaction, wherein the decomposing the color of the pixel comprises, for each type of light interaction: extracting a color component representing a contribution by the respective type of light interaction to color for the scene; extracting a reflectance component representing a contribution by the respective type of light interaction to texture for the scene; computing an irradiance component representing a contribution by the respective type of light interaction to lighting for the scene; and assessing a residual component representing a contribution by all unmatched paths to lighting for the scene; for each of the components, extract one or more motion vectors for each of the components in order to capture motion in the scene; and compute, based on the motion vectors, a final contribution of each of the components to the color of the pixel.