Techniques for improved lighting models for appearance capture

Number: 12322039
Published: 2025-06-03
Filed: 2022-12-14
Assignee: Disney Enterprises, INC.
Inventors: Urnau Gotardo; Paulo Fabiano et al.
CPC: G06T15/04; G06T15/06; G06T15/506; G06T7/0012; H04N13/111; H04N13/243; H04N13/254
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Improved lighting-model appearance-capture rendering technique.

Abstract

Various embodiments include a system for rendering an object, such as human skin or a human head, from captured appearance data comprising a plurality of texels. The system includes a processor executing a texture space indirect illumination module. The system determines texture coordinates of a vector originating from a first texel where the vector intersects a second texel. The system renders the second texel from the viewpoint of the first texel based on appearance data at the second texel. Based on the rendering of the second texel, the system determines an indirect lighting intensity incident to the first texel from the second texel. The system updates appearance data at the first texel based on a direct lighting intensity and the indirect lighting intensity. The system renders the first texel based on the updated appearance data at the first texel.

Background

BACKGROUND Field of the Various Embodiments (1) Various embodiments relate generally to appearance capture systems and, more specifically, to techniques for improved lighting models for appearance capture. Description of the Related Art (2) One aspect of rendering realistic human faces is to capture the appearance of the skin, which has a complex multi-layer structure with combined surface and subsurface reflectance properties. Appearance capture techniques estimate geometry and reflectance properties of various objects (such as a human head, facial skin, and/or the like) by performing a computationally intensive inverse rendering optimization in which one or more images are re-rendered a large number of times and compared to real images captured by multiple cameras at various positions and orientations. While state-of-the-art appearance capture techniques have achieved impressive results, such techniques often make simplifying assumptions that do not always hold in reality, even though these simplifying assumptions help to reduce computational complexity and rendering time. (3) One such simplifying assumption is that light sources are often modeled as a collection of point light sources (referred to as an environment map) that are distant from the face. The light rays from each of these light sources are, therefore, assumed to strike all points on the face at the same incident angle (direction). Further, the directional distribution of incoming light is assumed to be spatial

Claims

1. A computer-implemented method for rendering an object from captured appearance data comprising a plurality of texels, the method comprising: determining, based on a projection of a plurality of 3D rays from a 3D position of a first texel, texture coordinates of a first vector originating from the first texel where the first vector intersects a second texel; rendering the second texel from a viewpoint of the first texel based on appearance data at the second texel; based on the rendering of the second texel, determining an indirect lighting intensity incident to the first texel from the second texel; updating appearance data at the first texel based on a direct lighting intensity and the indirect lighting intensity incident to the first texel from the second texel; and rendering the first texel based on the updated appearance data at the first texel. || 9. One or more non-transitory computer-readable media storing program instructions that, when executed by one or more processors, cause the one or more processors to perform steps of: determining, based on a projection of a plurality of 3D rays from a 3D position of a first texel on a surface of an object, texture coordinates of a first vector originating from the first texel where the first vector intersects a second texel; rendering the second texel from a viewpoint of the first texel based on appearance data at the second texel; based on the rendering of the second texel, determining an indirect lighting intensity incident to the first texel from the second texel; updating appearance data at the first texel based on a direct lighting intensity and the indirect lighting intensity; and rendering the first texel based on the updated appearance data at the first texel. || 17. A system, comprising: one or more memories storing instructions; and one or more processors coupled to the one or more memories and, when executing the instructions: determine, based on a projection of a plurality of 3D rays from a 3D position of a first texel on a surface of an object, texture coordinates of a first vector originating from the first texel where the first vector intersects a second texel; render the second texel from a viewpoint of the first texel based on appearance data at the second texel; based on the rendering of the second texel, determine an indirect lighting intensity incident to the first texel from the second texel; update appearance data at the first texel based on a direct lighting intensity and the indirect lighting intensity; and render the first texel based on the updated appearance data at the first texel.