Techniques for capturing dynamic appearance of skin

Number: 10818038
Published: 2020-10-27
Filed: 2018-12-05
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Beeler; Thabo, Ghosh; Abhijeet, Bradley; Derek, Riviere; Jeremy, Urnau Gotardo; Paulo F.
CPC: G06T15/506; G06T7/97; G06T7/593; G06T7/60; G06T7/90
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

VFX rendering technique capturing dynamic skin appearance (granted).

Abstract

Embodiments of the present disclosure techniques for modeling and capturing the dynamic appearance of skin. These techniques can couple dynamic reflectance parameters for skin (albedo and specular reflectance) with dynamic geometry. The disclosed techniques allow for capture and modeling of the dynamic appearance of skin for an actor. The techniques can re-render the actor's face accurately to accurately model the appearance of skin including the albedo of skin that can change primarily due to blood flow. The techniques can also re-render the actor's face accurately under multiple different lighting conditions.

Background

BACKGROUND(1) Creating and rendering realistic humans is becoming ever more important in computer graphics, with applications ranging from visual effects for entertainment, to educational and training scenarios, and even medical use cases. Digital humans pose a formidable challenge for animation techniques since their virtual appearance is comprised of many different components including shape, motion and material properties. In order to create a compelling and believable overall virtual character each of these components must be modeled realistically.(2) One of the most important challenges is to faithfully reproduce the way light interacts with skin, which can be characterized as appearance modeling. Early work has modeled skin appearance using static parameters for diffuse and specular reflectance, as described by a bidirectional reflectance distribution function (BRDF). General BRDF acquisition from human subjects is extremely challenging because the space can be highly-dimensional and hence would require dense sampling of incoming and outgoing light rays at every point on the surface, even without considering changes in surface and physiological state of the subject.(3) Static appearance capture falls short for creating convincing reproductions because, as skin shape changes over time, appearance does not remain static. Appearance also changes dynamically as a result of various factors, including changes in blood flow and in skin microstructure. While solely linear blend

Claims

1. A method for acquiring dynamic properties of facial skin, the method comprising: illuminating a face of a subject under a first illumination condition; capturing a plurality of images of a portion of the face of the subject using a plurality of cameras, the cameras having overlapping coverage of a patch of skin; generating a three-dimensional geometry model of the patch of skin using the plurality of images; applying pressure firmly on the patch of skin; releasing the pressure on the patch of skin; capturing a second plurality of images of the face of the subject over time; and computing a subject-specific blood flow curve from the second plurality of images of the patch of skin; and generating a plurality of time-varying parameter maps using the subject-specific blood flow curve for rendering the face of the subject. 11. A non-transitory, computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: illuminating a face of a subject under a first illumination condition; capturing a plurality of images of a portion of the face of the subject using a plurality of cameras, the cameras having overlapping coverage of a patch of skin; generating a three-dimensional geometry model of the patch of skin using the plurality of images; applying pressure firmly on the patch of skin; releasing the pressure on the patch of skin; capturing a second plurality of images of the face of the subject over time; and computing a subject-specific blood flow curve from the second plurality of images of the patch of skin; and generating a plurality of time-varying parameter maps using the subject-specific blood flow curve for rendering the face of the subject. 17. A system for acquiring dynamic properties of facial skin, the system comprising: a multi-view stereo-rig comprised of a plurality of cameras arranged for overlapping coverage of a face of a subject; a plurality of light sources arranged to illuminate the subject with a first illumination condition; one or more processors; a computer-readable memory coupled to the one or more processors, the computer-readable memory storing instructions that cause the one or more processors to: capture a plurality of images of the face of the subject using a plurality of cameras, the cameras having overlapping coverage of a patch of skin; generate a three-dimensional geometry model of the patch of skin using the plurality of images; apply pressure firmly on the patch of skin; release the pressure on the patch of skin; capture a second plurality of images of the face of the subject over time; and compute a subject-specific blood flow curve from the second plurality of images of the patch of skin; and generate a plurality of time-varying parameter maps using the subject-specific blood flow curve for rendering the face of the subject.