FACE MICRO DETAIL RECOVERY VIA PATCH SCANNING, INTERPOLATION, AND STYLE TRANSFER

Number: 20250118027
Published: 2025-04-10
Filed: 2024-10-04
Assignee: DISNEY ENTERPRISES, INC.
Inventors: BRADLEY; Derek Edward et al.
CPC: G06T15/04; G06T17/20; G06T19/00; G06T19/20
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Facial micro-detail recovery VFX technique.

Abstract

The present invention sets forth a technique for performing face micro detail recovery. The technique includes generating one or more skin texture displacement maps based on images of one or more skin surfaces. The technique also includes transferring, via one or more machine learning models, stylistic elements included in the one or more skin texture displacement maps onto one or more regions included in a modified three-dimensional (3D) facial reconstruction. The technique further includes generating a final 3D facial reconstruction that includes structural elements included in the 3D facial reconstruction and the stylistic elements included in the one or more skin texture displacement maps.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and generative modeling and, more specifically, to techniques for performing face micro detail recovery. Description of the Related Art

The accurate representation of facial micro details, including wrinkles and other skin texture, is an important step in creating, rendering, and animating digital avatars and other three-dimensional (3D) character representations. A digital avatar may be a static representation of a character as a still image or a dynamic representation of a user in, e.g., a chat, telepresence, or entertainment application. Accurately modeling facial micro details provides a digital avatar with a more natural and organic appearance, rather than an overly smooth or plastic appearance. Realistic micro details are of particular importance in high-end visual productions, where extreme close-up shots are common.

Existing techniques for representing facial micro details may uniformly apply one or more generated noise patterns to an avatar's skin. The applied noise pattern modifies the uniform representation of skin texture to simulate micro details in the skin. These techniques are generally computationally simple and may be suitable for real-time modification of a digital avatar.

One drawback of noise-based texture modification is that the uniformly applied noise patterns do not include sufficient structure to accuratel

Claims

1. A computer-implemented method for performing face micro detail recovery, the computer-implemented method comprising: generating one or more skin texture displacement maps based on images of one or more skin surfaces; transferring, via one or more machine learning models, stylistic elements included in the one or more skin texture displacement maps onto one or more regions included in a modified three-dimensional (3D) facial reconstruction; and generating a final 3D facial reconstruction that includes structural elements included in the 3D facial reconstruction and the stylistic elements included in the one or more skin texture displacement maps. || 10. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: generating one or more skin texture displacement maps based on images of one or more skin surfaces; transferring, via one or more machine learning models, stylistic elements included in the one or more skin texture displacement maps onto one or more regions included in a modified three-dimensional (3D) facial reconstruction; and generating a final 3D facial reconstruction that includes structural elements included in the 3D facial reconstruction and the stylistic elements included in the one or more skin texture displacement maps. || 19. A system comprising: one or more memories storing instructions; and one or more processors for executing the instructions to: generate one or more skin texture displacement maps based on images of one or more skin surfaces; transfer, via one or more machine learning models, stylistic elements included in the one or more skin texture displacement maps onto one or more regions included in a modified three-dimensional (3D) facial reconstruction; and generate a final 3D facial reconstruction that includes structural elements included in the 3D facial reconstruction and the stylistic elements included in the one or more skin texture displacement maps.