APPEARANCE SYNTHESIS OF DIGITAL FACES

Number: 20210279938
Published: 2021-09-09
Filed: 2020-06-08
Assignee: DISNEY ENTERPRISES, INC.
Inventors: CHANDRAN; Prashanth, BEELER; Dominik Thabo, BRADLEY; Derek Edward
CPC: G06T17/00; G06N3/045; G06N3/04; G06N3/09; G06T15/04; G06T19/00; G06T15/10; G06N3/0464; G06N3/08; G06N3/0475; G06N3/094; G06N3/047
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

ML digital-face appearance synthesis technique.

Abstract

Techniques are disclosed for generating digital faces. In some examples, a style-based generator receives as inputs initial tensor(s) and style vector(s) corresponding to user-selected semantic attribute styles, such as the desired expression, gender, age, identity, and/or ethnicity of a digital face. The style-based generator is trained to process such inputs and output low-resolution appearance map(s) for the digital face, such as a texture map, a normal map, and/or a specular roughness map. The low-resolution appearance map(s) are further processed using a super-resolution generator that is trained to take the low-resolution appearance map(s) and low-resolution 3D geometry of the digital face as inputs and output high-resolution appearance map(s) that align with high-resolution 3D geometry of the digital face. Such high-resolution appearance map(s) and high-resolution 3D geometry can then be used to render standalone images or the frames of a video that include the digital face.

Background

BACKGROUND Technical Field

Embodiments of the present disclosure relate generally to computer science and computer graphics and, more specifically, to appearance synthesis of digital faces. Description of the Related Art

Realistic digital faces are required for various computer graphics and computer vision applications. For example, digital faces are oftentimes used in virtual scenes of film productions and video games. Depending on the desired level of quality, an artist can spend hours, days, weeks, or even months trying to create a realistic-looking digital face.

A digital face typically includes three-dimensional (3D) geometry of a face as well as one or more two-dimensional (2D) appearance maps that specify surface characteristics of the face. For example, a texture map is an appearance map that can specify colors on the surface of the face. As another example, a normal map is an appearance map that can specify bumps and dents on the surface of the face. As yet another example, a specular roughness map is an appearance map that can specify shininess on the surface of the face. To render realistic images effectively, the different appearance maps need to align with the 3D geometry of the face. For example, colors associated with facial features such as freckles, moles, and/or blemishes on a texture map should be aligned with the specific 3D geometry of the face corresponding to those facial features. As another example, the colors of a texture map may

Claims

1. A computer-implemented method for rendering one or more images of a digital face, the method comprising: generating, via a first machine learning model, one or more first appearance maps based on a user selection of one or more styles associated with one or more attributes of a digital face; generating, via a second machine learning model, one or more second appearance maps and a first three-dimensional (3D) geometry associated with the digital face based on the one or more first appearance maps and a second 3D geometry associated with the digital face; and rendering one or more images including the digital face based on the one or more second appearance maps and the first 3D geometry. || 10. A non-transitory computer-readable storage medium including instructions that, when executed by a processing unit, cause the processing unit to perform steps for rendering one or more images of a digital face, the steps comprising: generating, via a first machine learning model, one or more first appearance maps based on a user selection of one or more styles associated with one or more attributes of a digital face; generating, via a second machine learning model, one or more second appearance maps and a first three-dimensional (3D) geometry associated with the digital face based on the one or more first appearance maps and a second 3D geometry associated with the digital face; and rendering one or more images including the digital face based on the one or more second appearance maps and the first 3D geometry. || 20. A computing device comprising: a memory storing an application; and a processor coupled to the memory, wherein when executed by the processor, the application causes the processor to: generate, via a first machine learning model, one or more first appearance maps based on a user selection of one or more styles associated with one or more attributes of a digital face; generate, via a second machine learning model, one or more second appearance maps and a first three-dimensional (3D) geometry associated with the digital face based on the one or more first appearance maps and a second 3D geometry associated with the digital face; and render one or more images including the digital face based on the one or more second appearance maps and the first 3D geometry.