TECHNIQUES FOR ENHANCING SKIN RENDERS USING NEURAL NETWORK PROJECTION FOR RENDERING COMPLETION

Number: 20220237751
Published: 2022-07-28
Filed: 2021-11-29
Assignee: DISNEY ENTERPRISES, INC.
Inventors: BRADLEY; Derek Edward, CHANDRAN; Prashanth, URNAU GOTARDO; Paulo Fabiano, RIVIERE; Jeremy, WINBERG; Sebastian Valentin, ZOSS; Gaspard
CPC: G06N3/045; G06N3/047; G06N3/0475; G06N3/08; G06N3/094; G06T11/00; G06T15/503; G06T5/60; G06T5/77; G06V10/82; G06V40/165; G06V40/171
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Neural skin-rendering enhancement technique.

Abstract

Techniques are disclosed for generating photorealistic images of head portraits. A rendering application renders a set of images that include the skin of a face and corresponding masks indicating pixels associated with the skin in the images. An inpainting application performs a neural projection technique to optimize a set of parameters that, when input into a generator model, produces a set of projection images, each of which includes a head portrait in which (1) skin regions resemble the skin regions of the face in a corresponding rendered image; and (2) non-skin regions match the non-skin regions in the other projection images when the rendered set of images are standalone images, or transition smoothly between consecutive projection images in the case when the rendered set of images are frames of a video. The rendered images can then be blended with corresponding projection images to generate composite images that are photorealistic.

Background

BACKGROUND Field of the Disclosure

Embodiments of the present disclosure relate generally to computer science and computer-generated graphics and, more specifically, to techniques for enhancing skin renders using neural network projection for rendering completion. Description of the Related Art

Realistic digital head portraits are required for various computer graphics and computer vision applications. As used herein, a head portrait refers to a representation of a human head that includes regions corresponding to the skin of a face as well as non-skin regions that can correspond to eyes, ears, scalp hair, facial hair, inside of the mouth, parts of the neck and shoulder, etc. Digital head portraits oftentimes are used in the virtual scenes of film productions and in video games, among other things.

One approach for generating digital images of head portraits (also referred to herein as “head portrait images”) involves capturing human faces and rendering the captured faces in images. However, conventional facial capture techniques are limited to capturing the skin regions of faces. Non-skin regions that are required for a head portrait are typically not captured. To generate a head portrait image that includes both skin and non-skin regions, the non-skin regions that are not captured need to be filled in, or “inpainted,” after the captured skin regions are rendered. Conventional techniques for inpainting the non-skin regions of head portraits oftentimes ca

Claims

1. A computer-implemented method for rendering a head portrait, the method comprising: rendering a first set of images, wherein each image included in the first set of images comprises one or more skin regions associated with a face; determining a set of parameters based on one or more optimization operations, the first set of images, and a machine learning model; generating a second set of images based on the set of parameters and the machine learning model, wherein each image included in the second set of images comprises one or more skin regions associated with the face and one or more non-skin regions; and blending each image included in the first set of images with a corresponding image included in the second set of images. || 11. One or more non-transitory computer-readable storage media including instructions that, when executed by at least one processor, cause the at least one processor to performing steps for rendering a head portrait, the steps comprising: rendering a first set of images, wherein each image included in the first set of images comprises one or more skin regions associated with a face; determining a set of parameters based on one or more optimization operations, the first set of images, and a machine learning model; generating a second set of images based on the set of parameters and the machine learning model, wherein each image included in the second set of images comprises one or more skin regions associated with the face and one or more non-skin regions; and blending each image included in the first set of images with a corresponding image included in the second set of images. || 20. A system comprising: one or more memories storing instructions; and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to: render a first set of images, wherein each image included in the first set of images comprises one or more skin regions associated with a face, determine a set of parameters based on one or more optimization operations, the first set of images, and a machine learning model, generate a second set of images based on the set of parameters, and the machine learning model, wherein each image included in the second set of images comprises one or more skin regions associated with the face and one or more non-skin regions, and blend each image included in the first set of images with a corresponding image included in the second set of images.