- Number
- 12482076
- Published
- 2025-11-25
- Filed
- 2021-11-29
- Assignee
- Disney Enterprises, INC.
- Inventors
- Bradley; Derek Edward et al.
- CPC
- G06N3/045; G06N3/047; G06N3/0475; G06N3/08; G06N3/094; G06T11/00; G06T15/503; G06T5/60; G06T5/77; G06V10/82; G06V40/165; G06V40/171
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Neural-network-based character skin-render enhancement technique.
Abstract
Techniques are disclosed for generating photorealistic images of head portraits. A rendering application renders a set of images that include the skin of a face and corresponding masks indicating pixels associated with the skin in the images. An inpainting application performs a neural projection technique to optimize a set of parameters that, when input into a generator model, produces a set of projection images, each of which includes a head portrait in which (1) skin regions resemble the skin regions of the face in a corresponding rendered image; and (2) non-skin regions match the non-skin regions in the other projection images when the rendered set of images are standalone images, or transition smoothly between consecutive projection images in the case when the rendered set of images are frames of a video. The rendered images can then be blended with corresponding projection images to generate composite images that are photorealistic.
Background
BACKGROUND Field of the Disclosure (1) Embodiments of the present disclosure relate generally to computer science and computer-generated graphics and, more specifically, to techniques for enhancing skin renders using neural network projection for rendering completion. Description of the Related Art (2) Realistic digital head portraits are required for various computer graphics and computer vision applications. As used herein, a head portrait refers to a representation of a human head that includes regions corresponding to the skin of a face as well as non-skin regions that can correspond to eyes, ears, scalp hair, facial hair, inside of the mouth, parts of the neck and shoulder, etc. Digital head portraits oftentimes are used in the virtual scenes of film productions and in video games, among other things. (3) One approach for generating digital images of head portraits (also referred to herein as “head portrait images”) involves capturing human faces and rendering the captured faces in images. However, conventional facial capture techniques are limited to capturing the skin regions of faces. Non-skin regions that are required for a head portrait are typically not captured. To generate a head portrait image that includes both skin and non-skin regions, the non-skin regions that are not captured need to be filled in, or “inpainted,” after the captured skin regions are rendered. Conventional techniques for inpainting the non-skin regions of head portraits oftentimes cannot gene
Claims
1. A computer-implemented method for rendering a head portrait, the method comprising: rendering a first set of images, wherein each image included in the first set of images comprises one or more skin regions of a face in each image and excludes non-skin regions of the face in each image, the non-skin regions of the face including background pixels associated with non-facial content; determining a set of parameters based on one or more optimization operations, the first set of images, and a machine learning model; generating a second set of images based on the set of parameters and the machine learning model, wherein each image included in the second set of images comprises one or more skin regions associated with the face and one or more non-skin regions, wherein the set of parameters constrain the one or more skin regions of each image included in the second set of images to resemble one or more skin regions of a corresponding image included in the first set of images, and wherein the set of parameters constrain the one or more non-skin regions of each image included in the second set of images to correspond to non-skin regions of other images included in the second set of images; and blending each image included in the first set of images with a corresponding image included in the second set of images. ||
11. One or more non-transitory computer-readable storage media including instructions that, when executed by at least one processor, cause the at least one processor to performing steps for rendering a head portrait, the steps comprising: rendering a first set of images, wherein each image included in the first set of images comprises one or more skin regions of a face in each image and excludes non-skin regions of the face in each image, the non-skin regions of the face including background pixels associated with non-facial content; determining a set of parameters based on one or more optimization operations, the first set of images, and a machine learning model; generating a second set of images based on the set of parameters and the machine learning model, wherein each image included in the second set of images comprises one or more skin regions associated with the face and one or more non-skin regions, wherein the set of parameters constrain the one or more skin regions of each image included in the second set of images to resemble one or more skin regions of a corresponding image included in the first set of images, and wherein the set of parameters constrain the one or more non-skin regions of each image included in the second set of images to correspond to non-skin regions of other images included in the second set of images; and blending each image included in the first set of images with a corresponding image included in the second set of images. ||
20. A system comprising: one or more memories storing instructions; and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to: render a first set of images, wherein each image included in the first set of images comprises one or more skin regions of a face in each image and excludes non-skin regions of the face in each image, the non-skin regions of the face including background pixels associated with non-facial content, determine a set of parameters based on one or more optimization operations, the first set of images, and a machine learning model, generate a second set of images based on the set of parameters, and the machine learning model, wherein each image included in the second set of images comprises one or more skin regions associated with the face and one or more non-skin regions, wherein the set of parameters constrain the one or more skin regions of each image included in the second set of images to resemble one or more skin regions of a corresponding image included in the first set of images, and wherein the set of parameters constrain the one or more non-skin regions of each image included in the second set of images to correspond to non-skin regions of other images included in the second set of images, and blend each image included in the first set of images with a corresponding image included in the second set of images.