- Number
- 20250239038
- Published
- 2025-07-24
- Filed
- 2025-01-23
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Da Costa De Azevedo; Vinicius et al.
- CPC
- G06T19/20; G06T19/00; G06T17/20; G06T15/20
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Temporally coherent neural mesh-stylization rendering technique.
Abstract
A system includes a hardware processor and a memory storing software code and a style transfer machine learning (ML) model. The hardware processor is configured to execute the software code to receive an image and a style sample of a selected stylization for an original surface mesh depicted by the image, perform a view-independent reparametrization of the original surface mesh to provide a reparametrized surface mesh, render a three-dimensional (3-D) representation of the reparametrized surface mesh, and generate, using a plurality of virtual cameras, a plurality of perspective images of the 3-D representation. The hardware processor is further configured to execute the software code to stylize, using the style transfer ML model, the style sample and the plurality of perspective images of the 3-D representation, the original surface mesh, to provide a stylized version of the original surface mesh having the selected stylization.
Background
BACKGROUND
Since the advent of the first three-dimensional (3-D) animated movie, i.e., Toy Story, a sophisticated set of tools for modeling, animating and rendering assets has been developed. A recent industry trend has been to favor more stylized depictions over realistic representations, in order to support storytelling. This preference has prompted the development of additional tools that can support new design techniques. Among these recent techniques, image-based stylization of 3-D assets allows artists to achieve new unique looks.
However, conventional approaches to performing image-based stylization of 3-D assets tend to focus on volumetric data, to focus on static meshes, or they fail to provide artistic control and are therefore unsuitable for direct incorporation into animation and visual effects (VFX) pipelines. Moreover, conventional mesh appearance modelling techniques tend to be restricted to closely following the surface of the input mesh, or to be solely focused on texture synthesis. Thus, there remains a need in the art for a mesh stylization technique capable of producing sharp, temporally-coherent and controllable stylizations of dynamic meshes.
Claims
1. A system comprising: a hardware processor; and a system memory storing a software code and a style transfer machine learning (ML) model; the hardware processor configured to execute the software code to: receive an image and a style sample of a selected stylization for an original surface mesh depicted by the image; perform a view-independent reparametrization of the original surface mesh to provide a reparametrized surface mesh; render a three-dimensional (3-D) representation of the reparametrized surface mesh; generate, using a plurality of virtual cameras, a plurality of perspective images of the 3-D representation of the reparametrized surface mesh; and stylize, using the style transfer ML model, the style sample and the plurality of perspective images of the 3-D representation of the reparametrized surface mesh, the original surface mesh, to provide a stylized version of the original surface mesh having the selected stylization. ||
8. A method for use by a system including a hardware processor and a system memory storing a software code and a style transfer machine learning (ML) model, the method comprising: receiving, by the software code executed by the hardware processor, an image and a style sample of a selected stylization for an original surface mesh depicted by the image; performing a view-independent reparametrization of the original surface mesh, by the software code executed by the hardware processor, to provide a reparametrized surface mesh; rendering, by the software code executed by the hardware processor, a three-dimensional (3-D) representation of the reparametrized surface mesh; generating, by the software code executed by the hardware processor and using a plurality of virtual cameras, a plurality of perspective images of the 3-D representation of the reparametrized surface mesh; and stylizing, by the software code executed by the hardware processor and using the style transfer ML model, the style sample and the plurality of perspective images of the 3-D representation of the reparametrized surface mesh, the original surface mesh, to provide a stylized version of the original surface mesh having the selected stylization. ||
15. A computer-readable non-transitory storage medium having stored thereon a software code and a style transfer machine learning (ML) model, wherein when executed by a hardware processor the software code instantiates a method comprising: receiving an image and a style sample of a selected stylization for an original surface mesh depicted by the image; performing a view-independent reparametrization of the original surface mesh to provide a reparametrized surface mesh; rendering, a three-dimensional (3-D) representation of the reparametrized surface mesh; generating, using a plurality of virtual cameras, a plurality of perspective images of the 3-D representation of the reparametrized surface mesh; and stylizing, using the style transfer ML model, the style sample and the plurality of perspective images of the 3-D representation of the reparametrized surface mesh, the original surface mesh, to provide a stylized version of the original surface mesh having the selected stylization.