- Number
- 11995749
- Published
- 2024-05-28
- Filed
- 2020-03-05
- Assignee
- DISNEY ENTERPRISES, INC.
- Inventors
- Borer; Dominik, Buhmann; Jakob, Guay; Martin
- CPC
- G06N3/0475; G06N3/09; G06N3/0464; G06N20/00; G06N3/045; G06T13/40
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Rig-space neural rendering technique for character digital assets.
Abstract
Various embodiments disclosed herein provide techniques for generating image data of a three-dimensional (3D) animatable asset. A rendering module executing on a computer system accesses a machine learning model that has been trained via first image data of the 3D animatable asset generated from first rig vector data. The rendering module receives second rig vector data. The rendering module generates, via the machine learning model, a second image data of the 3D animatable asset based on the second rig vector data.
Background
BACKGROUND Field of the Various Embodiments (1) The various embodiments relate generally to computer-based animation and, more specifically, to rig-space neural rendering of digital assets. Description of the Related Art (2) Oftentimes, animated and live-action movie productions for feature films, short subjects, and/or the like employ computer-based animation and/or computer-generated imagery (CGI). Typically, such movie productions are intended to be displayed on large-format displays, such as cinema screens. Accordingly, movie production professionals create high resolution three-dimensional (3D) characters and other synthetic 3D objects to generate high quality images capable of being projected onto such large displays. Such 3D characters and 3D objects are referred to herein as “3D assets.” These high-resolution 3D assets are animated by means of complex proprietary rigs, where a rig provides structure to the 3D asset that enable the 3D asset to move. Computer animators manipulate the rig to cause the 3D asset to move, using a process known as “deformation.” Surface geometry defines the look of the 3D asset. As the rig moves, the surface geometry moves correspondingly, resulting in a lifelike 3D asset that can move and interact with other assets in the movie production. Due to the complexity of high-resolution 3D assets developed for movie productions, these 3D assets are typically not animated in real-time. For example, each second of animation for a 3D asset may includ
Claims
1. A computer-implemented method for generating image data of a scene including a three-dimensional (3D) animatable asset, the method comprising: accessing a machine learning model that has been trained via first image data of the 3D animatable asset generated by rendering movements of the 3D animatable asset based on first rig vector data that is associated with a plurality of poses of an animation rig usable to deform the 3D animatable asset via a plurality of control points included in the animation rig, wherein at least one of the movements of the 3D animatable asset is rendered from a plurality of virtual camera views; receiving second rig vector data that includes a plurality of rig parameter values associated with the plurality of control points included in the animation rig; and generating, via the machine learning model, second image data of the 3D animatable asset based on the second rig vector data, wherein generating the second image data comprises inputting a one-dimensional (1D) array of the plurality of rig parameter values included in the second rig vector data into the machine learning model that outputs the second image data. ||
12. A computer-implemented method for generating image data of a scene including a three-dimensional (3D) animatable asset, the method comprising: accessing a machine learning model that has been trained via first image data of the 3D animatable asset generated by rendering movements of the 3D animatable asset based on first rig vector data that is associated with a plurality of rig poses, wherein at least one of the movements of the 3D animatable asset is rendered from a plurality of virtual camera views; receiving second rig vector data that includes a plurality of rig parameter values; generating, via the machine learning model, second image data of the 3D animatable asset based on the second rig vector data, wherein generating the second image data comprises inputting the plurality of rig parameter values included in the second rig vector data into the machine learning model; and compositing the second image data with image data associated with one or more additional 3D animatable assets to generate a composited scene, wherein the second image data includes first albedo image data and first depth image data, and wherein compositing the second image data with the image data associated with the one or more additional 3D animatable assets comprises: determining a difference between the first depth image data with second depth image data associated with a second 3D animatable asset included in the one or more additional 3D animatable assets; and displaying at least one of the first albedo image data or the second albedo image data associated with the second 3D animatable asset based on the difference. ||
13. One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: accessing a machine learning model that has been trained via first image data of a 3D animatable asset generated by rendering movements of the 3D animatable asset based on first rig vector data that is associated with a plurality of poses of an animation rig usable to deform the 3D animatable asset via a plurality of control points included in the animation rig, wherein at least one of the movements of the 3D animatable asset is rendered from a plurality of virtual camera views; receiving second rig vector data that includes a plurality of rig parameter values associated with the plurality of control points included in the animation rig; and generating, via the machine learning model, second image data of the 3D animatable asset based on the second rig vector data, wherein generating the second image data comprises inputting a one-dimensional (1D) array of the plurality of rig parameter values included in the second rig vector data into the machine learning model that outputs the second image data. ||
21. A system, comprising: a memory that includes instructions; and a processor that is coupled to the memory and, when executing the instructions: accesses a machine learning model that has been trained via first image data of a 3D animatable asset generated by rendering movements of the 3D animatable asset based on first rig vector data that is associated with a plurality of poses of an animation rig usable to deform the 3D animatable asset via a plurality of control points included in the animation rig, wherein at least one of the movements of the 3D animatable asset is rendered from a plurality of virtual camera views, receives second rig vector data that includes a plurality of rig parameter values associated with the plurality of control points included in the animation rig, and generates, via the machine learning model, second image data of the 3D animatable asset based on the second rig vector data, wherein generating the second image data comprises inputting a one-dimensional (1D) array rcprcscntation of the plurality of rig parameter values included in the second rig vector data into the machine learning model that outputs the second image data.