TRANSFORMER-BASED SHAPE MODELS

Number: 20230104702
Published: 2023-04-06
Filed: 2022-02-18
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Bradley; Derek Edward et al.
CPC: G06T11/00; G06T19/20; G06N3/045; G06N3/084; G06N3/09; G06T7/50; G06T7/70
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Transformer-based shape/geometry modeling technique.

Abstract

A technique for synthesizing a shape includes generating a first plurality of offset tokens based on a first shape code and a first plurality of position tokens, wherein the first shape code represents a variation of a canonical shape, and wherein the first plurality of position tokens represent a first plurality of positions on the canonical shape. The technique also includes generating a first plurality of offsets associated with the first plurality of positions on the canonical shape based on the first plurality of offset tokens. The technique further includes generating the shape based on the first plurality of offsets and the first plurality of positions.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to transformer-based shape models. Description of the Related Art

Realistic digital representations of faces, hands, bodies, and other recognizable objects are required for various computer graphics and computer vision applications. For example, digital representations of real-world deformable objects are oftentimes used in virtual scenes of film or television productions and in video games.

One technique for representing a digital shape involves using a data-driven parametric shape model to characterize realistic variations in the appearance of the shape. The data-driven parametric shape model is typically built from a dataset of scans of the same type of shape and represents a new shape as a combination of existing shapes in the dataset.

One common parametric shape model includes a linear three-dimensional (3D) morphable model that expresses new faces, bodies, and/or other shapes as linear combinations of prototypical basis shapes from a dataset. However, the linear 3D morphable model is unable to represent continuous, nonlinear deformations that are common to faces and other recognizable shapes. At the same time, linear combinations of input shapes generated by the linear 3D morphable model can lead to unrealistic motion or physically impossible shapes. Thus, when the linear 3D morphab

Claims

1. A computer-implemented method for synthesizing a shape, the computer-implemented method comprising: generating a first plurality of offset tokens based on a first shape code and a first plurality of position tokens, wherein the first shape code represents a variation of a canonical shape, and wherein the first plurality of position tokens represent a first plurality of positions on the canonical shape; generating a first plurality of offsets associated with the first plurality of positions on the canonical shape based on the first plurality of offset tokens; and generating the shape based on the first plurality of offsets and the first plurality of positions. || 11. One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: generating a first plurality of offset tokens based on a first shape code and a first plurality of position tokens, wherein the first shape code represents a variation of a canonical shape, and wherein the first plurality of position tokens represent a first plurality of positions on the canonical shape; generating a first plurality of offsets associated with the first plurality of positions on the canonical shape based on the first plurality of offset tokens; and generating a shape based on the first plurality of offsets and the first plurality of positions. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to: generate a first plurality of offset tokens based on a first shape code and a first plurality of position tokens, wherein the first shape code represents a variation of a canonical shape, and wherein the first plurality of position tokens represent a first plurality of positions on the canonical shape; generate a first plurality of offsets associated with the first plurality of positions on the canonical shape based on the first plurality of offset tokens; and generate a shape based on the first plurality of offsets and the first plurality of positions.