BOOTSTRAPPED ROTOSCOPE-MATTING TRAINING

Number: 20260278805
Published: 2026-09-17
Filed: 2025-03-12
Assignee: Disney Enterprises, Inc.,
Inventors: ORTIZ; Raphaël François
CPC: G06T7/194; G06T5/50; G06V10/44; G06V20/70; G06T2207/20081; G06T2207/20221
Verdict: Low Notable software
First reported: 2026-W38 (2026-09-18)
Source: Google Patents · FreePatentsOnline

The keeper's note

Trains a computer-vision model to auto-rotoscope/matte VFX footage by synthesizing composite training data from real and rendered foreground objects.

Abstract

Embodiments provide techniques for training machine learning models for rotoscoping and matting in VFX workflows. The method includes generating one or more foreground objects, comprising capturing real-world objects against one or more backgrounds using a color triangulation setup, or generating synthetic objects using multi-dimensional rendering models, generating one or more composite training samples by combining at least one of the one or more foreground objects with one or more simulated backgrounds, generating a first training dataset for a segmentation model, comprising annotating at least one of the one or more composite training samples to create a corresponding segmentation mask, and training the segmentation model to predict a segmentation mask for an input sample using the first training dataset.

Background

BACKGROUND

Rotoscoping and matting are integral processes in visual effects (VFX) workflows but remain repetitive and labor-intensive. Rotoscoping involves detecting and segmenting elements within a scene, typically using splines, control points, and tools like magnetic lassos or automated trackers to assist users. After rotoscoping generates a rough cutout, matting is performed to capture fine details and partial transparency, generating alpha mattes to define pixel-level transparency values and separating an image into layers accordingly. This separation allows for completing compositing tasks, such as background replacement or integrating foreground objects into new scenes. Recent developments in procedural techniques and machine learning-based methods have introduced partial automation into the VFX workflow, but challenges remain in handling complex edges and transparency, as well as maintaining temporal coherence in video applications. SUMMARY

One embodiment presented in this disclosure provides a method, including generating one or more foreground objects, comprising at least one of capturing real-world objects against one or more backgrounds using a color triangulation setup, or generating synthetic objects using multi-dimensional rendering models, generating one or more composite training samples by combining at least one of the one or more foreground objects with one or more simulated backgrounds, generating a first training dataset for a segmentation m

Claims

1. A method, comprising: generating one or more foreground objects, comprising at least one of: capturing real-world objects against one or more backgrounds using a color triangulation setup, or generating synthetic objects using multi-dimensional rendering models; generating one or more composite training samples by combining at least one of the one or more foreground objects with one or more simulated backgrounds; generating a first training dataset for a segmentation model, comprising: annotating at least one of the one or more composite training samples to create a corresponding segmentation mask; and training the segmentation model to predict a segmentation mask for an input sample using the first training dataset. || 12. A non-transitory computer-readable medium containing computer program code that, when executed by operation of one or more computer processors, performs an operation comprising: generating one or more foreground objects, comprising at least one of: capturing real-world objects against one or more backgrounds using a color triangulation setup, or generating synthetic objects using multi-dimensional rendering models; generating one or more composite training samples by combining at least one of the one or more foreground objects with one or more simulated backgrounds; generating a first training dataset for a segmentation model, comprising: annotating at least one of the one or more composite training samples to create a corresponding segmentation mask; and training the segmentation model to predict a segmentation mask for an input sample using the first training dataset. || 20. A system, comprising: one or more memories collectively containing one or more programs; and one or more processors, wherein the one or more processors are configured to, individually or collectively, perform an operation comprising: generating one or more foreground objects, comprising at least one of: capturing real-world objects against one or more backgrounds using a color triangulation setup, or generating synthetic objects using multi-dimensional rendering models; generating one or more composite training samples by combining at least one of the one or more foreground objects with one or more simulated backgrounds; generating a first training dataset for a segmentation model, comprising: annotating at least one of the one or more composite training samples to create a corresponding segmentation mask; and training the segmentation model to predict a segmentation mask for an input sample using the first training dataset.