Image-Space UV Map-Based Tracking of Image Regions in Video

Number: 20260268706
Published: 2026-09-10
Filed: 2025-03-10
Assignee: Disney Enterprises, Inc.
Inventors: Zoss; Gaspard, Bradley; Derek Edward
CPC: G06V40/171; G06T13/40; G06T15/205; G06T15/40; G06T19/00; G06V10/267; G06V10/54; G06V10/7715
Verdict: Low Notable software
First reported: 2026-W37 (2026-09-11)
Source: Google Patents · FreePatentsOnline

The keeper's note

Tracks a region on a moving object across video frames using AI-predicted UV maps so a mask can follow it automatically — a VFX/compositing tracking tool.

Abstract

A system includes a hardware processor and a memory storing software code. The hardware processor executes the software code to receive a sequence of images each including a depiction of the same object, and for each image of the sequence of images: extract visual features of the object depicted in the image and process the visual features to predict an image-space UV map of the object. The hardware processor further executes the software code to receive a masking input identifying a portion of the object to be overlaid by a mask, and mask, using the masking input and the image-space UV map predicted for each image, a respective portion of each image of the sequence of images corresponding to the identified portion to produce a masked sequence of images.

Background

BACKGROUND

The processing performed in visual effects pipelines typically includes isolating and tracking specific parts of a video in order to achieve color correction, painting, masking or even extraction of individual elements. Often referred to as rotoscoping, these actions are often used to isolate part of a face such as eye bags, the lips, and the like, in order to apply localized effects. These localized effects can include digital make up, skin smoothing, de-aging, and/or the addition or removal of features such as scars or pimples, for example.

In the conventional art, rotoscoping is often performed on a frame-by-frame basis and typically requires many hours of manual labor to produce a good result. Although existing techniques such as optical flow allow propagating the isolated areas to neighboring frames of video, the effects of occlusions and drift limit the benefits of those existing techniques and often continue to require significant manual work to correct. Consequently, there is a need in the art for an automated technique for mapping rotoscoping effects across video frames included in a video sequence.

Claims

1. A system comprising: a hardware processor; and a memory storing a software code; the hardware processor configured to execute the software code to: receive a sequence of images, each image of the sequence of images including a depiction of a same object; for each image of the sequence of images; extract a plurality of visual features of the same object depicted in the image; and process the plurality of visual features to predict an image-space UV map of the same object depicted in the image; receive a masking input identifying a portion of the same object to be overlaid by a mask; and mask, using the masking input and the image-space UV map predicted for each image of the sequence of images, a respective portion of each image of the sequence of images corresponding to the identified portion to produce a masked sequence of images. || 11. A method for use by a system including a hardware processor and a memory storing a software code, the method comprising: receiving, by the software code executed by the hardware processor, a sequence of images, each image of the sequence of images including a depiction of a same object; for each image of the sequence of images; extracting, by the software code executed by the hardware processor, a plurality of visual features of the same object depicted in the image; and processing, by the software code executed by the hardware processor, the plurality of visual features to predict an image-space UV map of the same object depicted in the image; receiving, by the software code executed by the hardware processor, a masking input identifying a portion of the same object to be overlaid by a mask; and masking, by the software code executed by the hardware processor and using the masking input and the image-space UV map predicted for each image of the sequence of images, a respective portion of each image of the sequence of images corresponding to the identified portion to produce a masked sequence of images.