SPEED AND FLEXIBILITY IN STYLE TRANSFER

Number: 20250342565
Published: 2025-11-06
Filed: 2024-05-03
Assignee: DISNEY ENTERPRISES, INC.
Inventors: ORTIZ; Raphael Francois et al.
CPC: G06T5/60; G06T5/50
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Fast/flexible creative-ML style-transfer technique.

Abstract

One embodiment of the present invention sets forth a technique for performing style transfer. The technique includes converting, via a trained variational autoencoder, a first set of features associated with a content sample into a second set of features from a feature space associated with one or more style samples. The technique also includes computing one or more losses based on the first set of features and the second set of features. The technique further includes generating a style transfer result based on the content sample and the one or more losses, where the style transfer result includes one or more content-based attributes of the content sample and one or more style-based attributes of the style sample.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to techniques for improving speed and flexibility in style transfer. Description of the Related Art

Style transfer refers to a technique for transferring the “style” of a first image onto a second image without modifying the content of the second image. For example, colors, patterns, and/or other style-based attributes of the first image may be transferred onto one or more faces, buildings, bridges, and/or other objects in the second image without removing the objects from the second image or adding new objects to the second image.

Neural style transfer (NST) refers to a category of style transfer techniques that leverage convolutional neural networks (CNNs) to perform style transfer. NST techniques typically extract features from both the content and style images using a pre-trained CNN and modify the features of the content image to match those of the style image. The modified features are then used to generate a new image that has the content of the original image and the style of the style image. For example, an encoder neural network could be used to generate feature maps for both the content and style images. A mean and standard deviation may be calculated for one or more portions of the feature map for the style image, and the corresponding portion(s) of the feature map for the content im

Claims

1. A computer-implemented method for performing style transfer, the method comprising: converting, via a trained variational autoencoder, a first set of features associated with a content sample into a second set of features from a feature space associated with one or more style samples; computing one or more losses based on the first set of features and the second set of features; and generating a style transfer result based on the content sample and the one or more losses, wherein the style transfer result comprises one or more content-based attributes of the content sample and one or more style-based attributes of the one or more style samples. || 11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: converting, via a trained variational autoencoder, a first set of features associated with a content sample into a second set of features from a feature space associated with one or more style samples; computing one or more losses based on the first set of features and the second set of features; and generating a style transfer result based on the content sample and the one or more losses, wherein the style transfer result comprises one or more content-based attributes of the content sample and one or more style-based attributes of the one or more style samples. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: converting, via a trained variational autoencoder, a first set of features associated with a content sample into a second set of features from a feature space associated with one or more style samples; computing one or more losses based on the first set of features and the second set of features; and generating a style transfer result based on the content sample and the one or more losses, wherein the style transfer result comprises one or more content-based attributes of the content sample and one or more style-based attributes of the one or more style samples.