SEMI-SUPERVISED STYLE TRANSFER

Number: 20250342567
Published: 2025-11-06
Filed: 2024-05-03
Assignee: DISNEY ENTERPRISES, INC.
Inventors: ORTIZ; Raphael Francois et al.
CPC: G06T5/50; G06T5/60
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Semi-supervised creative-ML style-transfer technique.

Abstract

One embodiment of the present invention sets forth a technique for performing style transfer. The technique includes training a neural network based on (i) one or more supervised losses computed between a first set of training output produced by the neural network from a first set of training content samples and a set of stylized samples corresponding to the first set of training content samples, and (ii) one or more unsupervised losses computed using a second set of training output produced by the neural network from a second set of training content samples to generate a trained neural network. The technique also includes inputting a content sample into the trained neural network, and generating, via execution of the trained neural network, a style transfer result that comprises one or more content-based attributes of the content sample and one or more style-based attributes of the set of stylized samples.

Background

BACKGROUND Field of the Various Embodiments

Embodiments of the present disclosure relate generally to machine learning and computer vision and, more specifically, to techniques for performing semi-supervised style transfer. Description of the Related Art

Style transfer refers to a technique for transferring the “style” of a first image onto a second image without modifying the content of the second image. For example, colors, patterns, and/or other style-based attributes of the first image may be transferred onto one or more faces, buildings, bridges, and/or other objects in the second image without removing the objects from the second image or adding new objects to the second image.

Neural style transfer (NST) refers to a category of style transfer techniques that leverage convolutional neural networks (CNNs) to perform style transfer. NST techniques typically extract features from both the content and style images using a pre-trained CNN and modify the features of the content image to match those of the style image. The modified features are then used to generate a new image that has the content of the original image and the style of the style image. For example, an encoder neural network could be used to generate feature maps for both the content and style images. A mean and standard deviation may be calculated for one or more portions of the feature map for the style image, and the corresponding portion(s) of the feature map for the content image may

Claims

1. A computer-implemented method for performing style transfer, the method comprising: training a neural network based on (i) one or more supervised losses computed between a first set of training output produced by the neural network from a first set of training content samples and a set of stylized samples corresponding to the first set of training content samples, and (ii) one or more unsupervised losses computed using a second set of training output produced by the neural network from a second set of training content samples to generate a trained neural network; inputting a content sample into the trained neural network; and generating, via execution of the trained neural network, a style transfer result that comprises one or more content-based attributes of the content sample and one or more style-based attributes of the set of stylized samples. || 11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: training a neural network based on (i) one or more supervised losses computed between a first set of training output produced by the neural network from a first set of training content samples and a set of stylized samples corresponding to the first set of training content samples, and (ii) one or more unsupervised losses computed using a second set of training output produced by the neural network from a second set of training content samples to generate a trained neural network; inputting a content sample into the trained neural network; and generating, via execution of the trained neural network, a style transfer result that comprises one or more content-based attributes of the content sample and one or more style-based attributes of the set of stylized samples. || 20. A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: determining a plurality of parameters corresponding to a trained neural network, wherein the trained neural network is generated by training a neural network based on (i) one or more supervised losses computed between a first set of training output produced by the neural network from a first set of training content samples and a set of stylized samples corresponding to the first set of training content samples and (ii) one or more unsupervised losses computed using a second set of training output produced by the neural network from a second set of training content samples; inputting a content sample into the trained neural network; and generating, via execution of the trained neural network, a style transfer result that comprises one or more content-based attributes of the content sample and one or more style-based attributes of the set of stylized samples.