Image style transfer

Number: 12725320
Published: 2026-09-01
Filed: 2024-01-29
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Pernias Pascual de Pobil; Pablo, Iglesias Navarro; Santiago, Moore; Robert B., Juboor; David N.
CPC: G06T11/10; G06T2211/441
Verdict: Set aside dropped in weekly review
In edition: 2026-W36
Source: Google Patents · FreePatentsOnline

The keeper's note

Techniques for generating modified images using content information and style information are disclosed.

Abstract

Techniques for generating modified images using content information and style information are disclosed. First image data comprising image content information is received, and a content encoder generates a first embedding by extracting the image content information from the first image data. A second embedding generated by a style encoder is received, the second embedding comprising style information of second image data. The style information comprises color information and texture information. A decoder generates a modified image using the first embedding and the second embedding, the modified image comprising the image content information of the first image data and the style information of the second image data.

Background

CROSS REFERENCE TO RELATED APPLICATION (1) This application is related to the Applicant's concurrently filed application titled “Facial Image Swapping,” which is incorporated herein by reference in its entirety for all purposes. FIELD (2) Described embodiments relate generally to generating modified images, such as modified images comprising content information of a first image and style information of a second image. BACKGROUND (3) Digital images can be modified in various ways to generate modified images. For example, modified images can be generated to combine characteristics or content of images. Images can also be modified by adding or removing content. Image manipulations can be applied manually or using various algorithms. Current processes do not enable true blending of image content (e.g., person or object representations) to be modified in various styles and allow a user to change the amount of style influence on the modified content. SUMMARY (4) The following Summary is for illustrative purposes only and does not limit the scope of the technology disclosed in this document. (5) In an embodiment, a computer-implemented method of generating modified images using content information and style information is disclosed. First image data is received including image content information. The image content information can include positional information of a set of features in the first image data. A first embedding (e.g., a content embedding) is generated using a content en

Claims

1. A computer-implemented method of generating modified images using content information and style information, the method comprising: receiving first image data comprising image content information, wherein the image content information comprises positional information of a first object and a second object in the first image data; generating, using a content encoder, a first embedding comprising the image content information, wherein the content encoder extracts the image content information from the first image data to generate the first embedding; receiving a second embedding and a third embedding generated by a style encoder, the second embedding comprising first style information of second image data, the third embedding comprising second style information of third image data, wherein the first style information and the second style information each comprises at least one of color information or texture information; and generating, by a decoder, a modified image using the first embedding, the second embedding, and the third embedding, wherein the modified image comprises the image content information of the first image data, the first style information of the second image data, and the second style information of the third image data, wherein the generating the modified image comprises: applying a first style to the first object based on the first style information, wherein the applying the first style comprises applying a first user-selected weight of the first style to the first object; and applying a second style to the second object based on the second style information, wherein the applying the second style comprises applying a second user-selected weight of the second style to the second object. || 10. A non-transitory computer-readable medium carrying instructions that, when executed by a computing system, cause the computing system to perform operations comprising: receive first image data comprising image content information, wherein the image content information comprises positional information of a first object and a second object in the first image data; generate, using a content encoder, a content embedding comprising the image content information, wherein the content encoder extracts the image content information from the first image data to generate the content embedding; receive a first style embedding and a second style embedding generated by a style encoder, the first style embedding comprising first style information of second image data, the second style embedding comprising second style information of third image data, wherein the first style information comprises at least one of first color information or texture information, and wherein the second style information comprises at least one of second color information or texture information; and generate, by a decoder, a modified image using the content embedding, the first style embedding, and the second style embedding, wherein the modified image comprises the image content information of the first image data, the first style information of the second image data, and the second style information of the third image data, wherein the modified image comprises a first style applied to the first object and a second style applied to the second object, wherein the first style is applied based on a first user-selected weight of the first style to the first object, and wherein the second style is applied based on a second user-selected weight of the second style to the second object. || 15. A computing system comprising: at least one processor; and at least one non-transitory memory carrying instructions that, when executed by the at least one processor, cause the computing system to perform operations comprising: receive first image data comprising image content information, wherein the image content information comprises positional information of a first object and a second object in the first image data; generate, using a content encoder, a content embedding comprising the image content information, wherein the content encoder extracts the image content information from the first image data to generate the content embedding; receive a first style embedding and a second style embedding generated by a style encoder, the first style embedding comprising first style information of second image data, the second style embedding comprising second style information of third image data, wherein the first style information comprises at least one of first color information or texture information, and wherein the second style information comprises at least one of second color information or texture information; and generate, by a decoder, a modified image using the content embedding, the first style embedding, and the second style embedding, wherein the modified image comprises the image content information of the first image data, the first style information of the second image data, and the second style information of the third image data, wherein the modified image comprises a first style applied to the first object based on a first weight and a second style applied to the second object based on a second weight, wherein the first weight and the second weight are selected by a user. || 20. A computer-implemented method of generating modified images, the method comprising: receiving first image data comprising image content information, wherein the image content information comprises positional information of a first object and a second object in the first image data; generating, using a content encoder, a first embedding comprising the image content information, wherein the content encoder extracts the image content information from the first image data to generate the first embedding; receiving a second embedding and a third embedding generated by a style encoder, the second embedding comprising first style information of second image data, the third embedding comprising second style information of third image data, wherein the first style information and the second style information each comprises at least one of color information or texture information; and generating, by a decoder, a modified image using the first embedding, the second embedding, and the third embedding, wherein the modified image comprises the image content information of the first image data, the first style information of the second image data, and the second style information of the third image data, wherein the generating the modified image comprises: applying a first weight of a first style to the first object based on the first style information; and applying a second weight of a second style to the second object based on the second style information, wherein the first weight and the second weight are selected by a user. || 23. A computer-implemented method of generating modified images, the method comprising: receiving first image data comprising image content information, wherein the image content information comprises positional information of a first object and a second object in the first image data; generating, using a content encoder, a first embedding comprising the image content information, wherein the content encoder extracts the image content information from the first image data to generate the first embedding; receiving a second embedding and a third embedding generated by a style encoder, the second embedding comprising first style information of second image data, the third embedding comprising second style information of third image data, wherein the first style information and the second style information each comprises at least one of color information or texture information; and generating, by a decoder, a modified image using the first embedding, the second embedding, and the third embedding, wherein the modified image comprises the image content information of the first image data, the first style information of the second image data, and the second style information of the third image data, wherein the generating the modified image comprises: applying a first weight of a first style to the first object based on the first style information; applying a second weight of a second style to the second object based on the second style information; and discarding superfluous information of the first image data based on the second embedding or the third embedding. || 26. A computer-implemented method of generating modified images using content information and style information, the method comprising: receiving first image data comprising image content information, wherein the image content information comprises positional information of a first object, a second object, and a third in the first image data; generating, using a content encoder, a first embedding comprising the image content information, wherein the content encoder extracts the image content information from the first image data to generate the first embedding; receiving a second embedding and a third embedding generated by a style encoder, the second embedding comprising first style information of second image data, the third embedding comprising second style information of third image data, wherein the first style information and the second style information each comprises at least one of color information or texture information; and generating, by a decoder, a modified image using the first embedding, the second embedding, and the third embedding, wherein the modified image comprises the image content information of the first image data, the first style information of the second image data, and the second style information of the third image data, wherein the generating the modified image comprises: applying a first style to the first object based on the first style information; applying a second style to the second object based on the second style information; and applying a third style to the third object based on third style information, wherein the first style, the second style, and the third style are different styles.