Automated image synthesis using a comb neural network architecture

Number: 10902571
Published: 2021-01-26
Filed: 2019-06-20
Assignee: Disney Enterprises, Inc.
Inventors: Naruniec; Jacek, Weber; Romann, Schroers; Christopher
CPC: G06T5/50; G06T5/77; G06N3/04; G06N3/0464; G06N3/045; G06N3/082; G06T5/60; G06N3/088; G06N3/0455; G06N3/0895; G06T7/32; G06T11/00
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Generative neural-network image-synthesis technique.

Abstract

An image synthesis system includes a computing platform having a hardware processor and a system memory storing a software code including a neural encoder and multiple neural decoders each corresponding to a respective persona. The hardware processor executes the software code to receive target image data, and source data that identifies one of the personas, and to map the target image data to its latent space representation using the neural encoder. The software code further identifies one of the neural decoders for decoding the latent space representation of the target image data based on the persona identified by the source data, uses the identified neural decoder to decode the latent space representation of the target image data as the persona identified by the source data to produce a swapped image data, and blends the swapped image data with the target image data to produce one or more synthesized images.

Background

BACKGROUND (1) The transfer of a visual image from a source to a target domain is an important problem in visual effects. One exemplary application of such image transfer involves the transfer of a performance from a target performer to a source actor, which may be necessary if the source actor is deceased or must be portrayed at a different age. (2) Unfortunately, many conventional approaches to performing image transfer produce low resolution images with heavy artifacting. Although techniques for producing higher resolution images exist, they are typically time consuming and painstakingly manual processes, requiring the careful structuring of filmed scenes, the placement of physical landmarks on the target performer, and manual fitting of a computer generated likeness on the target performer's face. Moreover, despite the higher resolution achievable using such manual and costly methods, an uncanny aesthetic effect often remains. SUMMARY (3) There are provided systems and methods for performing automated image synthesis using a comb neural network architecture, substantially as shown in and/or described in connection with at least one of the figures, and as set forth more completely in the claims.

Claims

1. An image synthesis system comprising: a computing platform having a hardware processor and a system memory storing a software code including a neural encoder and a plurality of neural decoders each corresponding to a respective one of a plurality of personas; the hardware processor configured to execute the software code to: receive a target image data, and a source data that identifies one of the plurality of personas; map the target image data to a latent space representation of the target image data using the neural encoder; identify one of the plurality of neural decoders for decoding the latent space representation of the target image data based on the one of the plurality of personas identified by the source data; use the identified one of the plurality of neural decoders to decode the latent space representation of the target image data as the one of the plurality of personas identified by the source data to produce a swapped image data; and blend the swapped image data with the target image data to produce one or more synthesized images. || 11. A method for use by an image synthesis system including a computing platform having a hardware processor and a system memory storing a software code including a neural encoder and a plurality of neural decoders each corresponding to a respective one of a plurality of personas, the method comprising: receiving, by the software code executed by the hardware processor, a target image data, and a source data that identifies one of the plurality of personas; mapping, by the software code executed by the hardware processor, the target image data to a latent space representation of the target image data using the neural encoder; identifying, by the software code executed by the hardware processor, one of the plurality of neural decoders for decoding the latent space representation of the target image data based on the one of the plurality of personas identified by the source data; using the identified one of the plurality of neural decoders, by the software code executed by the hardware processor, to decode the latent space representation of the target image data as the one of the plurality of personas identified by the source data to produce a swapped image data; and blending, by the software code executed by the hardware processor, the swapped image data with the target image data to produce one or more synthesized images.