WEAKLY-SUPERVISED SPATIAL CONTEXT NETWORKS

Number: 20190138850
Published: 2019-05-09
Filed: 2017-11-09
Assignee: Disney Enterprises, Inc.
Inventors: SIGAL; Leonid et al.
CPC: G06F18/214; G06N3/045; G06N3/0455; G06N3/0464; G06N3/08; G06N3/084; G06N3/09; G06N3/096; G06V10/255; G06V10/454; G06V10/768
Verdict: Set aside weakly-supervised spatial context networks, generic CV research
Source: Google Patents · FreePatentsOnline

Abstract

Systems, methods and articles of manufacture for training a convolutional neural network for feature recognition within digital images. A spatial context neural network is trained using a plurality of patches cropped from a plurality of digital images, the spatial context neural network comprising a first convolutional neural network configured to predict a feature representation for a first specified portion of a first digital image, a second convolutional neural network configured to compute a feature representation for a second specified portion of a second digital image, and a spatial context module that accepts output of the first and second convolutional neural networks as input. The second convolutional neural network is refined by regressing features of the second convolutional neural network to features of the first convolutional neural network. The refined second convolutional neural network is used to recognize one or more features within a third digital image.

Background

BACKGROUNDField of the Invention

The present disclosure relates to digital image processing, and more specifically, to improved techniques for recognizing features within digital images using a spatial context neural network.Description of the Related Art

Recent advances in object categorization, detection and segmentation have been fueled by high capacity deep learning models from large, labeled data sets. However, the large-scale human supervision that is required for many of these methods limits their use for some applications. This is particularly the case with fine-grained object-level tasks such as detection or segmentation, where the annotation requirements can be become so costly and unwieldy as to render the methods impractical. One solution is to use a pre-trained model for other, potentially unrelated, image tasks, and while such pre-trained models can produce effective and highly generic feature representations, fine-tuning with task-specific labeled samples is often necessary. Moreover, while unsupervised learning techniques can be used to potentially address some of these challenges, unsupervised models have not produced representations that can rival pre-trained models, much less surpass them.SUMMARY

One embodiment provides a method and system that include training a spatial context neural network using a plurality of patches cropped from a plurality of digital images. The spatial context neural network includes a first convolutional neural ne

Claims

1. A method, comprising: training a spatial context neural network using a plurality of patches cropped from a plurality of digital images, the spatial context neural network comprising: a first convolutional neural network configured to predict a feature representation for a first specified portion of a first digital image; a second convolutional neural network configured to compute a feature representation for a second specified portion of a second digital image; and a spatial context module that accepts output of the first and second convolutional neural networks as input; refining the second convolutional neural network, by regressing features of the second convolutional neural network to features of the first convolutional neural network; and using the refined second convolutional neural network to recognize one or more features within a third digital image. 19. A system, comprising: one or more computer processors; and a non-transitory memory containing computer program code that, when executed by operation of the one or more computer processors, performs an operation comprising: training a spatial context neural network using a plurality of patches cropped from a plurality of digital images, the spatial context neural network comprising: a first convolutional neural network configured to predict a feature representation for a first specified portion of a first digital image; a second convolutional neural network configured to compute a feature representation for a second specified portion of a second digital image; and a spatial context module that accepts output of the first and second convolutional neural networks as input; refining the second convolutional neural network, by regressing features of the second convolutional neural network to features of the first convolutional neural network; and using the refined second convolutional neural network to recognize one or more features within a third digital image. 20. A non-transitory computer-readable memory containing computer program code that, when executed by operation of the one or more computer processors, performs an operation comprising: retrieving a first convolutional neural network configured to predict a feature representation for a first patch of a first digital image; refining the first convolutional neural network by regressing features of the first convolutional neural network to features of a second convolutional neural network, wherein the second convolutional neural network is configured to predict a feature representation for a second patch of the first digital image, based on an offset between the first patch and the second patch; and using the refined second convolutional neural network to recognize one or more features within a second digital image.