METHODS AND SYSTEMS OF PERFORMING VIDEO OBJECT SEGMENTATION

Number: 20170236290
Published: 2017-08-17
Filed: 2016-02-16
Assignee: Disney Enterprises, Inc.
Inventors: Sorkine Hornung; Alexander, Perazzi; Federico, Wang; Oliver
CPC: G06T7/162; G06T7/143; G06V10/764; G06T7/90; G06T7/194; G06T7/215; G06V10/85; G06T7/269; G06F18/2414; G06V20/49; G06V10/426; G06F18/295; G06T7/11
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Video object segmentation using fully connected object proposals, pruned to retain the highest quality candidates.

Abstract

Techniques and systems are described for performing video segmentation using fully connected object proposals. For example, a number of object proposals for a video sequence are generated. A pruning step can be performed to retain high quality proposals that have sufficient discriminative power. A classifier can be used to provide a rough classification and subsampling of the data to reduce the size of the proposal space, while preserving a large pool of candidate proposals. A final labeling of the candidate proposals can then be determined, such as a foreground or background designation for each object proposal, by solving for a posteriori probability of a fully connected conditional random field, over which an energy function can be defined and minimized.

Background

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and payment of the necessaryfee.

Illustrative embodiments of the present invention are described in detail below with reference to the following drawing figures:

FIG. 1 illustrates an example of a system for performing video segmentation, in accordance with an embodiment of the present invention.

FIG. 2 illustrates an example of an object proposal refinement engine, in accordance with an embodiment of the present invention.

FIG. 3 illustrates an example of rebalancing or refining a set of proposals by selecting those with higher discriminative power, in accordance with an embodiment of the present invention.

FIG. 4 illustrates an example of a process of performing video segmentation, in accordance with one embodiment of the present invention.

FIG. 5A-FIG. 5C illustrate an example of video frames with a final segmentation using the techniques described herein.

FIG. 6A-FIG. 6C illustrate an example of video frames with a final segmentation using the techniques described herein.

FIG. 7A-FIG. 7C illustrate an example of video frames with a final segmentation using the techniques described herein.

FIG. 8 shows an example of a computer system that may be used in various embodiments of the p

Claims

1. A computer-implemented method of segmenting video data, the method comprising: obtaining input data, the input data including a plurality of video frames; computing object proposals for the plurality of video frames, an object proposal including a region of a video frame that is a candidate for containing at least a portion of a background object or atleast a portion of a foreground object; refining the computed object proposals into a refined set of object proposals, the refined set including fewer object proposals than the computed object proposals; generating a fully-connected random field, the fully-connected random field connecting the object proposals of the refined set of object proposals across the plurality of video frames; and solving for a posteriori probability of the fully-connected random field, the posteriori probability providing a background designation or a foreground designation for each object proposal of the refined set of object proposals. 8. A system for segmenting video data, comprising: a memory storing a plurality of instructions; and one or more processors configurable to: obtain input data, the input data including a plurality of video frames; compute object proposals for the plurality of video frames, an object proposal including a region of a video frame that is a candidate for containing at least a portion of a background object or at least a portion of a foreground object; refine the computed object proposals into a refined set of object proposals, the refined set including fewer object proposals than the computed object proposals; generate a fully-connected random field, the fully-connected random field connecting the object proposals of the refined set of object proposals across the plurality of video frames; and solve for a posteriori probability of the fully-connected random field, the posteriori probability providing a background designation or a foreground designation for each object proposal of the refined set of object proposals. 15. A computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising: instructions that cause the one or more processors to obtain input data, the input data including a plurality of video frames; instructions that cause the one or more processors to compute object proposals for the plurality of video frames, an object proposal including a region of a video frame that is a candidate for containing at least a portion of a background object or at least a portion of a foreground object; instructions that cause the one or more processors to refine the computed object proposals into a refined set of object proposals, the refined set including fewer object proposals than the computed object proposals; instructions that cause the one or more processors to generate a fully-connected randomfield, the fully-connected random field connecting the object proposals of the refined set of object proposals across the plurality of video frames; and instructions that cause the one or more processors to solve for a posteriori probability of the fully-connected random field, the posteriori probability providing a background designation or a foreground designation for each object proposal of the refined set of object proposals.