Methods and systems of detecting object boundaries

Number: 10121254
Published: 2018-11-06
Filed: 2013-08-29
Assignee: Disney Enterprises, Inc.
Inventors: Wang; Oliver; Smolic; Aljosa
CPC: G06T7/215; G06T7/12; G06T7/174
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

CV object-boundary detection.

Abstract

Methods and systems described herein detect object boundaries of videos. A window around the pixel may be followed in adjacent image frames of the image frame to determine object boundaries. Inconsistencies in image patches over a temporal window are detected, and each pixel of the image frame of a video is assigned an object boundary probability. The pixel may belong to a texture edge if the window content does not change throughout the adjacent image frames, or the pixel may belong to an object boundary if the window content changes. A probability value indicating the likelihood of the pixel belonging to an object boundary is determined based on the window content change and is assigned to the corresponding pixel.

Background

TECHNICAL FIELD(1) The present application relates generally to computer vision and image processing and analysis, more particularly, some embodiments relate to methods and systems of detecting object boundaries in videos.DESCRIPTION OF THE RELATED ART(2) Edges are basic information in visual perception and an important factor for image processing. Consequently, many applications benefit from a differentiation between object contour edges and texture edges. Most of these applications use a notion of image edge strength to approximate object boundaries in scenes. However, this approximation does not distinguish between actual object boundaries and texture edges. This uncertainty is also a fundamental constraint in most of today's image-based methods.(3) Edge detection refers to mathematical methods that aim at identifying points in a digital image at which the image brightness changes sharply or, more formally, has discontinuities. The points at which image brightness changes sharply are typically organized into curved line segments termed edges. Existing edge detection algorithms still remain difficult to divide the set of detected edges into object contours and texture edges. This complicacy may be partially due to the fact that the edge type cannot be guessed from the edge strength: strong gradients might belong to texture edges whereas object contours can have weak gradients. Efficient algorithms exist to detect texture edges based on some texture statistics, but many text

Claims

1. A computer-implemented method of detecting object boundaries in a video, comprising: identifying a pixel of an image frame of a video; defining a patch for the pixel; identifying a set of neighboring image frames for the image frame, the set of neighboring image frames comprising an image frame that comes before and an image frame that comes after the image frame in the video; detecting a set of patch matches to the patch from the set of neighboring image frames by searching a plurality of patch transformations of the patch in each image frame of the set of neighboring image frames, wherein each patch match of the set of patch matches is detected by comparing pixels of the patch with pixels of a patch that is being matched, wherein the set of patch matches comprises a patch match for each image frame of the set of neighboring image frames; and calculating an object boundary probability value for the patch corresponding to the pixel by applying a difference metric to the patch and the set of patch matches; wherein the difference metric distinguishes patches corresponding to texture edges from patches corresponding to object boundaries, wherein the difference metric is a function of the set of patch matches, and wherein the difference metric is based on color and gradient information of the patch and the set of patch matches. 14. A system of detecting object boundaries in a video, comprising: a non-transitory memory storing a set of instructions; and a processor coupled to the non-transitory memory, wherein the set of instructions are configured to cause the processor to perform: identifying a pixel of an image frame of a video; defining a patch for the pixel; identifying a set of neighboring image frames for the image frame, the set of neighboring image frames comprising an image frame that comes before and an image frame that comes after the image frame in the video; detecting a set of patch matches to the patch from the set of neighboring image frames by searching a plurality of patch transformations of the patch in each image frame of the set of neighboring image frames, wherein each patch match of the set of patch matches is detected by comparing pixels of the patch with pixels of a patch that is being matched, wherein the set of patch matches comprises a patch match for each image frame of the set of neighboring image frames; and calculating an object boundary probability value for the patch corresponding to the pixel by applying a difference metric to the patch and the set of patch matches; wherein the difference metric distinguishes patches corresponding to texture edges from patches corresponding to object boundaries, wherein the difference metric is a function of the set of patch matches, and wherein the difference metric is based on color and gradient information of the patch and the set of patch matches.