DESIGNING EFFECTIVE INTER-PIXEL INFORMATION FLOW FOR NATURAL IMAGE MATTING

Number: 20180225827
Published: 2018-08-09
Filed: 2018-02-02
Assignee: Disney Enterprises, Inc.; ETH Zürich (Eidgenössische Technische Hochschule Zürich)
Inventors: Aksoy; Yagiz; Aydin; Tunc Ozan
CPC: G06T7/11; G06T7/90; G06T7/187; G06T7/162
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Natural image matting (VFX graphics).

Abstract

Embodiments can provide a strategy for controlling information flow both from known opacity regions to unknown regions, as well as within the unknown region itself. This strategy is formulated through the use and refinement of various affinity definitions. As a result of this strategy, a final linear system can be obtained, which can be solved in closed form. One embodiment pertains to identifying opacity information flows. The opacity information flow may include one or more of flows from pixels in the image that have similar colors to a target pixel, flows from pixels in the foreground and background to the target pixel, flows from pixels in the unknown opacity region in the image to the target pixel, flows from pixels immediately surrounding the target pixels in the image to the target pixel, and any other flow.

Background

BACKGROUND

Image matting refers to pixel estimation in images and video based on foreground and background image information. The matte defines which pixels are foreground, which are background. For pixels along the boundary or in semi-transparent regions such as hair, the matte defines the mixture of foreground and background at each pixel. Mathematically, image matting requires expressing pixel colors in the transition regions from foreground to background using a convex combination of their underlying foreground and background colors that formed the mixed-color pixel. The weight, or the opacity, of the foreground color is typically referred to as the alpha value of the pixel. Extracting the opacity information of foreground objects from an image is known as natural image matting.

The numerous natural matting methods in the literature can be mainly categorized as either sampling-based or affinity-based. Sampling-based methods typically propose a way of gathering numerous samples from the background and foreground regions defined by a trimap, and select the best-fitting pair according to their individually defined criteria to represent an unknown pixel as a mixture of foreground and background.

Affinity-based matting methods mainly make use of the pixel-similarity metrics that rely on color similarity or spatial proximity, and propagate the alpha values from regions with known opacity. Local affinity definitions look at a local patch around the pixel locati

Claims

1. A method for estimating opacity for a pixel in an image, the method being implemented by a processor configured to execute computer programs, the method comprising: obtaining regions in the image, the regions including a foreground region, a background region, and an unknown region, wherein the foreground region is fully opaque, the background is fully transparent and individual pixels the unknown region has an unknown opacity; and for each target pixel in the unknown region, estimating opacity for the target pixel by propagating opacity information from other pixels in the image to the pixel, wherein the opacity estimation comprises: identifying a first set of pixels that have colors similar to the target pixel; determining a first amount of opacity information flow from the first set of pixels to the target pixel; and estimating the opacity for the target pixel based on the first amount of opacity information flow and the second amount of opacity information flow. 11. A system for estimating opacity for a pixel in an image, the system comprising a processor configured to execute computer programs such that when the computer programs are executed, the system is caused to perform: obtaining regions in the image, the regions including a foreground region, a background region, and an unknown region, wherein the foreground region is fully opaque, the background is fully transparent and individual pixels the unknown region has an unknown opacity; and for each target pixel in the unknown region, estimating opacity for the target pixel by propagating opacity information from other pixels in the image to the pixel, wherein the opacity estimation comprises: identifying a first set of pixels that have colors similar to the target pixel; determining a first amount of opacity information flow from the first set of pixels to the target pixel; and estimating the opacity for the target pixel based on the first amount of opacity information flow and the second amount of opacity information flow.