Iterative method for salient foreground detection and multi-object segmentation
A system and method that performs iterative foreground detection and multi-object segmentation in an image is disclosed herein. A new background prior is introduced to improve the foreground segmentation results. Three complimentary methods detect and segment foregrounds containing multiple objects. The first method performs an iterative segmentation of the image to pull out the salient objects in the image. In a second method, a higher dimensional embedding of the image graph is used to estimate the saliency score and extract multiple salient objects. A third method uses a metric to automatically pick the number of eigenvectors to consider in an alternative method to iteratively compute the image saliency map. Experimental results show that these methods succeed in accurately extracting multiple foreground objects from an image.
1. A method for detecting and segmenting multiple foreground objects, comprising:
(a) constructing an image adjacency graph for an image by performing superpixel image segmentation of the image with an augmented background model;
(b) constructing a Laplacian matrix of the image adjacency graph;
(c) embedding the k smallest eigenvectors corresponding to nonzero eigenvalues as a k-dimensional embedding of graph nodes in the image adjacency graph;
(d) calculating a new saliency score by:
(i) calculating a distance between a k-dimensional embedding of a background node and a node i;
(ii) renormalizing all of the distances to lie in the range between [0, 1] to generate relevant saliency scores;
(e) computing an overall image saliency score for a saliency map generated by the relevant saliency scores;
(f) repeating steps (c) to (e) for a different k ranging from one to a predetermined number; and
(g) choosing the saliency map with highest overall image saliency score.
2. The method of claim 1 wherein the image adjacency graph comprises a reduced image representation of the image, in which a group of pixels are represented by an average color of pixels in a corresponding superpixel and represented by a node in a graph, and local relationships in the image are represented by connecting two regions in the graph if the corresponding superpixels share a border in the original image.
3. The method of claim 1 , wherein the augmented background model comprises a multi-color background model obtained by clustering the superpixel colors represented in the Lab color space.
4. The method of claim 1 wherein the k-dimensional embedding comprises a numerical representation for each node of the image region adjacency graph, where each embedding consists of k numerical descriptors corresponding to a particular node that are obtained from the k eigenvectors in consideration.
5. The method of claim 1 wherein the overall image saliency score is computed by combining a silhouette score and a mean image saliency.
6. The method of claim 1 wherein the overall image saliency map is computed by creating a new greyscale image of a size identical to the image and assigning a saliency score to each pixel that corresponds to the saliency score of the superpixel to which the pixel belongs.
7. A method for detecting and segmenting multiple foreground objects in an image, comprising;
(a) computing a number of iterations, n, which represents the largest percentage difference between dimensions of two subsequent eigenvalues;
(b) constructing an image adjacency graph by performing superpixel image segmentation of the image with an augmented background model;
(c) computing a set of saliency scores from the image adjacency graph for a current dimension k when n does not equal 1 and beginning with dimension k equal to 2;
(d) extracting a set of superpixels having a saliency score larger than a predetermined threshold;
(e) computing a new saliency score for each superpixel in the set of extracted superpixels;
(f) repeat steps (c) to (e) until dimension k is equal to n;
(g) computing an image saliency map based on a set of newest saliency scores.
8. The method of claim 7 wherein the image adjacency graph comprises a reduced image representation of the image, in which a group of pixels are represented by an average color of the pixels in a corresponding superpixel and represented by a node in a graph, and local relationships in the image are represented by connecting two regions in the graph if the corresponding superpixels share a border in the original image.
9. The method of claim 7 , wherein the augmented background model comprises a multi-color background model obtained by clustering superpixel colors represented in the Lab color space.
10. The method of claim 7 wherein all saliency scores are computed by using the Fiedler vector and rescaling the resulting scores between [0; 1].
11. The method of claim 7 , wherein the threshold is computed as a function of mean image saliency score.