Fast image parsing by graph adaptive dynamic programming (GADP) performing classification, detection, and segmentation simultaneously
View Patent ↗Systems and methods are disclosed to perform image parsing on one or more images by identifying a set of similar regions from each image; assigning one or more region labels to each region and generating multiple hypotheses for region label assignment; and detecting class, location and boundary of each object in the image, wherein object classification, detection and segmentation are performed jointly during image parsing.
1. A computer implemented method to perform image parsing on one or more images, comprising:
identifying a set of similar regions from each image;
assigning one or more region labels to each region and generating multiple hypotheses for region label assignment; and
detecting class, location and boundary of each object in the image,
wherein object classification, detection and segmentation are performed simultaneously during image parsing.
2. The method of claim 1 , wherein assigning the region labels comprises applying a graph adaptive dynamic programming (GADP).
3. The method of claim 2 , comprising minimizing an objective function
S
^
=
arg
min
S
∈
Ω
∑
i
=
1
N
ζ
(
s
i
❘
S
i
p
)
where
ζ( s i |S i p )=−log( p ( x i |s i ) p ( s i |S i p ))
X=[x 1 , x 2 , . . . x N ] being a set of N image regions and S=[s 1 , s 2 , . . . , s N ] being a state array of a corresponding region and a state space s i εS={0, 1, . . . , K} where s i =0 indicates a background category with K foreground categories.
4. The method of claim 1 , comprising
assigning probabilities of each region belonging to each object class as a basis for label assignment for each region; and
optimizing the label assignment so that an overall assigning probability for all regions is maximized.
5. The method of claim 1 , comprising performing an initial region label assignment using dynamic programming.
6. The method of claim 1 , comprising applying a hierarchical image analysis to parse the image into three level of patches including a first level where pixels are grouped into super pixels based on similarity, a second level where the GADP groups the super-pixels into regions; and a third level where multiple hypotheses of region combinations are analyzed by a multi-class support vector machine (SVM).
7. The method of claim 1 , comprising performing image segmentation to the image and reducing a number of elements to be processed to a small number of regions.
8. The method of claim 1 , comprising ensuring the same object label is adjacent homogenous pixels.
9. The method of claim 1 , wherein each region comprises a set of pixels grouped based on similarities including adjacency, color and smoothness.
10. The method of claim 1 , comprising encoding each region with information including shape (boundary) and context (neighboring regions).
11. An image parsing system, comprising:
means for identifying a set of similar regions from each image;
means for assigning one or more region labels to each region and for generating multiple hypotheses for region label assignment; and
means for detecting class, location and boundary of each object in the image,
wherein object classification, detection and segmentation are performed simultaneously during image parsing.
12. The system of claim 11 , wherein means for assigning the region labels comprises means for applying a graph adaptive dynamic programming (GADP).
13. The system of claim 12 , comprising means for minimizing an objective function
S
^
=
arg
min
S
∈
Ω
∑
i
=
1
N
ζ
(
s
i
❘
S
i
p
)
where
ζ( s i |S i p )=−log( p ( x i |s i ) p ( s i |S i p ))
X=[x 1 , x 2 , . . . x N ] being a set of N image regions and S=[s 1 , s 2 , . . . , s N ] being a state array of a corresponding region and a state space s i εS={0, 1, . . . , K} where s i =0 indicates a background category with K foreground categories.
14. The system of claim 11 , comprising
means for assigning probabilities of each region belonging to each object class as a basis for label assignment for each region; and
means for optimizing the label assignment so that an overall assigning probability for all regions is maximized.
15. The system of claim 11 , comprising means for performing an initial region label assignment using dynamic programming.
16. The system of claim 11 , comprising means for applying a hierarchical image analysis to parse the image into three level of patches including a first level where pixels are grouped into super pixels based on similarity, a second level where the GADP groups the super-pixels into regions; and a third level where multiple hypotheses of region combinations are analyzed by a multi-class support vector machine (SVM).
17. The system of claim 11 , comprising means for performing image segmentation to the image and reducing a number of elements to be processed to a small number of regions.
18. The system of claim 11 , comprising means for ensuring the same object label is adjacent homogenous pixels.
19. The system of claim 11 , wherein each region comprises a set of pixels grouped based on similarities including adjacency, color and smoothness.
20. The system of claim 11 , comprising encoding each region with information including shape (boundary) and context (neighboring regions).