Linear spatial pyramid matching using sparse coding
View Patent ↗Systems and methods are disclosed to classify an input image by determining a spatial-pyramid image representation based on sparse coding; determining a descriptor for each interest point in the input image; encoding the descriptor; and applying max pooling to form the spatial pyramid representation of images.
1. A method to classify an input image, comprising
a. determining a spatial-pyramid image representation based on sparse coding;
b. determining a descriptor for each interest point in the input image;
c. encoding the descriptor; and
d. applying max-pooling to form the spatial-pyramid image representation
e. solving an unconstrained convex optimization problem
min
w
c
{
J
(
w
c
)
=
w
c
2
+
C
∑
i
=
1
n
ℓ
(
w
c
;
y
i
c
,
z
i
)
}
where y i c =1 if y i =c, otherwise y i c =−1, and l(w c ;y i c , z i ) is a hinge loss function, where w is a data vector, z is test datum, C is a constant, and J is a cost function.
2. The method of claim 1 , wherein the interest point is detected by scanning the input image and detecting a set of key points representing objects in the input image.
3. The method of claim 2 , comprising selecting key points located on a predefined grid.
4. The method of claim 1 , comprising determining a descriptor for each interest point.
5. The method of claim 1 , comprising determining a descriptor feature vector to represent a visual pattern at a predetermined location.
6. The method of claim 1 , wherein the descriptor comprises a raw image patch or a transformed representation of the raw image patch.
7. The method of claim 1 , wherein a transformed representation comprises a local feature, a SIFT feature, or a SURF feature.
8. The method of claim 1 , comprising performing a nonlinear transformation to encode each descriptor into a compact representation.
9. The method of claim 1 , comprising applying local max pooling on multiple spatial scales to incorporate translation and scale invariance.
10. The method of claim 1 , comprising determining a spatial pyramid representation of the image.
11. The method of claim 1 , comprising determining statistics of codes for each region of the input image in a set of locations and scales, and concatenating the statistics to form a vector.
12. The method of claim 1 , comprising determining code statistics by computing an absolute value of codes.
13. The method of claim 1 , comprising applying a linear support vector machine (SVM) based on a sparse coding of a local feature.
14. The method of claim 13 , wherein a linear SVM kernel is used and wherein an image I i is represented by z i , comprising determining:
∑
l
=
0
2
∑
s
=
1
2
l
∑
t
=
1
2
l
〈
z
i
l
(
s
,
t
)
,
z
j
l
(
s
,
t
)
〉
where z i l (s,t) comprise max pooling statistics of descriptor sparse codes in a (s,t)-th segment of image I i in a scale level l.
15. The method of claim 1 , comprising applying a pooling function F as a max pooling function on absolute sparse codes
z
j
=
max
{
u
1
j
,
u
2
j
,
…
,
u
Mj
}
,
where z j is a j-th element of z, u ij is a matrix element at i-th row and j-th column of cluster membership indicator U, and M is a number of local descriptors.
16. The method of claim 1 , comprising applying sparse coding followed by multi-scale spatial max pooling and applying a linear SVM kernel based on SIFT sparse codes.