Method of generating feature vector, generating histogram, and learning classifier for recognition of behavior
Example embodiments disclose a method of generating a feature vector, a method of generating a histogram, a learning unit classifier, a recognition apparatus, and a detection apparatus, in which a feature point is detected from an input image based on a dominant direction analysis of a gradient distribution, and a feature vector corresponding to the detected feature point is generated.
1. A method of generating a feature vector, the method comprising:
detecting a feature point from an input image based on a dominant direction of a gradient distribution in the input image, the detecting including,
detecting a pixel corresponding to a window having a contrast of coherence in a dominant direction; and
generating a feature vector corresponding to the feature point.
2. The method of claim 1 , wherein the
contrast of coherence is an indication of differences between at least one eigenvalue associated with the detected pixel and eigenvalues of other pixels in the window.
3. The method of claim 1 , wherein the generating of the feature vector comprises:
accumulating a strength of coherence in a dominant direction within a local area corresponding to the feature point to generate the feature vector.
4. The method of claim 1 , wherein the detecting of the feature point comprises:
calculating a gradient for a plurality of pixels comprised in the input image;
calculating a structure tensor for the plurality of pixels based on the gradient;
calculating a maximum eigenvalue for the plurality of pixels by performing an Eigen analysis on the structure tensor; and
determining the feature point through a contrast amongst maximum eigenvalues.
5. The method of claim 4 , wherein the calculating of the structure tensor for the plurality of pixels comprises:
when the input image is a video image, calculating a structure tensor of a single pixel based on a matrix
[
∑
B
G
x
2
∑
B
G
x
G
y
∑
B
G
x
G
t
∑
B
G
x
G
y
∑
B
G
y
2
∑
B
G
y
G
t
∑
B
G
x
G
t
∑
B
G
y
G
t
∑
B
G
t
2
]
,
wherein G x denotes a gradient in an x axis direction, G y denotes a gradient in a y axis direction, G t denotes a gradient in a time axis direction, and B denotes a predetermined size of a block comprising the single pixel.
6. The method of claim 4 , wherein the calculating of the structure tensor for the plurality of pixels comprises:
when the input image is a still image, calculating a structure tensor of a single pixel based on a matrix
[
∑
B
G
x
2
∑
B
G
x
G
y
∑
B
G
x
G
y
∑
B
G
y
2
]
,
wherein G x denotes a gradient in an x axis direction, G y denotes gradient in a y axis direction, and B denotes a predetermined size of a block comprising the single pixel.
7. The method of claim 4 , wherein the determining of the feature point comprises:
calculating a sum of differences between a strength of coherence of a single pixel and strengths of coherence of neighboring pixels within a window corresponding to the single pixel; and
determining the single pixel to be the feature point when the sum is greater than a threshold value.
8. The method of claim 7 , wherein the strength of coherence comprises at least one of:
a maximum eigenvalue of the single pixel; and
a difference between the maximum eigenvalue and another eigenvalue of the single pixel.
9. The method of claim 1 , wherein the generating of the feature vector comprises:
with respect to a local pixels comprised in a local region,
mapping the dominant direction of a local pixel to predetermined bins; and
accumulating a dominant direction energy of the local pixel in the mapped bin.
10. The method of claim 9 , wherein the dominant direction energy is a strength of coherence associated with a gradient of the local pixel, and the dominant direction corresponds to a maximum eigenvalue associated with the gradient of the local pixel.
11. The method of claim 9 , wherein the predetermined bins quantize a time and a space of the input image to a predetermined number when the input image is a video image.
12. The method of claim 9 , wherein the predetermined bins quantize the space of the input image to a predetermined number when the input image is a still image.
13. The method of claim 1 , wherein the gradient comprises:
when the input image is a video image, a gradient in an x axis direction in a frame comprised in the input image, a gradient in a y axis direction in the frame comprised in the input image, and a gradient in a time axis direction between frames comprised in the input image.
14. The method of claim 1 , wherein the gradient comprises:
when the input image is a still image, a gradient in an x axis direction in a frame comprised in the input image and a gradient in a y axis direction in the frame comprised in the input image.
15. The method of claim 1 , further comprising:
generating of the feature vector corresponding to the input image based on a frequency characteristic extracted from the input image.
16. The method of claim 1 , further comprising:
dividing the input image into combinations of a plurality of global regions using a spatial pyramid; and
extracting frequency characteristics of the plurality of global regions comprised in the combinations of the plurality of global regions, and generating feature vectors corresponding to the plurality of global regions.
17. A non-transitory computer-readable medium comprising a program for instructing a computer to perform the method of claim 1 .
18. A method of image processing, the method comprising:
detecting a plurality of feature points comprised in an input image based on a dominant direction of a gradient distribution in the input image;
generating a plurality of feature vectors corresponding to the plurality of feature points, the generating including
accumulating a strength of coherence in the dominant direction within a local region corresponding to the plurality of feature points in order to generate the plurality of feature vectors; and
mapping the plurality of feature vectors to codewords in a codebook, to generate a histogram corresponding to the input image based on the mapped feature vectors.
19. The method of claim 18 , further comprising:
analyzing the input image based on a learned parameter and the histogram.
20. The method of claim 19 , wherein the learned parameter is generated by pre-learning a plurality of training images based on the dominant direction analysis of the gradient distribution.
21. The method of claim 19 , wherein the analyzing of the input image comprises:
recognizing content of the input image.
22. The method of claim 21 , wherein the content of the input image comprises at least one of:
a behavior of a performer comprised in the input image; and
an object comprised in the input image.
23. The method of claim 18 , wherein the generating of the histogram comprises:
normalizing values corresponding to the codewords, and
generating a normalized histogram.
24. The method of claim 18 , further comprising:
detecting a set of feature points from a plurality of training images based on the dominant direction analysis;
generating feature vectors corresponding to a portion of feature points selected arbitrarily from among the set of feature points;
clustering the feature vectors corresponding to the portion of feature points; and
generating the codebook.
25. The method of claim 18 , wherein the detecting of the plurality of feature points comprises:
detecting pixels corresponding to windows having a contrast of coherence in a dominant direction to be the plurality of feature points.
26. The method of claim 18 , wherein the
strength of coherence is based on a difference between eigenvalues associated with the feature point.
27. The method of claim 18 , further comprising:
generating a global feature vector corresponding to the input image based on a frequency characteristic extracted from the input image; and
combining the histogram and the global feature vector.
28. The method of claim 18 , further comprising:
unsupervised learning of the histogram; and
generating a mid-level feature vector.
29. An apparatus for learning a classifier, the apparatus comprising:
a detector configured to detect a plurality of feature points comprised in a plurality of training images based on a dominant direction analysis of a gradient distribution, wherein the detector is configured to detect pixels corresponding to windows having a contrast of coherence in a dominant direction to be the plurality of feature points;
a generator configured to generate a plurality of feature vectors corresponding to the plurality of feature points;
a mapper configured to map the plurality of feature vectors to codewords comprised in a given codebook, and generate a histogram corresponding to the plurality of training images; and
a learning unit configured to learn a classifier based on the histogram.
30. The apparatus of claim 29 , wherein the learning unit is configured to receive the histogram and a label of the plurality of training images to learn the classifier.
31. The apparatus of claim 29 , further comprising:
a clustering unit configured to cluster the plurality of feature vectors, and generate the codebook.
32. The apparatus of claim 29 , wherein the contrast of coherence is an indication of a difference between at least one eigenvalue associated with the detected pixel and eigenvalues of other pixels in the window.
33. The apparatus of claim 29 , wherein the generator is configured to accumulate a strength of coherence.
34. The apparatus of claim 29 , wherein the mapper is configured to map a set of the plurality of feature vectors generated from a corresponding training image from among the plurality of training images, and generates a histogram corresponding to the corresponding training image.
35. The apparatus of claim 29 , wherein the mapper is configured to
normalize values corresponding to the codewords, and
generate a normalized histogram.
36. A recognition apparatus, the apparatus comprising:
a detector configured to detect a plurality of feature points in an input image based on a dominant direction analysis of a gradient distribution;
a generator configured to generate a plurality of feature vectors corresponding to the plurality of feature points, wherein the generator is configured to accumulate a strength of coherence in a dominant direction of a local region corresponding to the plurality of feature points in order to generate the plurality of feature vectors;
a mapper configured to map the plurality of feature vectors in a given codebook, and generate a histogram corresponding to the input image; and
a recognizer configured to recognize content of the input image based on a learned parameter and the histogram.
37. The apparatus of claim 36 , wherein the content of the input image comprises
at least one of a behavior of a performer comprised in the input image; and
an object comprised in the input image.
38. The apparatus of claim 36 , wherein the learned parameter is generated by pre-learning a plurality of training images based on the dominant direction analysis of the gradient distribution.
39. The apparatus of claim 36 , wherein the detector is configured to detect pixels corresponding to windows having a contrast of coherence in a dominant direction to be the plurality of feature points.
40. The apparatus of claim 36 , wherein the strength of coherence is based on a difference between eigenvalues associated with the feature point.
41. A detection apparatus, the apparatus comprising:
a feature point detector configured to detect a plurality of feature points comprised in at least a portion of an input image based on a dominant direction analysis of a gradient distribution, wherein the feature point detector is configured to detect pixels corresponding to window having a contrast of coherence in a dominant direction to be the plurality of feature points;
a generator configured to generate a plurality of feature vectors corresponding to the plurality of feature points;
a mapper configured to map the plurality of feature vectors to codewords comprised in a given codebook, and generate a histogram corresponding to the at least one portion; and
a region detector configured to compare a reference histogram to the histogram, and detect a region corresponding to the reference histogram.
42. The apparatus of claim 41 , wherein the feature point detector is configured to detect a plurality of reference feature points comprised in a reference image based on the dominant direction analysis of the gradient distribution,
the generator is configured to generate a plurality of reference feature vectors corresponding to the plurality of reference feature points, and
the mapper is configured to map the plurality of reference feature vectors to the codewords, and generates the reference histogram corresponding to the reference image.