Image classification employing image vectors compressed using vector quantization
View Patent ↗Local descriptors are extracted from an image. An image vector is generated having vector elements indicative of parameters of mixture model components of a mixture model representing the extracted local descriptors. The image vector is compressed using a vector quantization algorithm to generate a compressed image vector. Optionally, the compressing comprises splitting the image vector into a plurality of sub-vectors each including at least two vector elements, compressing each sub-vector independently using the vector quantization algorithm, and concatenating the compressed sub-vectors to generate the compressed image vector. Optionally, each sub-vector includes only vector elements indicative of parameters of a single mixture model component, and any sparse sub-vector whose vector elements are indicative of parameters of a mixture model component that does not represent any of the extracted local descriptors is not compressed.
1. A non-transitory storage medium storing instructions executable by a processor to perform a method comprising:
extracting local descriptors from an image;
generating an image vector having vector elements indicative of parameters of mixture model components of a mixture model representing the extracted local descriptors; and
compressing the image vector using a vector quantization algorithm to generate a compressed image vector by operations including splitting the image vector into a plurality of sub-vectors wherein each sub-vector includes at least two vector elements and each sub-vector includes only vector elements indicative of parameters of a single mixture model component, compressing each sub-vector independently using the vector quantization algorithm, and concatenating the compressed sub-vectors to generate the compressed image vector.
2. The non-transitory storage medium as set forth in claim 1 , wherein the mixture model comprises a Gaussian mixture model and the vector elements are indicative of at least (i) mean parameters and (ii) variance or covariance parameters of Gaussian mixture model components.
3. The non-transitory storage medium as set forth in claim 2 , wherein the vector elements are not indicative of weight parameters of the Gaussian mixture model components.
4. The non-transitory storage medium as set forth in claim 1 , wherein the image vector is a Fisher vector comprising a gradient of the mixture model or of a composition of the mixture model in a vector space defined by parameters of the mixture model.
5. The non-transitory storage medium as set forth in claim 1 , wherein:
the compressing of each sub-vector does not compress any sparse sub-vector whose vector elements are indicative of parameters of a mixture model component that does not represent any of the extracted local descriptors.
6. The non-transitory storage medium as set forth in claim 1 , wherein the method further comprises:
performing the extracting, generating, and splitting operations on a corpus of codebook training images to generate codebook training sub-vector sets; and
learning a codebook for each codebook training sub-vector set, the learned codebooks being used in the compressing.
7. The non-transitory storage medium as set forth in claim 1 , wherein the method further comprises:
performing the extracting, generating, and compressing operations on a corpus of training images to generate compressed training vectors; and
iteratively learning a classifier by iteratively repeating the operations of (i) uncompressing one or more of the compressed training vectors and (ii) performing an update learning operation using the one or more uncompressed training vectors to update the classifier.
8. The non-transitory storage medium as set forth in claim 7 , wherein:
the operation (ii) performs the update learning on a pool of uncompressed training vectors populated by the operation (i), and
the iterative learning further iteratively repeats an operation (iii) comprising selectively purging uncompressed training vectors from the pool of uncompressed training vectors.
9. The non-transitory storage medium as set forth in claim 7 , wherein the method further comprises:
classifying an input image by performing the extracting and generating operations on the input image to generate an input image vector and applying the learned classifier to the input image vector.
10. The non-transitory storage medium as set forth in claim 9 , wherein the classifying does not include performing the compressing operation on the input image vector.
11. A method comprising:
extracting local descriptors from an image;
generating an image vector having vector elements indicative of parameters of mixture model components of a mixture model representing the extracted local descriptors;
splitting the image vector into a plurality of sub-vectors wherein each sub-vector includes at least two vector elements and each sub-vector includes only vector elements indicative of parameters of a single mixture model component;
compressing each sub-vector independently using a vector quantization algorithm; and
concatenating the compressed sub-vectors to generate a compressed image vector representing the image;
wherein the extracting, generating, splitting, compressing, and concatenating are performed by a digital processor.
12. The method as set forth in claim 11 , wherein the mixture model comprises a Gaussian mixture model and the vector elements are indicative of at least (i) mean parameters and (ii) variance or covariance parameters of Gaussian mixture model components and the image vector comprises a Fisher vector.
13. The method as set forth in claim 11 , wherein:
the compressing and concatenating operations do not compress and concatenate any sub-vector whose vector elements are indicative of parameters of a mixture model component that does not represent any of the extracted local descriptors.
14. An apparatus comprising:
a digital processor configured to perform:
a method of generating a corpus of compressed training image vectors from a corpus of training images including:
extracting local descriptors from a training image,
generating an image vector having vector elements indicative of parameters of mixture model components of a mixture model representing the extracted local descriptors,
compressing the image vector, and
repeating the extracting, generating, and compressing for a plurality of training images of the corpus of training images;
wherein the compressing of the image vector includes:
splitting the image vector into a plurality of sub-vectors wherein each sub-vector includes at least two vector elements and each sub-vector includes only vector elements indicative of parameters of a single mixture model component;
compressing each sub-vector independently using the vector quantization algorithm; and
concatenating the compressed sub-vectors to generate the compressed image vector.
15. The apparatus as set forth in claim 14 wherein the digital processor is further configured to perform a learning method that iteratively learns a classifier by iteratively repeating operations including:
uncompressing a sub-set of the compressed training image vectors to generate a pool of uncompressed training vectors,
performing an update learning operation using the pool of uncompressed training vectors to update the classifier, and
selectively purging uncompressed training vectors from the pool of uncompressed training vectors during the iterative repeating to keep the pool of uncompressed training vectors within a size limit.
16. The apparatus as set forth in claim 14 , wherein the image vector is compressed using a vector quantization algorithm.
17. The apparatus as set forth in claim 15 , wherein:
the digital processor is further configured to perform:
an image classification method including:
extracting local descriptors from an input image,
generating an input image vector having vector elements indicative of parameters of mixture model components of a mixture model representing the local descriptors extracted from the input image, and
applying the classifier learned by the learning method to the input image vector to generate a classification for the input image.
18. The apparatus as set forth in claim 17 , wherein the image classification method does not include compressing the generated input image vector.
19. The apparatus as set forth in claim 14 , wherein the compressing of each sub-vector does not compress any sparse sub-vector whose vector elements are indicative of parameters of a mixture model component that does not represent any of the extracted local descriptors.