Multi-view image clustering techniques using binary compression
This disclosure relates to improved techniques for performing multi-view image clustering. The techniques described herein utilize machine learning functions to optimize the image clustering process. Multi-view features are extracted from a collection of images. A machine learning function is configured to jointly learn a fused binary representation that combines the multi-view features and one or more binary cluster structures that can be used to partition the images. A clustering function utilizes the fused binary representation and the one or more binary cluster structures to generate one or more image clusters based on the collection of images.
1. A system for performing multi-view image clustering comprising:
one or more computing devices comprising one or more processors and one or more non-transitory storage devices for storing instructions, wherein execution of the instructions by the one or more processors causes the one or more computing devices to:
process a collection of images to obtain multi-view features corresponding to a plurality of views;
execute a machine learning function that is configured to jointly learn a fused binary representation that combines the multi-view features and one or more binary cluster structures, wherein the machine learning function executes an algorithm that alternates between optimizing a first objective associated with the fused binary representation and a second objective associated with the one or more binary cluster structures; and
execute a clustering function that utilizes the fused binary representation and the one or more binary cluster structures to generate one or more image clusters based on the collection of images.
2. The system of claim 1 , wherein executing the machine learning function to learn a fused binary representation includes projecting the multi-view features onto a Hamming space.
3. The system of claim 2 , wherein the machine learning function is configured, at least in part, to:
identify shared information among the plurality of views;
identify view-specific information associated with the plurality of views; and
utilize the shared information and the view-specific information to derive the fused binary representation in a manner that preserves the shared information and the view-specific information associated with the plurality of views.
4. The system of claim 1 , wherein execution of the instructions by the one or more processors further causes the one or more computing devices to:
factorize the fused binary representation into one or more cluster centroid matrices and one or more clustering indicator matrices.
5. The system of claim 4 , wherein the one or more cluster centroid matrices identify one or more centroids associated with each of the one or more image clusters, and the one or more clustering indicator matrices identify distance measures from the one or more centroids.
6. The system of claim 1 , wherein execution of the instructions by the one or more processors further causes the one or more computing devices to:
normalize the multi-view features associated with each of the plurality of views.
7. The system of claim 6 , wherein normalizing the multi-view features includes transforming features corresponding to each of the plurality of views to zero-centered vectors.
8. The system of claim 1 , wherein the machine learning function executes the algorithm until an acceptable convergence is obtained with respect to optimizing the first objective associated with the fused binary representation and the second objective associated with optimizing the one or more binary cluster structures.
9. The system of claim 1 , wherein the system is integrated with a computer vision application.
10. A method for performing multi-view image clustering comprising:
processing a collection of images to obtain multi-view features corresponding to a plurality of views, the collection of images being stored on one or more non-transitory storage devices;
executing, with one or more processors, a machine learning function that is configured to jointly learn a fused binary representation that combines the multi-view features and one or more binary cluster structures, wherein the machine learning function executes an algorithm that alternates between optimizing a first objective associated with the fused binary representation and a second objective associated with the one or more binary cluster structures; and
executing, with one or more processors, a clustering function that utilizes the fused binary representation and the one or more binary cluster structures to generate one or more image clusters based on the collection of images.
11. The method of claim 10 , wherein executing the machine learning function to learn a fused binary representation includes projecting the multi-view features onto a Hamming space.
12. The method of claim 11 , wherein the machine learning function is configured, at least in part, to:
identify shared information among the plurality of views;
identify view-specific information associated with the plurality of views; and
utilize the shared information and the view-specific information to derive the fused binary representation in a manner that preserves the shared information and the view-specific information associated with the plurality of views.
13. The method of claim 10 , further comprising
factorizing the fused binary representation into one or more cluster centroid matrices and one or more clustering indicator matrices.
14. The method of claim 13 , wherein the one or more cluster centroid matrices identify one or more centroids associated with each of the one or more image clusters, and the one or more clustering indicator matrices identify distance measures from the one or more centroids.
15. The method of claim 10 , further comprising
normalizing the multi-view features associated with each of the plurality of views.
16. The method of claim 15 , wherein normalizing the multi-view features includes transforming features corresponding to each of the plurality of views to zero-centered vectors.
17. The method of claim 10 , wherein the machine learning function executes the algorithm until an acceptable convergence is obtained with respect to optimizing the first objective associated with the fused binary representation and the second objective associated with optimizing the one or more binary cluster structures.
18. A computer program product for performing multi-view image clustering, the computer program product comprising a non-transitory computer-readable medium including codes for causing a computer to:
process a collection of images to obtain multi-view features corresponding to a plurality of views;
execute a machine learning function that is configured to jointly learn a fused binary representation that combines the multi-view features and one or more binary cluster structures, wherein the machine learning function executes an algorithm that alternates between optimizing a first objective associated with the fused binary representation and a second objective associated with the one or more binary cluster structures; and
execute a clustering function that utilizes the fused binary representation and the one or more binary cluster structures to generate one or more image clusters based on the collection of images.
19. A system for performing multi-view image clustering comprising:
one or more computing devices comprising one or more processors and one or more non-transitory storage devices for storing instructions, wherein execution of the instructions by the one or more processors causes the one or more computing devices to:
process a collection of images to obtain multi-view features corresponding to a plurality of views;
execute a machine learning function that is configured to jointly learn a fused binary representation that combines the multi-view features and one or more binary cluster structures, wherein the fused binary representation is factorized into one or more cluster centroid matrices and one or more clustering indicator matrices; and
execute a clustering function that utilizes the fused binary representation and the one or more binary cluster structures to generate one or more image clusters based on the collection of images.
20. A method for performing multi-view image clustering comprising:
processing a collection of images to obtain multi-view features corresponding to a plurality of views, the collection of images being stored on one or more non-transitory storage devices;
executing, with one or more processors, a machine learning function that is configured to jointly learn a fused binary representation that combines the multi-view features and one or more binary cluster structures, wherein the fused binary representation is factorized into one or more cluster centroid matrices and one or more clustering indicator matrices; and
executing, with one or more processors, a clustering function that utilizes the fused binary representation and the one or more binary cluster structures to generate one or more image clusters based on the collection of images.
21. A computer program product for performing multi-view image clustering, the computer program product comprising a non-transitory computer-readable medium including codes for causing a computer to:
process a collection of images to obtain multi-view features corresponding to a plurality of views;
execute a machine learning function that is configured to jointly learn a fused binary representation that combines the multi-view features and one or more binary cluster structures, wherein the fused binary representation is factorized into one or more cluster centroid matrices and one or more clustering indicator matrices; and
execute a clustering function that utilizes the fused binary representation and the one or more binary cluster structures to generate one or more image clusters based on the collection of images.