System and method for cascading image clustering using distribution over auto-generated labels
Embodiments of the present invention provide a system that can be used to classify a feedback image in a user review into a semantically meaningful class. During operation, the system analyzes the captions of feedback images in a set of user reviews and determines a set of training labels from the captions. The system then trains an image classifier with the set of training labels and the feedback images. Subsequently, the system generates a signature for a respective feedback image in a new set of user reviews using the image classifier. The signature indicates a likelihood of the image matching a respective label in the set of training labels. Based on the signature, the system can allocate the image to an image cluster.
1. A computer-implemented method for facilitating cascading image clustering, the method comprising:
generating, by a computer, a set of training labels from captions of feedback images in a first set of user reviews;
training an image classifier with the set of training labels and the feedback images in the first set of user reviews;
generating a first signature for a first image in feedback images in a second set of user reviews using the image classifier, wherein the first signature indicates a likelihood of the first image matching a respective label in the set of training labels; and
allocating the first image to a first image cluster based on the first signature without relying on labels in the set of training labels.
2. The method of claim 1 , wherein allocating the first image to the first image cluster comprises:
determining a cosine distance between the first signature and signatures for rest of the feedback images in the second set of user reviews; and
determining that the cosine distance is below a threshold.
3. The method of claim 1 , wherein the first image cluster further comprises neighbor images of the first image; and
wherein the method further comprises determining whether to merge the first image cluster with a second image cluster based on an average distance between a respective image of the first image cluster and a respective image of the second image cluster.
4. The method of claim 1 , wherein generating the set of training labels comprises:
identifying a set of phrases frequently appearing in the captions of feedback images in the first set of user reviews by applying text analysis on the captions of feedback images; and
allocating a predetermined number of most frequent phrases in the set as training labels.
5. The method of claim 4 , wherein the set of phrases includes one or more of: a noun, an adjective, and a noun-adjective pair.
6. The method of claim 1 , wherein the likelihood is expressed as a respective probability of the first image matching a respective label in the set of training labels.
7. The method of claim 1 , wherein the first image cluster corresponds to a topic not represented in the set of training labels.
8. The method of claim 1 , further comprising storing the first signature in association with the first image.
9. A computer system for facilitating cascading image clustering, the system comprising:
a processor; and
a storage device storing instructions that when executed by the processor cause the processor to perform a method, the method comprising:
generating a set of training labels from captions of feedback images in a first set of user reviews;
training an image classifier with the set of training labels and the feedback images in the first set of user reviews;
generating a first signature for a first image in feedback images in a second set of user reviews using the image classifier, wherein the first signature indicates a likelihood of the first image matching a respective label in the set of training labels; and
allocating the first image to a first image cluster based on the first signature without relying on labels in the set of training labels.
10. The computer system of claim 9 , wherein allocating the first image to the first image cluster comprises:
determining a cosine distance between the first signature and signatures for rest of the feedback images in the second set of user reviews; and
determining that the cosine distance is below a threshold.
11. The computer system of claim 9 , wherein the first image cluster further comprises neighbor images of the first image; and
wherein the method further comprises determining whether to merge the first image cluster with a second image cluster based on an average distance between a respective image of the first image cluster and a respective image of the second image cluster.
12. The computer system of claim 9 , wherein generating the set of training labels comprises:
identifying a set of phrases frequently appearing in the captions of feedback images in the first set of user reviews by applying text analysis on the captions of feedback images; and
allocating a predetermined number of most frequent phrases in the set as training labels.
13. The computer system of claim 12 , wherein the set of phrases includes one or more of: a noun, an adjective, and a noun-adjective pair.
14. The computer system of claim 9 , wherein the likelihood is expressed as a respective probability of the first image matching a respective label in the set of training labels.
15. The computer system of claim 9 , wherein the first image cluster corresponds to a topic not represented in the set of training labels.
16. The computer system of claim 9 , wherein the method further comprises storing the first signature in association with the first image.
17. The computer system of claim 9 , wherein the likelihood is expressed as a respective probability of the first image matching a respective label in the set of training labels.
18. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:
generating a set of training labels from captions of feedback images in a first set of user reviews;
training an image classifier with the set of training labels and the feedback images in the first set of user reviews;
generating a first signature for a first image in feedback images in a second set of user reviews using the image classifier, wherein the first signature indicates a likelihood of the first image matching a respective label in the set of training labels; and
allocating the first image to a first image cluster based on the first signature without relying on labels in the set of training labels.
19. The computer-readable storage medium of claim 18 , wherein allocating the first image to the first image cluster comprises:
determining a cosine distance between the first signature and signatures for rest of the feedback images in the second set of user reviews; and
determining that the cosine distance is below a threshold.
20. The computer-readable storage medium of claim 18 , wherein generating the set of training labels comprises:
identifying a set of phrases frequently appearing in the captions of feedback images in the first set of user reviews by applying text analysis on the captions of feedback images; and
allocating a predetermined number of most frequent phrases in the set as training labels.