System and method for signature-based unsupervised clustering of data elements
A method and system for signature-based unsupervised clustering of data elements. The method comprises receiving a plurality of clusters; generating a triangular matrix respective of the clusters; generating a signature for each of the clusters; generating a match score between each of two different clusters; storing the match score in a cell of the triangular matrix corresponding to the two clusters; determining whether any of the match scores is above a predefined threshold value; clustering every two clusters that are determined to have a score above a predetermined threshold; and repeating the generation of a triangular matrix respective of the clusters until a single cluster is reached. The system comprises an interface; a processor; a memory for storing at least one cluster; and a memory coupled to the processor, the memory containing instructions that, when executed by the processor, configure the system to perform the steps of the method.
1. A computerized method for signature-based unsupervised clustering of data elements, comprising:
receiving a plurality of clusters, each cluster comprising at least a data element;
generating an upper triangular matrix respective of the clusters;
generating a signature for each of the clusters, wherein a signature is generated from multiple patches of a multimedia data element, wherein multiple patches are of random length and random position within the multimedia data element;
generating a match score between each of two different clusters;
storing the match score in a cell of the upper triangular matrix corresponding to the two clusters;
determining whether any of the match scores is above a predefined threshold value;
clustering every two clusters that are determined to have a score above a predetermined threshold; and
repeating the generation of an upper triangular matrix respective of the clusters until a single cluster is reached, wherein the clustering on the respective generated signatures creates clusters that include a collection of signatures respective of the multimedia data elements.
2. The computerized method of claim 1 , further comprising:
determining whether a single cluster cannot be reached; and
stopping the generation of the upper triangular matrix upon determination that a single cluster cannot be reached.
3. The computerized method of claim 1 , wherein a new threshold value is utilized for each iteration.
4. The computerized method of claim 3 , wherein the clustering score is determined to be below the threshold value if two clusters of the previously generated clusters include at least one overlapping data element or if a multimedia data element is matched to itself.
5. The computerized method of claim 2 , wherein the single cluster cannot be reached when no two multimedia data elements have a clustering score above the threshold value or when no two previously clustered elements have a clustering score above the threshold value.
6. The computerized method of claim 1 , wherein each of the data elements is at least one of: audio, image, video, a video frame, fragment of audio, fragment of image, fragment of video, a fragment of a video frame, and an image of a signal.
7. The computerized method of claim 6 , wherein the image of a signal is at least one of: a medical signal, a geophysical signal, a subsonic signal, a supersonic signal, an electromagnetic signal, and an infrared signal.
8. The computerized method of claim 1 , wherein each signature is generated by a plurality of computational cores enabled to receive a plurality of multimedia data elements, each computational core having properties that are independent from properties of the other the computational cores.
9. The computerized method of claim 8 , wherein each computational core generates a respective first signature element and a respective second signature element responsive to a data element, the first signature element being a robust signature.
10. The computerized method of claim 8 , further comprising:
setting the properties of each of the plurality of computational cores to have at least some statistical independence from other of the plurality of computational cores, the properties being set independently of each other core.
11. A non-transitory computer readable medium containing instructions that, when executed, perform the method of claim 1 .
12. An apparatus for performing unsupervised clustering of data elements, comprising:
an interface for allowing access to a plurality of data elements;
at least one processing unit;
a storage unit for storing at least one cluster of data elements; and
a memory coupled to the at least one processing unit, the memory containing instructions that, when executed by the at least one processing unit, configure the apparatus to:
receive a plurality of clusters, each cluster comprising a data element;
generate an upper triangular matrix respective of the clusters, the upper triangular matrix stored in the memory;
generate a signature for each of the clusters, wherein a signature is generated from multiple patches of a multimedia data element, wherein multiple patches are of random length and random position within the multimedia data element;
generate a match score between each of two different clusters;
store the match score in a cell of the upper triangular matrix corresponding to the two clusters of the match score;
determine whether any of the match scores is above a predefined threshold value;
cluster every two clusters that are determined to have a score above a predetermined threshold; and
repeat the generation of the upper triangular matrix respective of the clusters until a single cluster is reached, wherein the clustering on the respective generated signatures creates clusters that include a collection of signatures respective of the multimedia data elements.
13. The apparatus of claim 1 , further comprising:
a signature generator configured to generate a signature for each of the clusters responsive to the at least one processing unit.
14. The apparatus of claim 13 , wherein the signature generator further comprises:
a plurality of computational cores enabled to receive a plurality of multimedia data elements, each computational core having properties that are independent from properties of other of the computational cores.
15. The apparatus of claim 14 , wherein each computational core generates a respective first signature element and a respective second signature element responsive to a data element, the first signature element being a robust signature.
16. The apparatus of claim 14 , wherein the properties of each of the plurality of computational cores are set independently of each other core to have at least some statistical independence from other of the plurality of computational cores.
17. The apparatus of claim 14 , wherein the apparatus is further configured to:
determine whether a single cluster cannot be reached; and
stop the generation of the new cluster upon determination that a single cluster cannot be reached.
18. The apparatus of claim 14 , wherein a new threshold value is utilized at each iteration.
19. The apparatus of claim 14 , wherein the clustering score is determined to be below the threshold if two clusters of the previously generated clusters include at least one overlapping multimedia data element or a multimedia data element is matched to itself.
20. The apparatus of claim 2 , wherein the clustering score is determined to be below the threshold if two clusters of the previously generated clusters include at least one overlapping multimedia data element or a multimedia data element is matched to itself.
21. The apparatus of claim 16 , wherein each of the data elements is at least one of: audio, image, video, a video frame, fragment of audio, fragment of image, fragment of video, a fragment of a video frame, and an image of a signal.
22. The apparatus of claim 21 , wherein the image of a signal is at least one of: a medical signal, a geophysical signal, a subsonic signal, a supersonic signal, an electromagnetic signal, and an infrared signal.