Multiple stage indexing of audio content
View Patent ↗Techniques of content unification are disclosed. In some example embodiments, a computer-implemented method comprises: determining clusters based a comparison of a plurality of audio content using a first matching criteria, each cluster of the plurality of clusters comprising at least two audio content from the plurality of audio content; for each cluster of the plurality of clusters, determining a representative audio content for the cluster from the at least two audio content of the cluster; loading the corresponding representative audio content of each cluster into an index; matching the query audio content to one of the representative audio contents using a first matching criteria; determining the corresponding cluster of the matched representative audio content; and identifying a match between the query audio content and at least one of the audio content of the cluster of the matched representative audio content based on a comparison using a second matching criteria.
1. A computer-implemented method comprising:
determining, by at least one hardware processor, a plurality of clusters based on a comparison of a plurality of audio contents using a first matching criteria, each cluster of the plurality of clusters comprising at least two audio contents from the plurality of audio contents;
for each cluster of the plurality of clusters, determining, by the at least one hardware processor, a representative audio content for the cluster from the at least two audio contents of the cluster;
loading, by the at least one hardware processor, the corresponding representative audio content of each cluster in the plurality of clusters into an index, the index including the representative audio content stored in association with a hash value, the hash value based on permutations of a binary vector formed using a spectral representation of the audio content;
generating a first comparison, by the at least one hardware processor, of a query audio content to each one of the representative audio contents in the index using the first matching criteria, the first comparison including a comparison of at least one of a content duration ratio, a bit error rate at a matching location, or a length of matching positions;
matching, by the at least one hardware processor, the query audio content to one of the representative audio contents based on the generated first comparison;
determining, by the at least one hardware processor, the corresponding cluster of the matched one of the representative audio contents;
generating a second comparison, by the at least one hardware processor, of the query audio content to each one of the at least two audio contents of the determined corresponding cluster of the matched one of the representative audio contents using a second matching criteria different from the first matching criteria; and
identifying, by the at least one hardware processor, a match between the query audio content and at least one of the audio contents of the determined corresponding cluster of the matched one of the representative audio contents based on the generated second comparison of the determined corresponding cluster using the first and the second matching criteria, the match used to determine similarity of the queried audio content to the representative audio content;
wherein the determining the plurality of clusters comprises comparing fingerprint data of each of the plurality of audio contents using the first matching criteria, the comparison including a granular comparison based on a sub-cluster of each of the plurality of clusters.
2. The computer-implemented method of claim 1 , wherein the fingerprint data comprises a nano-fingerprint.
3. The computer-implemented method of claim 2 , wherein a query grid is generated for each query nano-fingerprint using a set of blocks of the query nano-fingerprint, the set of blocks having a width of two blocks, a list of hash values determined using the query grid.
4. The computer-implemented method of claim 1 , wherein the corresponding representative audio content of each cluster in the plurality of clusters is the only audio content of the cluster that is loaded into the index.
5. The computer-implemented method of claim 1 , wherein the matching of the query audio content to one of the representative audio contents comprises comparing fingerprint data of the query audio content with fingerprint data of each of the representative audio contents in the index using the first matching criteria.
6. The computer-implemented method of claim 5 , wherein the fingerprint data comprises a nano-fingerprint.
7. The computer-implemented method of claim 1 , wherein comparing the query audio content to each one of the at least two audio contents of the determined corresponding cluster comprises comparing fingerprint data of the query audio content with fingerprint data of each one of the at least two audio contents of the determined corresponding cluster using the second matching criteria.
8. The computer-implemented method of claim 7 , wherein the fingerprint data comprises a micro-fingerprint.
9. The computer-implemented method of claim 7 , wherein the fingerprint data comprises a full fingerprint.
10. The computer-implemented method of claim 1 , wherein each one of the plurality of audio contents comprises a song.
11. A system comprising:
at least one processor; and
a non-transitory computer-readable medium storing executable instructions that, when executed, cause the at least one processor to perform operations comprising:
determining a plurality of clusters based on a comparison of a plurality of audio contents using a first matching criteria, each cluster of the plurality of clusters comprising at least two audio contents from the plurality of audio contents;
for each cluster of the plurality of clusters, determining a representative audio content for the cluster from the at least two audio contents of the cluster;
loading the corresponding representative audio content of each cluster in the plurality of clusters into an index, the index including the representative audio content stored in association with a hash value, the hash value based on permutations of a binary vector formed using a spectral representation of the audio content;
generating a first comparison of a query audio content to each one of the representative audio contents in the index using the first matching criteria, the first comparison including at a comparison of at least one of a content duration ratio, a bit error rate at a matching location, or a length of matching positions;
matching the query audio content to one of the representative audio contents based on the generated first comparison;
determining the corresponding cluster of the matched one of the representative audio contents;
generating a second comparison of the query audio content to each one of the at least two audio contents of the determined corresponding cluster of the matched one of the representative audio contents using a second matching criteria different from the first matching criteria; and
identifying a match between the query audio content and at least one of the audio contents of the determined corresponding cluster of the matched one of the representative audio contents based on the generated second comparison of the determined corresponding cluster using the first and the second matching criteria, the match used to determine similarity of the queried audio content to the representative audio content;
wherein the determining the plurality of clusters comprises comparing fingerprint data of each of the plurality of audio contents using the first matching criteria, the comparison including a granular comparison based on a sub-cluster of each of the plurality of clusters.
12. The system of claim 11 , wherein the fingerprint data comprises a nano-fingerprint.
13. The system of claim 11 , wherein the corresponding representative audio content of each cluster in the plurality of clusters is the only audio content of the cluster that is loaded into the index.
14. The system of claim 11 , wherein the matching of the query audio content to one of the representative audio contents comprises comparing fingerprint data of the query audio content with fingerprint data of each of the representative audio contents in the index using the first matching criteria.
15. The system of claim 14 , wherein the fingerprint data comprises a nano-fingerprint.
16. The system of claim 11 , wherein comparing the query audio content to each one of the at least two audio contents of the determined corresponding cluster comprises comparing fingerprint data of the query audio content with fingerprint data of each one of the at least two audio contents of the determined corresponding cluster using the second matching criteria.
17. The system of claim 16 , wherein the fingerprint data comprises a micro-fingerprint.
18. The system of claim 16 , wherein the fingerprint data comprises a full fingerprint.
19. A non-transitory machine-readable storage medium, tangibly embodying a set of instructions that, when executed by at least one processor, causes the at least one processor to perform operations comprising:
determining a plurality of clusters based on a comparison of a plurality of audio contents using a first matching criteria, each cluster of the plurality of clusters comprising at least two audio contents from the plurality of audio contents;
for each cluster of the plurality of clusters, determining a representative audio content for the cluster from the at least two audio contents of the cluster;
loading the corresponding representative audio content of each cluster in the plurality of clusters into an index, the index including the representative audio content stored in association with a hash value, the hash value based on permutations of a binary vector formed using a spectral representation of the audio content;
generating a first comparison of a query audio content to each one of the representative audio contents in the index using the first matching criteria, the first comparison including a comparison of at least one of a content duration ratio, a bit error rate at a matching location, or a length of matching positions;
matching the query audio content to one of the representative audio contents based on the generated first comparison;
determining the corresponding cluster of the matched one of the representative audio contents;
generating a second comparison of the query audio content to each one of the at least two audio contents of the determined corresponding cluster of the matched one of the representative audio contents using a second matching criteria different from the first matching criteria; and
identifying a match between the query audio content and at least one of the audio contents of the determined corresponding cluster of the matched one of the representative audio contents based on the generated second comparison of the determined corresponding cluster using the first and the second matching criteria, the match used to determine similarity of the queried audio content to the representative audio content;
wherein the determining the plurality of clusters comprises comparing fingerprint data of each of the plurality of audio contents using the first matching criteria, the comparison including a granular comparison based on a sub-cluster of each of the plurality of clusters.