Automatic identification of sound recordings
View Patent ↗Copies of original sound recordings are identified by extracting features from the copy, creating a vector of those features, and comparing that vector against a database of vectors. Identification can be performed for copies of sound recordings that have been subjected to compression and other manipulation such that they are not exact replicas of the original. Computational efficiency permits many hundreds of queries to be serviced at the same time. The vectors may be less than 100 bytes, so that many millions of vectors can be stored on a portable device.
1. A computer-implemented method comprising:
identifying a leader fingerprint as a cluster leader to represent a subset of fingerprints included in a cluster of fingerprints forming part of a plurality of fingerprints extracted from multiple media files;
calculating, by a processor of a media processing machine, a mathematical distance between a query fingerprint and the leader fingerprint, wherein the mathematical distance between the query fingerprint and the leader fingerprint comprises a first mathematical distance; and
determining that the query fingerprint corresponds to the cluster of fingerprints, the determining based on the mathematical distance, wherein the identifying comprises calculating a second mathematical distance between the query fingerprint and an additional fingerprint of the cluster.
2. The computer-implemented method of claim 1 , wherein the first mathematical distance is determined using a first distance measurement function, and wherein the second mathematical distance is determined using a second distance measurement function.
3. The computer-implemented method of claim 1 , wherein the query fingerprint comprises a first vector of first values and the additional fingerprint comprises a second vector of second values, and wherein the identifying comprises:
calculating a sum of absolute values comprising an absolute difference between at least one of the first values and at least one of the second values.
4. The computer-implemented method of claim 1 , further comprising:
partitioning the plurality of fingerprints into multiple clusters, the multiple clusters including the cluster comprising the subset.
5. The computer-implemented method of claim 4 , wherein the partitioning comprises:
generating a hierarchy associated with the multiple clusters.
6. The computer-implemented method of claim 1 , wherein the cluster of fingerprints includes a sub-cluster of fingerprints, further comprising:
determining that the query fingerprint corresponds to the sub-cluster of fingerprints.
7. The computer-implemented method of claim 1 , wherein the leader fingerprint comprises a first vector of first values and the query fingerprint comprises a second vector of second values, and wherein the mathematical distance comprises a vector difference between the first and the second vectors.
8. The computer-implemented method of claim 1 , wherein each fingerprint in the cluster of fingerprints comprises a vector of values, wherein the determining is based on a threshold comparison with the mathematical distance, further comprising:
generating the threshold based on an aggregate of values corresponding to the vectors of values in the cluster of fingerprints.
9. The computer-implemented method of claim 8 , wherein the generating comprises:
calculating a standard deviation based on the aggregate of values corresponding to the vectors of values in the cluster of fingerprints.
10. The computer-implemented method of claim 7 , wherein the determining comprises:
calculating a maximum of absolute values, each of the absolute values comprising an absolute difference between at least one of the first values and at least one of the second values.
11. The computer-implemented method of claim 7 , wherein the determining comprises:
calculating a sum of absolute values comprising an absolute difference between at least one of the first values and at least one of the second values.
12. The computer-implemented method of claim 7 , wherein the determining comprises:
calculating a difference between an arithmetic mean and a geometric mean, the arithmetic mean and the geometric mean derived from the first values and the second values.
13. The computer-implemented method of claim 7 , wherein the determining comprises:
calculating logarithms of an arithmetic mean and of a geometric mean, the arithmetic mean and the geometric mean derived from the first values and the second values.
14. The computer-implemented method of claim 7 , wherein the determining comprises:
calculating a square root of a sum of squares of absolute values, each of the absolute values comprising an absolute difference between at least one of the first values and at least one of the second values.
15. A system comprising:
a storage unit to store a cluster of fingerprints, the cluster comprising a subset of a plurality of fingerprints extracted from multiple media files; and
a processor communicatively coupled to the storage unit, the processor to access at least some of the cluster of fingerprints, to identify a leader fingerprint of the cluster as a cluster leader to represent the subset, to calculate a mathematical distance between a query fingerprint and the leader fingerprint, and to determine that the query fingerprint corresponds to the cluster of fingerprints, the determining based on the mathematical distance, wherein the mathematical distance between the query fingerprint and the leader fingerprint comprises a first mathematical distance, wherein the processor is to calculate a second mathematical distance between the query fingerprint and an additional fingerprint of the cluster, wherein the first mathematical distance is determined using a first distance measurement function, and wherein the second mathematical distance is determined using a second distance measurement function.
16. The system of claim 15 , wherein the processor is to partition the plurality of fingerprints into multiple clusters, the multiple clusters including the cluster comprising the subset.
17. A system comprising:
a storage unit to store a cluster of fingerprints, the cluster comprising a subset of a plurality of fingerprints extracted from multiple media files; and
a processor communicatively coupled to the storage unit, the processor to access at least some of the cluster of fingerprints, to identify a leader fingerprint of the cluster as a cluster leader to represent the subset, to calculate a mathematical distance between a query fingerprint and the leader fingerprint, and to determine that the query fingerprint corresponds to the cluster of fingerprints, the determining based on the mathematical distance, wherein the leader fingerprint comprises a first vector of first values, the query fingerprint comprises a second vector of second values, the mathematical distance comprises a vector difference between the first and the second vectors, and the processor is to calculate a sum of absolute values comprising an absolute difference between at least one of the first values and at least one of the second values.
18. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform a method comprising:
identifying a leader fingerprint as a cluster leader to represent a subset of fingerprints included in a cluster of fingerprints forming part of a plurality of fingerprints extracted from multiple media files;
calculating a mathematical distance between a query fingerprint and the leader fingerprint, wherein the mathematical distance between the query fingerprint and the leader fingerprint comprises a first mathematical distance; and
determining that the query fingerprint corresponds to the cluster of fingerprints, the determining based on the mathematical distance, wherein the identifying comprises calculating a second mathematical distance between the query fingerprint and an additional fingerprint of the cluster.
19. A system comprising:
a storage unit to store first and second reference fingerprints; and
a processor communicatively coupled to the storage unit, the processor to extract first and second candidate fingerprints from a portion of an unidentified recording, the first candidate fingerprint having a candidate frequency variation from an original recording, the second candidate fingerprint having a candidate amplitude variation from the original recording, wherein the processor is to compare the first candidate fingerprint with the first reference fingerprint and the second candidate fingerprint with the second reference fingerprint to find a match for the unidentified recording among reference recordings, the first reference fingerprint having a reference frequency variation from the original recording and the second reference fingerprint having a reference amplitude variation from the original recording.
20. The system of claim 19 , wherein the processor is to combine first and second results to determine whether the first and second reference fingerprints correspond to the first and second candidate fingerprints, the first result arising from the comparison of the first candidate fingerprint with the first reference fingerprint and the second result arising from the comparison of the second candidate fingerprint with the second reference fingerprint.
21. The system of claim 19 , wherein the portion of the unidentified recording has a duration of less than five seconds.
22. A system comprising:
a storage unit to store an audio sample of an unidentified recording; and
a processor communicatively coupled to the storage unit, the processor to determine overlapping audio frames from the audio sample, to extract frequency spectra of the audio sample corresponding to at least one of the overlapping audio frames, to modify the frequency spectra using an ear model transfer function to generate weighted frequency spectra, to generate a matrix of power values based on the weighted frequency spectra, the matrix having a time dimension and a frequency dimension, to transform a normalized version of the matrix of power values to a log scale set of transformed values, and to generate a candidate fingerprint for the audio sample based on the transformed values.
23. The system of claim 22 , wherein the processor is to extract the frequency spectra by applying a discrete cosine transform to the audio sample, wherein the overlapping audio frames have a substantially uniform duration, and wherein at least some of the overlapping audio frames overlap by about half of the substantially uniform duration.
24. The system of claim 22 , wherein the storage unit is to store a reference fingerprint of a reference recording, and wherein the processor is to identify the unidentified recording based on a comparison of the candidate fingerprint to the reference fingerprint.
25. A system comprising:
a storage unit to store a plurality of reference fingerprints; and
a processor communicatively coupled to the storage unit, the processor to partition a portion of an unidentified recording into time-frequency regions covering at least one of at least three ranges of time and one of at least three ranges of frequency, to weight the time-frequency regions to produce weighted time-frequency regions, to compute a candidate value of a candidate fingerprint using the weighted time-frequency regions, and to determine that the candidate value matches a reference value corresponding to one of the plurality of reference fingerprints.
26. The system of claim 25 , wherein the processor is to weight the time-frequency regions by emphasizing a particular time-frequency region covering at least one of a substantially central range of time or a substantially central range of frequency included in the time-frequency regions.
27. The system of claim 25 , wherein the processor is to identify the unidentified recording based on a determination that the candidate value matches the reference value.
28. A system comprising:
a storage unit to store a reference recording; and
a processor communicatively coupled to the storage unit, the processor to extract a principal fingerprint from a principal portion of the reference recording, to extract auxiliary fingerprints from auxiliary portions of the reference recording, the auxiliary portions being separated in time by a substantially regular interval, to compute reference parameters based on the principal fingerprint and on the auxiliary fingerprints, to generate a reference fingerprint of the reference recording, the reference fingerprint including the principal fingerprint and the reference parameters, and to store the reference fingerprint, wherein the processor is to extract an initial fingerprint from an initial portion an unknown recording, to extract subsequent fingerprints from subsequent portions of the unknown recording, the subsequent portions following the initial portion in time and being separated by the substantially regular time interval, to compute candidate parameters based on the initial fingerprint and on the subsequent fingerprints, to generate a candidate fingerprint of the unknown recording, the candidate fingerprint including the initial fingerprint and the candidate parameters, and to identify the unknown recording based on a correlation between the reference fingerprint and the candidate fingerprint.
29. The system of claim 28 , wherein the reference parameters comprise reference distance measures from the principal fingerprint to each of the auxiliary fingerprints.
30. The system of claim 28 , wherein the reference parameters comprises reference power differences between the principal portion and each of the auxiliary portions.
31. The system of claim 28 , wherein the candidate parameters comprise candidate distance measures from the initial candidate fingerprint to the subsequent fingerprints.
32. The system of claim 28 , wherein the candidate parameters comprise candidate power differences between the initial portion and the subsequent portions.