Granted Patent B2

US 8,452,586 · App. 12/629,821 · Granted May 28, 2013

Abstract

Claims (105)

1. A method of creating a resilient fingerprint of a reference sound or a query sound, the method including:

using a processing device, repeatedly characterizing loudness of sound components grouped in filter banks of sample frames and residing in a computer readable memory, wherein the filter banks have a narrower frequency range and have closer mid-frequency spacing in frequencies between about 750 hz and 1450 hz than above 1500 hz or below 500 hz;

flagging loudness peaks among the filter banks that are above cross-frequency and time masking thresholds in the sample frames, wherein the cross-frequency masking threshold for a particular frame is set relative to filter band characterizations of one or more masking sounds and the time masking threshold for successive frames is set relative to the filter band characterizations of the masking sounds; and

coding the flagged loudness peaks in relation to the cross-frequency and time masking thresholds to create a frame-by-frame, filter bank-by-filter bank characteristic matrix.

2. The method of claim 1 , whereby the flagging of loudness peaks above the cross-frequency and time masking thresholds avoids including in the frame-by-frame, filter bank-by-filter bank characteristic matrix of particular sound components that are likely to be eliminated from a frame sample by an auditory perception codec.

3. The method of claim 1 , further including flagging one or more filter banks adjoining a particular filter bank at which a loudness peak was flagged.

4. The method of claim 1 , further including flagging three or more peaks, centered at about the particular filter bank.

5. The method of claim 1 , further including flagging five or more peaks, centered at about the particular filter bank.

6. The method of claim 1 , wherein the flagging loudness peaks and flagging adjoining filter banks effectively broadens frequency widths of the loudness peaks.

7. The method of claim 1 , wherein the coding includes:

first coding the flagged loudness peaks within the particular frame against the cross-frequency masking threshold by first bands of relative loudness;

second coding the flagged loudness peaks within the particular frame against the time masking threshold by second bands of relative loudness;

combining the first coding and second coding to set filter bank-by-filter bank values in the characteristic matrix for the particular frame; and

repeating the first and second coding and the combining for numerous sample frames of the sound.

8. The method of claim 7 , wherein the values in the characteristic matrix are Boolean.

9. The method of claim 1 , further including:

creating reference characteristic matrices for numerous reference sounds and at least one query characteristic matrix for a query sound;

comparing alignments of the query characteristic matrix to at least some of the reference characteristic matrices, including identifying filter bank-by-filter bank positive peak matches, negative peak matches, peak in query but not in reference mismatches, and peak in reference but not in query mismatches;

frame-by-frame, deriving composite scores across filter banks that distinctly weight and combine the positive peak matches, the negative peak matches, the peak in query but not in reference mismatches and peak in reference but not in query mismatches;

combining the frame-by-frame composite scores into query scores for the alignments and selecting one or more best alignments of the query characteristic matrix to the reference characteristic matrices; and

organizing the composite scores for the best alignments to identify likely query-to-reference characteristic matrix matches.

10. Repeating the method of claim 9 for multiple versions of the query characteristic matrix, prepared from the query sound using versions of sample frames that have starting times within the query sound that are offset in time between the sample frame versions.

11. The method of claim 9 , further including comparing the composite scores for the best alignments and identifying as a true match one of the composite scores that is significantly better than any others of the composite scores, only if it is significantly better.

12. The method of claim 9 , further including comparing the composite scores for the best alignments and identifying as identical true matches a plurality of the composite scores that match each other and that are significantly better than any others of the composite scores, only if it is significantly better.

13. A method for accurately searching music references to find a music reference which matches a query including:

using a processing device, creating frame-by-frame, filter bank-by-filter bank reference characteristic matrices for numerous reference sounds and at least one frame-by-frame, filter bank-by-filter bank query characteristic matrix for a query sound;

comparing alignments of the query characteristic matrix to at least some of the reference characteristic matrices, including identifying filter bank-by-filter bank positive peak matches, negative peak matches, peak in query but not in reference mismatches, and peak in reference but not in query mismatches;

frame-by-frame, deriving a plurality of composite scores across filter banks that distinctly weight and combine the positive peak matches, the negative peak matches, the peak in query but not in reference characteristic mismatches and peak in reference but not in query mismatches;

combining the frame-by-frame composite scores into query scores for the alignments and selecting one or more best alignments of the query characteristic matrix to the reference characteristic matrices; and

organizing the composite scores for the best alignments to identify likely query-to-reference characteristic matrix matches.

14. Repeating the method of claim 13 for multiple versions of the query characteristic matrix, prepared from the query sound using versions of sample frames that have starting times within the query sound that are offset in time between the sample frame versions.

15. The method of claim 13 , further including comparing the composite scores for the best alignments and identifying as a true match one of the composite scores that is significantly better than any others of the composite scores, only if it is significantly better.

16. The method of claim 13 , further including comparing the composite scores for the best alignments and identifying as identical true matches a plurality of the composite scores that match each other and that are significantly better than any others of the composite scores, only if it is significantly better.

17. The method of claim 13 , wherein the scoring further includes:

first coding the flagged loudness peaks within a particular frame against a cross-frequency masking threshold by first bands of relative loudness;

second coding the flagged loudness peaks within the particular frame against a time masking threshold by second bands of relative loudness;

combining the first coding and second coding to set filter bank-by-filter bank characteristic matrix flags for the particular frame; and

repeating the first and second coding and the combining for numerous sample frames of the sound.

18. A device that creates a resilient fingerprint of a reference sound or a query sound, including:

at least one processor configured to carry out actions including:

using a processing device, repeatedly characterizing loudness of sound components grouped in filter banks of sample frames and residing in a computer readable memory, wherein the filter banks have a narrower frequency range and have closer mid-frequency spacing in frequencies between about 750 hz and 1450 hz than above 1500 hz or below 500 hz;

flagging loudness peaks among the filter banks that are above cross-frequency and time masking thresholds in the sample frames, wherein the cross-frequency masking threshold for a particular frame is set relative to filter band characterizations of one or more masking sounds and the time masking threshold for successive frames is set relative to the filter band characterizations of the masking sounds; and

coding the flagged loudness peaks in relation to the cross-frequency and time masking thresholds to create a frame-by-frame, filter bank-by-filter bank characteristic matrix.

19. The device of claim 18 , wherein the flagging of loudness peaks above the cross-frequency and time masking thresholds avoids including in the frame-by-frame, filter bank-by-filter bank characteristic matrix of particular sound components that are likely to be eliminated from a frame sample by an auditory perception codec.

20. The device of claim 18 , wherein the processor is further configured to carry out actions including flagging one or more filter banks adjoining a particular filter bank at which a loudness peak was flagged.

21. The device of claim 18 , wherein the coding includes:

first coding the flagged loudness peaks within the particular frame against the cross-frequency masking threshold by first bands of relative loudness;

second coding the flagged loudness peaks within the particular frame against the time masking threshold by second bands of relative loudness;

combining the first coding and second coding to set filter bank-by-filter bank values in the characteristic matrix for the particular frame; and

repeating the first and second coding and the combining for numerous sample frames of the sound.

22. The device of claim 18 , therein the processor is further configured to carry out actions including:

creating reference characteristic matrices for numerous reference sounds and at least one query characteristic matrix for a query sound;

comparing alignments of the query characteristic matrix to at least some of the reference characteristic matrices, including identifying filter bank-by-filter bank positive peak matches, negative peak matches, peak in query but not in reference mismatches, and peak in reference but not in query mismatches;

frame-by-frame, deriving composite scores across filter banks that distinctly weight and combine the positive peak matches, the negative peak matches, the peak in query but not in reference mismatches and peak in reference but not in query mismatches;

combining the frame-by-frame composite scores into query scores for the alignments and selecting one or more best alignments of the query characteristic matrix to the reference characteristic matrices; and

organizing the composite scores for the best alignments to identify likely query-to-reference characteristic matrix matches.

23. The device of claim 22 , wherein the processor is further configured to carry out actions including repeatedly analyzing multiple versions of the query characteristic matrix, prepared from the query sound using versions of sample frames that have starting times within the query sound that are offset in time between the sample frame versions.

24. The device of claim 22 , wherein the processor is further configured to carry out the action of comparing the composite scores for the best alignments and identifying as identical true matches a plurality of the composite scores that match each other and that are significantly better than any others of the composite scores, only if it is significantly better.

first coding the flagged loudness peaks within a particular frame against a cross-frequency masking threshold by first bands of relative loudness;

second coding the flagged loudness peaks within the particular frame against a time masking threshold by second bands of relative loudness;

combining the first coding and second coding to set filter bank-by-filter bank characteristic matrix flags for the particular frame; and

repeating the first and second coding and the combining for numerous sample frames of the sound.

25. A device that searches music references to find a music reference which matches a query, including:

at least one processor configured to carry out actions including:

creating frame-by-frame, filter bank-by-filter bank reference characteristic matrices for numerous reference sounds and at least one frame-by-frame, filter bank-by-filter bank query characteristic matrix for a query sound;

comparing alignments of the query characteristic matrix to at least some of the reference characteristic matrices, including identifying filter bank-by-filter bank positive peak matches, negative peak matches, peak in query but not in reference mismatches, and peak in reference but not in query mismatches;

frame-by-frame, deriving a plurality of composite scores across filter banks that distinctly weight and combine the positive peak matches, the negative peak matches, the peak in query but not in reference characteristic mismatches and peak in reference but not in query mismatches;

combining the frame-by-frame composite scores into query scores for the alignments and selecting one or more best alignments of the query characteristic matrix to the reference characteristic matrices; and

organizing the composite scores for the best alignments to identify likely query-to-reference characteristic matrix matches.

26. The device of claim 25 , wherein the processor is further configured to carry out actions including comparing the composite scores for the best alignments and identifying as a true match one of the composite scores that is significantly better than any others of the composite scores, only if it is significantly better.

27. The device of claim 25 , wherein the processor is further configured to carry out actions including comparing the composite scores for the best alignments and identifying as identical true matches a plurality of the composite scores that match each other and that are significantly better than any others of the composite scores, only if it is significantly better.

28. The device of claim 25 , wherein the scoring further includes:

29. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by at least one processor cause the processor to perform operations including:

using a processing device, repeatedly characterizing loudness of sound components grouped in filter banks of sample frames and residing in a computer readable memory, wherein the filter banks have a narrower frequency range and have closer mid-frequency spacing in frequencies between about 750 hz and 1450 hz than above 1500 hz or below 500 hz;

flagging loudness peaks among the filter banks that are above cross-frequency and time masking thresholds in the sample frames, wherein the cross-frequency masking threshold for a particular frame is set relative to filter band characterizations of one or more masking sounds and the time masking threshold for successive frames is set relative to the filter band characterizations of the masking sounds; and

coding the flagged loudness peaks in relation to the cross-frequency and time masking thresholds to create a frame-by-frame, filter bank-by-filter bank characteristic matrix.

30. The computer storage medium of claim 29 , wherein the flagging of loudness peaks above the cross-frequency and time masking thresholds avoids including in the frame-by-frame, filter bank-by-filter bank characteristic matrix of particular sound components that are likely to be eliminated from a frame sample by an auditory perception codec.

31. The computer storage medium of claim 29 , wherein the instructions further cause the processor to perform operations including flagging one or more filter banks adjoining a particular filter bank at which a loudness peak was flagged.

32. The computer storage medium of claim 29 , wherein the coding includes:

first coding the flagged loudness peaks within the particular frame against the cross-frequency masking threshold by first bands of relative loudness;

second coding the flagged loudness peaks within the particular frame against the time masking threshold by second bands of relative loudness;

combining the first coding and second coding to set filter bank-by-filter bank values in the characteristic matrix for the particular frame; and

repeating the first and second coding and the combining for numerous sample frames of the sound.

33. The computer storage medium of claim 29 , wherein the instructions further cause the processor to perform operations including:

creating reference characteristic matrices for numerous reference sounds and at least one query characteristic matrix for a query sound;

comparing alignments of the query characteristic matrix to at least some of the reference characteristic matrices, including identifying filter bank-by-filter bank positive peak matches, negative peak matches, peak in query but not in reference mismatches, and peak in reference but not in query mismatches;

frame-by-frame, deriving composite scores across filter banks that distinctly weight and combine the positive peak matches, the negative peak matches, the peak in query but not in reference mismatches and peak in reference but not in query mismatches;

combining the frame-by-frame composite scores into query scores for the alignments and selecting one or more best alignments of the query characteristic matrix to the reference characteristic matrices; and

organizing the composite scores for the best alignments to identify likely query-to-reference characteristic matrix matches.

34. The computer storage medium of claim 33 , wherein the instructions further cause the processor to perform operations including analyzing multiple versions of the query characteristic matrix, prepared from the query sound using versions of sample frames that have starting times within the query sound that are offset in time between the sample frame versions.

35. The computer storage medium of claim 33 , wherein the instructions further cause the processor to perform operations including comparing the composite scores for the best alignments and identifying as identical true matches a plurality of the composite scores that match each other and that are significantly better than any others of the composite scores, only if it is significantly better.

36. A computer readable storage medium that searches music references to find a music reference which matches a query, including:

at least one processor configured to carry out actions including:

creating frame-by-frame, filter bank-by-filter bank reference characteristic matrices for numerous reference sounds and at least one frame-by-frame, filter bank-by-filter bank query characteristic matrix for a query sound;

comparing alignments of the query characteristic matrix to at least some of the reference characteristic matrices, including identifying filter bank-by-filter bank positive peak matches, negative peak matches, peak in query but not in reference mismatches, and peak in reference but not in query mismatches;

frame-by-frame, deriving a plurality of composite scores across filter banks that distinctly weight and combine the positive peak matches, the negative peak matches, the peak in query but not in reference characteristic mismatches and peak in reference but not in query mismatches;

combining the frame-by-frame composite scores into query scores for the alignments and selecting one or more best alignments of the query characteristic matrix to the reference characteristic matrices; and

organizing the composite scores for the best alignments to identify likely query-to-reference characteristic matrix matches.

37. The computer storage medium of claim 36 , wherein the instructions further cause the processor to perform operations including comparing the composite scores for the best alignments and identifying as a true match one of the composite scores that is significantly better than any others of the composite scores, only if it is significantly better.

38. The computer storage medium of claim 36 , wherein the instructions further cause the processor to perform operations including comparing the composite scores for the best alignments and identifying as identical true matches a plurality of the composite scores that match each other and that are significantly better than any others of the composite scores, only if it is significantly better.

39. The computer storage medium of claim 36 , wherein the scoring further includes:

first coding the flagged loudness peaks within a particular frame against a cross-frequency masking threshold by first bands of relative loudness;

second coding the flagged loudness peaks within the particular frame against a time masking threshold by second bands of relative loudness;

combining the first coding and second coding to set filter bank-by-filter bank characteristic matrix flags for the particular frame; and

repeating the first and second coding and the combining for numerous sample frames of the sound.

Assignments (11)