IP Library Granted Patent US 11,854,557
Granted Patent B2
US 11,854,557 · App. 18/049,882 · Granted Dec 26, 2023

Audio fingerprinting

Inventors: Jinyu Han (Emeryville, CA); Robert Coover (Orinda, CA)
Assignee: Gracenote, Inc.
G10L19/018
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,854,557
App. No.
18/049,882
Granted
Dec 26, 2023
Kind
B2
Abstract

A machine may be configured to generate one or more audio fingerprints of one or more segments of audio data. The machine may access audio data to be fingerprinted and divide the audio data into segments. For any given segment, the machine may generate a spectral representation from the segment; generate a vector from the spectral representation; generate an ordered set of permutations of the vector; generate an ordered set of numbers from the permutations of the vector; and generate a fingerprint of the segment of the audio data, which may be considered a sub-fingerprint of the audio data. In addition, the machine or a separate device may be configured to determine a likelihood that candidate audio data matches reference audio data.

Claims (39)

1. A non-transitory machine readable medium comprising instructions for conserving computing resources, which when executed, cause a processor to at least:

determine a first and second group of frequencies in a plurality of frequencies, wherein the first group of frequencies includes frequencies that are higher than frequencies of the second group of frequencies;

identify a first subgroup of frequencies in the first group of frequencies and a second subgroup of frequencies in the second group of frequencies, wherein the first subgroup is identified based on energy values of the first group, and wherein the second subgroup is identified based on energy values of the second group;

generate a vector that assigns a first value to frequencies in the first subgroup and a second value to frequencies in the second subgroup;

generate an ordered set of permutations of the vector, wherein the ordered set of permutations is based on arranged instances of the first and second values;

generate a sequence that indicates an instance of the first value or of the second value within a permutation of the ordered set of permutations; and

generate a fingerprint of audio data based on the sequence.

2. The non-transitory machine readable medium of claim 1 , wherein at least one of the first and second group of frequencies is based on spectral data derived from the audio data.

3. The non-transitory machine readable medium of claim 1 , wherein at least one of the first subgroup of frequencies and the second subgroup of frequencies is identified based on ranked energy values for at least one of the first group of frequencies and the second subgroup of frequencies.

4. The non-transitory machine readable medium of claim 1 , wherein the first subgroup of frequencies is identified based on ranked energy values for the first group of frequencies, and wherein the second subgroup of frequencies is identified based on ranked energy values for the second group of frequencies.

5. The non-transitory machine readable medium of claim 1 , wherein the sequence is generated based on generating numbers by calculating a remainder from a modulo operation performed on a numerical representation of a lowest relative position occupied by any instance of the first or second values in the corresponding permutation.

6. The non-transitory machine readable medium of claim 1 , wherein the generated fingerprint comprises storing the sequence with a timestamp that indicates the audio data being fingerprinted.

7. The non-transitory machine readable medium of claim 1 , wherein the generated fingerprint comprises storing at least one portion of the sequence in a hash table corresponding to a timestamp that indicates the audio data being fingerprinted.

8. The non-transitory machine readable medium of claim 1 , wherein the ordered set of permutations are ordered by a number that is generated based on a position of a lowest frequency value.

9. The non-transitory machine readable medium of claim 1 , wherein the ordered set of permutations is generated based on performing a modulo operation.

10. The non-transitory machine readable medium of claim 9 , wherein the modulo operation is performed based on a position of a lowest frequency with a non-zero value.

11. A method for conserving computing resources, the method comprising:

determining a first and second group of frequencies in a plurality of frequencies, wherein the first group of frequencies includes frequencies that are higher than frequencies of the second group of frequencies;

identifying a first subgroup of frequencies in the first group of frequencies and a second subgroup of frequencies in the second group of frequencies, wherein the first subgroup is identified based on energy values of the first group, and wherein the second subgroup is identified based on energy values of the second group;

generating a vector that assigns a first value to frequencies in the first subgroup and a second value to frequencies in the second subgroup;

generating an ordered set of permutations of the vector, wherein the ordered set of permutations is based on arranged instances of the first and second values;

generating a sequence that indicates an instance of the first value or of the second value within a permutation of the ordered set of permutations; and

generating a fingerprint of audio data based on the sequence.

12. The method of claim 11 , wherein at least one of the first and second group of frequencies is based on spectral data derived from the audio data.

13. The method of claim 11 , wherein identifying at least one of the first subgroup of frequencies and the second subgroup of frequencies is based on ranked energy values for at least one of the first group of frequencies and the second subgroup of frequencies.

14. The method of claim 11 , wherein identifying the first subgroup of frequencies is based on ranked energy values for the first group of frequencies, and wherein identifying the second subgroup of frequencies is based on ranked energy values for the second group of frequencies.

15. The method of claim 11 , wherein generating the sequence further comprises generating numbers by calculating a remainder from a modulo operation performed on a numerical representation of a lowest relative position occupied by any instance of the first or second values in the corresponding permutation.

16. The method of claim 11 , wherein generating the fingerprint of the audio data further comprises storing the sequence with a timestamp that indicates the audio data being fingerprinted.

17. The method of claim 11 , wherein generating of the fingerprint of the audio data further comprises storing at least one portion of the sequence in a hash table corresponding to a timestamp that indicates the audio data being fingerprinted.

18. The method of claim 11 , wherein the ordered set of permutations are ordered by a number that is generated based on a position of a lowest frequency value.

19. The method of claim 11 , wherein the ordered set of permutations is generated based on performing a modulo operation.

20. An apparatus for conserving computing resources, the apparatus comprising:

a vector generator to:

determine a first and second group of frequencies in a plurality of frequencies, wherein the first group of frequencies includes frequencies that are higher than frequencies of the second group of frequencies;

identify a first subgroup of frequencies in the first group of frequencies and a second subgroup of frequencies in the second group of frequencies, wherein the first subgroup is identified based on energy values of the first group, and wherein the second subgroup is identified based on energy values of the second group; and

generate a vector that assigns a first value to frequencies in the first subgroup and a second value to frequencies in the second subgroup;

a scramble to generate an ordered set of permutations of the vector, wherein the ordered set of permutations is based on arranged instances of the first and second values;

a coder to generate a sequence that indicates an instance of the first value or of the second value within a permutation of the ordered set of permutations; and

a fingerprint generator to generate a fingerprint of audio data based on the sequence.

Assignments (4)
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 27, 2022
From: HAN, JINYU; COOVER, ROBERT
To: GRACENOTE, INC.
Reel/Frame 061565/0784 →
Continuity (5)
Continuation 16926286 · Jul 10, 2020
Continuation 16270113 · Feb 7, 2019
Continuation 15008042 · Jan 27, 2016
Continuation 14107923 · Dec 16, 2013
Related Publication 20230070089A1 · Mar 9, 2023