IP Library Granted Patent US 11,574,618
Granted Patent B2
US 11,574,618 · App. 16/850,855 · Granted Feb 7, 2023

Methods and apparatus for harmonic source enhancement

Inventor: Zafar Rafii (Berkley, CA)
Assignee: Gracenote, Inc.
G10K11/175G06F3/162G10L25/18
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,574,618
App. No.
16/850,855
Granted
Feb 7, 2023
Kind
B2
Abstract

Methods and apparatus for harmonic source enhancement are disclosed herein. An example apparatus includes an interface to receive a media signal. The example apparatus also includes a harmonic source enhancer to determine a magnitude spectrogram of audio corresponding to the media signal; generate a time-frequency mask based on the magnitude spectrogram; and apply the time-frequency mask to the magnitude spectrogram to enhance a harmonic source of the media signal.

Claims (48)

1. A system comprising:

an audio analyzer to:

determine a magnitude spectrogram of audio corresponding to a media signal;

generate a time-frequency mask based on the magnitude spectrogram; and

apply the time-frequency mask to the magnitude spectrogram to enhance a harmonic source of the media signal; and

an audio determiner to:

compare the media signal having the enhanced harmonic source with reference media signals to determine an instrument corresponding to the enhanced harmonic source; and

identify a genre corresponding to the media signal based on the identified instrument.

2. The system of claim 1 , wherein the audio analyzer determines the magnitude spectrogram by transforming the audio into a time-frequency representation using at least one of a Fourier transform, or a Constant Q transform.

3. The system of claim 2 , wherein the time-frequency representation includes respective columns corresponding to a spectrum at a particular instant.

4. The system of claim 3 , wherein the audio analyzer determines a Fourier transform at each of the respective columns.

5. The system of claim 4 , wherein the audio analyzer generates an inverse Fourier transform for each of the respective columns.

6. The system of claim 5 , wherein the audio analyzer divides the Fourier transform by an absolute value of the Fourier transform.

7. The system of claim 1 , wherein the system further comprises an audio analyzer to:

transmit the enhanced harmonic source to the audio determiner; and

obtain at least one of a classification of the audio or an identifier corresponding to the media signal from the audio determiner.

8. The system of claim 1 , wherein the audio determiner determines audio equalizer settings for the media based on classification, further including an audio equalizer to adjust equalizer settings of a media output device that output the media based on the determined audio equalizer settings.

9. The system of claim 1 , wherein the harmonic source is a first harmonic source, the audio determiner identifies the genre of the media based on comparison with a second harmonic source of the reference media signals.

10. A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to at least:

determine a magnitude spectrogram of audio corresponding to a media signal;

generate a time-frequency mask based on the magnitude spectrogram;

apply the time-frequency mask to the magnitude spectrogram to enhance a harmonic source of the media signal;

compare the media signal having the enhanced harmonic source with reference media signals to determine an instrument corresponding to the enhanced harmonic source; and

identify a genre corresponding to the media signal based on the identified instrument.

11. The non-transitory computer readable medium of claim 10 , wherein the instructions further cause the one or more processors to transform the audio into a time-frequency representation using at least one of a Fourier transform, or a Constant Q transform.

12. The non-transitory computer readable medium of claim 11 , wherein the time-frequency representation includes respective columns corresponding to a spectrum at a particular instant.

13. The non-transitory computer readable medium of claim 12 , wherein the instructions further cause the one or more processors to determine a Fourier transform at each of the respective columns.

14. The non-transitory computer readable medium of claim 13 , wherein the instructions further cause the one or more processors to generate an inverse Fourier transform for each of the respective columns.

15. The non-transitory computer readable medium of claim 14 , wherein the instructions further cause the one or more processors to divide the Fourier transform by an absolute value of the Fourier transform.

16. A method comprising:

determining, by executing an instruction with one or more processors, a magnitude spectrogram of audio corresponding to a media signal;

generating, by executing an instruction with the one or more processors, a time-frequency mask based on the magnitude spectrogram;

applying, by executing an instruction with the one or more processors, the time-frequency mask to the magnitude spectrogram to enhance a harmonic source of the media signal;

comparing, by executing an instruction with the one or more processors, the media signal having the enhanced harmonic source with reference media signals to determine an instrument corresponding to the enhanced harmonic source; and

classifying, by executing an instruction with the one or more processors, a genre corresponding to the media signal based on the identified instrument.

17. The method of claim 16 , further including transforming the audio into a time-frequency representation using at least one of a Fourier transform, or a Constant Q transform, the time-frequency representation includes respective columns corresponding to a spectrum at a particular instant.

18. The method of claim 17 , further including determining a Fourier transform at each of the respective columns.

19. The method of claim 18 , further including generating an inverse Fourier transform for each of the respective columns.

20. The method of claim 19 , further including dividing the Fourier transform by an absolute value of the Fourier transform.

21. A system comprising:

at least one memory;

instructions; and

one or more processors to execute the instructions to:

determine a magnitude spectrogram of audio corresponding to a media signal;

generate a time-frequency mask based on the magnitude spectrogram;

apply the time-frequency mask to the magnitude spectrogram to enhance a harmonic source of the media signal;

compare the media signal having the enhanced harmonic source with reference media signals to determine an instrument corresponding to the enhanced harmonic source; and

identify at least one of a song, an advertisement, a show, or a video game based on the identified instrument.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 2, 2020
From: RAFII, ZAFAR
To: GRACENOTE, INC.
Reel/Frame 053676/0330 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →