IP Library Granted Patent US 12,586,554
Granted Patent B2
US 12,586,554 · App. 18/743,215 · Granted Mar 24, 2026

Methods and apparatus to extract a pitch-independent timbre attribute from a media signal

Inventor: Zafar Rafii (Berkeley, CA)
Assignee: The Nielsen Company (US), LLC
G10H3/125G10H1/06G10H2210/056G10H2250/221G10H2250/235
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,586,554
App. No.
18/743,215
Granted
Mar 24, 2026
Kind
B2
Abstract

Methods and apparatus to extract a pitch-independent timbre attribute from a media signal are disclosed. An example apparatus includes an audio characteristic extractor to determine a logarithmic spectrum of an audio signal; transform the logarithmic spectrum of the audio signal into a frequency domain to generate a transform output; determine a magnitude of the transform output; and determine a timbre attribute of the audio signal based on an inverse transform of the magnitude.

Claims (31)

1 . A computing device comprising:

one or more processors; and

a tangible, non-transitory computer readable storage medium comprising instructions which, when executed cause the one or more processors to perform a set of operations comprising:

accessing a media signal;

extracting a pitch-independent timbre spectrum of the accessed media signal; and

classifying the accessed media signal based on data corresponding to the pitch-independent timbre spectrum.

2 . The computing device of claim 1 , wherein the media signal comprises an audio signal.

3 . The computing device of claim 1 , wherein the media signal comprises an audio component of a video signal.

4 . The computing device of claim 3 , wherein the set of operations further comprise extracting the media signal from the video signal.

5 . The computing device of claim 1 , wherein extracting a pitch-independent timbre of the accessed media signal comprises determining a logarithmic spectrum of the media signal and transforming the logarithmic spectrum of the media signal into a frequency domain to generate a transform output.

6 . The computing device of claim 5 , wherein determining the logarithmic spectrum of the media signal comprises using a constant Q transform.

7 . The computing device of claim 5 , wherein extracting a pitch-independent timbre of the accessed media signal comprises determining a magnitude of the transform output and a timbre attribute of the media signal based on an inverse transform of the magnitude.

8 . The computing device of claim 7 , wherein determining the transform of the logarithmic spectrum is based on a Fourier transform and determining the inverse transform is based on using an inverse Fourier transform.

9 . The computing device of claim 7 , wherein determining a timbre-independent pitch attribute of the media signal is based on an inverse transform of a complex argument of the transform of the logarithmic spectrum.

10 . The computing device of claim 1 , wherein the classification corresponds to at least one of an instrument or a genre.

11 . The computing device of claim 1 , wherein the set of operations further comprises identifying a media source of the media signal based on the classification.

12 . The computing device of claim 1 , wherein the set of operations further comprises comparing the pitch-independent timbre spectrum of the media signal to one or more reference pitch-independent timbre spectrums, and wherein classifying the media signal is based on matching one or more reference pitch-independent timbre spectrums to the extracted pitch-independent timbre spectrum.

13 . The computing device of claim 1 , the set of operations further comprises comparing the pitch-independent timbre spectrum of the media signal to one or more reference pitch-independent timbre spectrums, and based on determining that the extracted pitch-independent timbre spectrum does not match the one or more reference pitch-independent timbre spectrums, prompt for additional information corresponding to the media signal.

14 . The computing device of claim 1 , wherein the set of operations further comprises determining a device setting adjustment based on the classification.

15 . A tangible, non-transitory computer readable storage medium comprising instructions which, when executed cause one or more processors to perform a set of operations comprising:

accessing a media signal;

extracting a pitch-independent timbre spectrum of the accessed media signal; and

classifying the accessed media signal based on data corresponding to the pitch-independent timbre spectrum.

16 . The tangible, non-transitory computer readable storage medium of claim 15 , wherein the media signal comprises an audio signal.

17 . The tangible, non-transitory computer readable storage medium of claim 15 , wherein the media signal comprises an audio component of a video signal.

18 . The tangible, non-transitory computer readable storage medium of claim 17 , wherein the set of operations further comprises extracting the media signal from the video signal.

19 . The tangible, non-transitory computer readable storage medium of claim 15 , wherein extracting a pitch-independent timbre of the accessed media signal comprises determining a logarithmic spectrum of the media signal and transforming the logarithmic spectrum of the media signal into a frequency domain to generate a transform output.

20 . A computer-implemented method comprising:

accessing, by one or more processors, a media signal;

extracting, by the one or more processors, a pitch-independent timbre spectrum of the accessed media signal; and

classifying, by the one or more processors, the accessed media signal based on data corresponding to the pitch-independent timbre spectrum.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2024
From: RAFII, ZAFAR
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 067728/0685 →
Continuity (7)
Continuation 18357526 · Jul 24, 2023
Continuation 17157780 · Jan 25, 2021
Continuation 16821567 · Mar 17, 2020
Continuation 16659099 · Oct 21, 2019
Continuation 16239238 · Jan 3, 2019
Continuation 15920060 · Mar 13, 2018
Related Publication 20240331669A1 · Oct 3, 2024
References Cited (89)
US 2008007A · Dreffein · 1935 [cited by applicant]
US 3681530A · Manley · 1972 [cited by applicant]
US 6054646A · Pal et al. · 2000 [cited by applicant]
US 6363345B1 · Marash et al. · 2002 [cited by applicant]
US 7406356B2 · Peeters et al. · 2008 [cited by applicant]
US 7667125B2 · Taub · 2010 [cited by examiner]
US 8311821B2 · Breebaart et al. · 2012 [cited by applicant]
US 8380331B1 · Smaragdis · 2013 [cited by examiner]
US 8942977B2 · Chen · 2015 [cited by examiner]
US 9135923B1 · Chen · 2015 [cited by examiner]
US 9202472B1 · Sharifi et al. · 2015 [cited by applicant]
US 9916834B2 · Sukowski et al. · 2018 [cited by applicant]
US 10186247B1 · Rafii · 2019 [cited by examiner]
US 10482863B2 · Rafii · 2019 [cited by examiner]
US 10629178B2 · Rafii · 2020 [cited by examiner]
US 10902831B2 · Rafii · 2021 [cited by examiner]
US 11749244B2 · Rafii · 2023 [cited by examiner]
US 12051396B2 · Rafii · 2024 [cited by examiner]
US 20050108004A1 · Otani et al. · 2005 [cited by applicant]
US 20050211071A1 · Lu · 2005 [cited by examiner]
US 20070107584A1 · Kim · 2007 [cited by examiner]
US 20070131096A1 · Lu · 2007 [cited by examiner]
US 20070169613A1 · Kim et al. · 2007 [cited by applicant]
US 20070174274A1 · Kim et al. · 2007 [cited by applicant]
US 20070291958A1 · Jehan · 2007 [cited by examiner]
US 20080075303A1 · Kim et al. · 2008 [cited by applicant]
US 20080082323A1 · Bai · 2008 [cited by examiner]
US 20080190269A1 · Eom et al. · 2008 [cited by applicant]
US 20080271592A1 · Beckford · 2008 [cited by examiner]
US 20100131086A1 · Itoyama · 2010 [cited by examiner]
US 20100154619A1 · Taub · 2010 [cited by examiner]
US 20100241423A1 · Jackson et al. · 2010 [cited by applicant]
US 20110303075A1 · McMillen · 2011 [cited by examiner]
US 20130019739A1 · Vainaila · 2013 [cited by applicant]
US 20130151256A1 · Nakano et al. · 2013 [cited by applicant]
US 20130182862A1 · Disch · 2013 [cited by examiner]
US 20130339011A1 · Visser · 2013 [cited by examiner]
US 20140074469A1 · Zhidkov · 2014 [cited by applicant]
US 20140180674A1 · Neuhauser · 2014 [cited by examiner]
US 20140200889A1 · Chen · 2014 [cited by examiner]
US 20150262587A1 · Chen · 2015 [cited by applicant]
US 20160037275A1 · Drullinger et al. · 2016 [cited by applicant]
US 20160196812A1 · Rashad et al. · 2016 [cited by applicant]
US 20170094440A1 · Brown et al. · 2017 [cited by applicant]
US 20180018979A1 · Rolland et al. · 2018 [cited by applicant]
US 20180068646A1 · Esparza · 2018 [cited by examiner]
US 20180276540A1 · Xing · 2018 [cited by applicant]
US 20190287506A1 · Rafii · 2019 [cited by examiner]
US 20200051538A1 · Rafii · 2020 [cited by examiner]
US 20200219473A1 · Rafii · 2020 [cited by examiner]
US 20200286505A1 · Osborne · 2020 [cited by examiner]
US 20210090535A1 · Miles · 2021 [cited by examiner]
US 20210151021A1 · Rafii · 2021 [cited by examiner]
US 20210327400A1 · Rafii · 2021 [cited by examiner]
US 20230368761A1 · Rafii · 2023 [cited by examiner]
US 20240331669A1 · Rafii · 2024 [cited by examiner]
JP 2009508156 · 2009 [cited by applicant]
JP 2009508156A · 2009 [cited by examiner]
JP 2010518428 · 2010 [cited by applicant]
JP 2011217052 · 2011 [cited by applicant]
JP 2017040963 · 2017 [cited by applicant]
JP 2017090848 · 2017 [cited by applicant]
KR 101101384 · 2012 [cited by applicant]
KR 101757338 · 2017 [cited by applicant]
WO 2007029002 · 2007 [cited by applicant]
WO 2014202770 · 2014 [cited by applicant]
K. Jensen, “Pitch independent prototyping of musical sounds,” 1999 IEEE Third Workshop on Multimedia Signal Processing (Cat. No. 99TH8451), Copenhagen, Denmark, 1999, pp. 215-220, doi: 10.1109/MMSP. 1999.793823 (Year: 1… [cited by examiner]
European Patent Office, “Examination Report,” issued in connection with EP Application No. 19766557.3, dated Nov. 30, 2021,10 pages. [cited by applicant]
Duffieu et al., “A musically motivated mid-level representation for pitch estimation and musical audio source separation,” IEE Journal of Selected Topics in Signal Processing, vol. 5, No. 6, Oct. 1, 2011, submitted to H… [cited by applicant]
European Patent Office, “Communication pursuant to Rule 70(2) and 70a(2) EPC,” issued in connection with EP Application No. 19766557.3, dated Dec. 17, 2021, 1 pages. [cited by applicant]
Japanese Patent Office, “Notice of Reasons for Rejection,” issued in JP Application No. 2020-545802, dated Nov. 30, 2021, 7 pages. [cited by applicant]
Japanese Patent Office, “Notice of Reasons for Rejection,” issued in JP Application No. 2020-545802, dated Jul. 12, 2022, 5 pages. [cited by applicant]
Patent Cooperation Treaty, “International Search Report,” mailed in connection with International Patent Application No. PCT/US2019/021865, on Jun. 27, 2019, 4 pages. [cited by applicant]
Patent Cooperation Treaty, “Written Opinion of the International Search Authority, ” mailed in connection with International Patent Application No. PCT/US2019/021865, on Jun. 27, 2019, 3 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due”, issued in connection with U.S. Appl. No. 15/920,060, filed Sep. 11, 2019, 8 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 16/239,238, filed Mar. 22, 2019, 5 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 16/239,238, filed Jul. 15, 2019, 8 pages. [cited by applicant]
Marozeau, Jeremy et al., “The Dependency of Timbre on Fundamental Frequency,” The Journal of the Acoustical Society of America, Nov. 2003, pp. 2946-2957, 12 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 16/659,099, filed Dec. 18, 2019, 8 pages. [cited by applicant]
Patent Cooperation Treaty, “International Preliminary Report on Patentability,” mailed in connection with International Patent Application No. PCT/US2019/021865, filed Sep. 15, 2020, 4 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 16/821,567, filed Jun. 1, 2020, 7 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 16/821,567, filed Sep. 23, 2020, 5 pages. [cited by applicant]
Turnbull, D., “Automatic Music Annotation”, Internet Citation, May 24, 2005 (May 25, 2005), pp. 1-19, XP002569975. [cited by applicant]
Jensen, K., “Pitch independent prototyping of musical sounds,” 1999 IEEE Third Workshop on Multimedia Signal Processing (Cat. No. 99TH8451), Copenhagen, Denmark, 1999, pp. 215-220. [cited by applicant]
European Search Report in corresponding application No. 20891501.7 dated Oct. 13, 2023, 5 pages. [cited by applicant]
European Examination Report in corresponding application No. 19766557.3 dated Dec. 1, 2023, 6 pages. [cited by applicant]
Extended European Search Report for corresponding European Patent Application No. 25198533.9 dated Oct. 28, 2025, 10 pages. [cited by applicant]
Kostek, Bozena, “Musical Instrument Classification and Duet Analysis Employing Music Information Retrieval Techniques,” Proceedings of the IEEE, vol. 92, No. 4, pp. 712-729, (Apr. 2004). [cited by applicant]
Park et al., “Clustering Algorithm for Unsupervised Monaural Musical Sound Separation Based on Non-negative Matrix Factorization,” IEICE Trans. Fundamentals, vol. E95-A, No. 4, pp. 818-823, (Apr. 2012). [cited by applicant]