IP Library › Granted Patent US 12,681,938
Granted Patent B2
US 12,681,938 · App. 18/086,865 · Granted Jul 14, 2026

Methods and apparatus for determining a mood profile associated with media data

Inventors: Ching-Wei Chen (Oakland, CA); Kyogu Lee (Albany, CA); Peter C. DiMaria (Berkeley, CA); Markus K. Cremer (Orinda, CA)
Assignee: Gracenote, Inc.
G06F16/24578G06F16/4387G10H1/0008G10H2240/085G10H2240/131
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,681,938
App. No.
18/086,865
Granted
Jul 14, 2026
Kind
B2
Abstract

An example method involves comparing a primary element of a first piece of audio data to a primary element of a second piece of audio data; based on the comparing of the primary elements, determining that the first and second pieces of audio data have the same predominant mood category; in response to determining that the first and second pieces of audio data have the same predominant mood category, comparing a first mood score of the primary element of the first piece of audio data to a second mood score of the primary element of a second piece of audio data; determining that an output of the comparison of the two mood scores exceeds a threshold value; and in response to determining that the output of the comparison of the two mood scores exceeds the threshold value, providing an indicator to an application.

Claims (40)

1 . A computer-implemented method for matching of audio data comprising:

comparing, by one or more processors, a primary element of a first piece of audio data to a primary element of a second piece of audio data, wherein the primary element of the first piece of audio data identifies a predominant mood category of the first piece of audio data, wherein the primary element of the second piece of audio data identifies a predominant mood category of the second piece of audio data, and wherein a trained classifier generates mood-category scores and the primary elements used for the comparison are obtained by partial retrieval of the mood-category scores, wherein the partial retrieval reduces at least one of a required memory or a required processing power for the comparison;

determining, by the one or more processors, that the first and second pieces of audio data have the same predominant mood category based on comparing the primary elements of the first and second pieces of audio data to a correlates matrix;

in response to determining that the first and second pieces of audio data have the same predominant mood category, comparing, by the one or more processors, a first mood score of the primary element of the first piece of audio data to a second mood score of the primary element of a second piece of audio data;

determining, by the one or more processors, that an output of the comparison of the first mood score and the second mood score exceeds a threshold value; and

in response to determining that the output of the comparison of the first mood score and the second mood score exceeds the threshold value, providing an indicator to an application, by the one or more processors, that causes the application to perform an operation associated with the second piece of audio data.

2 . The computer-implemented method of claim 1 , wherein providing the indicator to the application comprises displaying a selectable indication of the second piece of audio data on a user interface.

3 . The computer-implemented method of claim 1 , wherein providing the indicator to the application comprises accessing the second piece of audio data from data storage.

4 . The computer-implemented method of claim 1 , wherein providing the indicator to the application comprises playing the second piece of audio data using a signal generation device.

5 . The computer-implemented method of claim 1 , wherein the threshold value is set automatically by the application.

6 . The computer-implemented method of claim 1 , wherein the threshold value is a user-preferred threshold set by a user of the application.

7 . The computer-implemented method of claim 1 , wherein providing the indicator to the application comprises adding the second piece of audio data to a playlist.

8 . The computer-implemented method of claim 1 , further comprising extracting, by the one or more processors, low-level audio feature data from an audio signal associated with the audio data to determine the primary element of the first piece of audio data.

9 . The computer-implemented method of claim 1 , wherein the partial retrieval reduces both the required memory and the processing power for the comparison.

10 . A non-transitory tangible machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations for matching of audio data comprising:

comparing a primary element of a first piece of audio data to a primary element of a second piece of audio data, wherein the primary element of the first piece of audio data identifies a predominant mood category of the first piece of audio data, wherein the primary element of the second piece of audio data identifies a predominant mood category of the second piece of audio data, and wherein a trained classifier generates mood-category scores, and wherein the primary elements used for the comparison are obtained by partial retrieval of the mood-category scores, wherein the partial retrieval reduces at least one of a required memory or a required processing power for the comparison;

determining that the first and second pieces of audio data have the same predominant mood category based on comparing the primary elements of the first and second pieces of audio data to a correlates matrix;

in response to determining that the first and second pieces of audio data have the same predominant mood category, comparing a first mood score of the primary element of the first piece of audio data to a second mood score of the primary element of a second piece of audio data;

determining that an output of the comparison of the first mood score and the second mood score exceeds a threshold value; and

in response to determining that the output of the comparison of the first mood score and the second mood score exceeds the threshold value, providing an indicator to an application that causes the application to perform an operation associated with the second piece of audio data.

11 . The non-transitory tangible machine-readable storage medium of claim 10 , wherein providing the indicator to the application comprises displaying a selectable indication of the second piece of audio data on a user interface.

12 . The non-transitory tangible machine-readable storage medium of claim 10 , wherein providing the indicator to the application comprises accessing the second piece of audio data from data storage.

13 . The non-transitory tangible machine-readable storage medium of claim 10 , wherein providing the indicator to the application comprises playing the second piece of audio data using a signal generation device.

14 . The non-transitory tangible machine-readable storage medium of claim 10 , wherein the threshold value is set automatically by the application.

15 . The non-transitory tangible machine-readable storage medium of claim 10 , wherein the threshold value is a user-preferred threshold set by a user of the application.

16 . The non-transitory tangible machine-readable storage medium of claim 10 , wherein providing the indicator to the application comprises adding the second piece of audio data to a playlist.

17 . A computing system comprising:

a memory that stores instructions;

a database;

one or more processors configured by the instructions to perform operations for matching of audio data comprising:

comparing, by the one or more processors, a primary element of a first piece of audio data to a primary element of a second piece of audio data, wherein the primary element of the first piece of audio data identifies a predominant mood category of the first piece of audio data, wherein the primary element of the second piece of audio data identifies a predominant mood category of the second piece of audio data, and wherein a trained classifier generates mood-category scores and the primary elements used for the comparing are obtained by partial retrieval of the mood-category scores, wherein the partial retrieval reduces at least one of a required memory or a required processing power for the comparison;

determining, by the one or more processors, that the first and second pieces of audio data have the same predominant mood category based on comparing the primary elements of the first and second pieces of audio data to a correlates matrix;

in response to determining that the first and second pieces of audio data have the same predominant mood category, comparing, by the one or more processors, a first mood score of the primary element of the first piece of audio data to a second mood score of the primary element of a second piece of audio data;

determining, by the one or more processors, that an output of the comparison of the first mood score and the second mood score exceeds a threshold value; and

in response to determining that the output of the comparison of the first mood score and the second mood score exceeds the threshold value, providing an indicator to an application, by the one or more processors, that causes the application to perform an operation associated with the second piece of audio data.

18 . The computing system of claim 17 , wherein providing, by the one or more processors, the indicator to the application comprises displaying a selectable indication of the second piece of audio data on a user interface.

19 . The computing system of claim 17 , wherein providing, by the one or more processors, the indicator to the application comprises accessing the second piece of audio data from data storage.

20 . The computing system of claim 17 , wherein providing, by the one or more processors, the indicator to the application comprises playing the second piece of audio data using a signal generation device.

21 . The computing system of claim 17 , wherein the threshold value is set automatically by the application.

22 . The computing system of claim 17 , wherein the threshold value is a user-preferred threshold set by a user of the application.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 22, 2022
From: CHEN, CHING-WEI; LEE, KYOGU; DIMARIA, PETER C.; CREMER, MARKUS K.
To: GRACENOTE, INC.
Reel/Frame 062182/0578 →
Continuity (6)
Continuation 17525498 · Nov 12, 2021
Continuation 16685281 · Nov 15, 2019
Continuation 15807378 · Nov 8, 2017
Continuation 14312007 · Jun 23, 2014
Continuation 12489861 · Jun 23, 2009
Related Publication 20230129425A1 · Apr 27, 2023
References Cited (110)
US 5918223A · Blum et al. · 1999 [cited by applicant]
US 6505160B1 · Levy et al. · 2003 [cited by applicant]
US 6539395B1 · Gjerdingen · 2003 [cited by examiner]
US 6545209B1 · Flannery · 2003 [cited by examiner]
US 6748395B1 · Picker · 2004 [cited by examiner]
US 6829368B2 · Meyer et al. · 2004 [cited by applicant]
US 6941275B1 · Swierczek · 2005 [cited by applicant]
US 6993532B1 · Platt · 2006 [cited by examiner]
US 7115808B2 · Lu · 2006 [cited by examiner]
US 7243104B2 · Bill · 2007 [cited by examiner]
US 7302574B2 · Conwell et al. · 2007 [cited by applicant]
US 7349552B2 · Levy et al. · 2008 [cited by applicant]
US 7396990B2 · Lu · 2008 [cited by examiner]
US 7415129B2 · Rhoads · 2008 [cited by applicant]
US 7461136B2 · Rhoads · 2008 [cited by applicant]
US 7587602B2 · Rhoads · 2009 [cited by applicant]
US 7590259B2 · Levy et al. · 2009 [cited by applicant]
US 7921369B2 · Bill · 2011 [cited by examiner]
US 8015186B2 · Kobayashi · 2011 [cited by applicant]
US 8071869B2 · Chen et al. · 2011 [cited by applicant]
US 8326584B1 · Wells et al. · 2012 [cited by applicant]
US 8373768B2 · Bill · 2013 [cited by examiner]
US 8443290B2 · Bill · 2013 [cited by examiner]
US 8805854B2 · Chen · 2014 [cited by examiner]
US 8996538B1 · Cremer · 2015 [cited by examiner]
US 9122752B2 · Bill · 2015 [cited by examiner]
US 9842146B2 · Chen · 2017 [cited by examiner]
US 10510328B2 · Nazer · 2019 [cited by examiner]
US 10558674B2 · Chen · 2020 [cited by examiner]
US 10909999B2 · Kelly · 2021 [cited by examiner]
US 11204930B2 · Chen · 2021 [cited by examiner]
US 11580120B2 · Chen · 2023 [cited by examiner]
US 20020018074A1 · Buil et al. · 2002 [cited by applicant]
US 20030205124A1 · Foote et al. · 2003 [cited by applicant]
US 20040068401A1 · Herre et al. · 2004 [cited by applicant]
US 20040083232A1 · Ronnewinkel · 2004 [cited by examiner]
US 20040237759A1 · Bill · 2004 [cited by examiner]
US 20050192934A1 · Ellis · 2005 [cited by examiner]
US 20050217463A1 · Kobayashi · 2005 [cited by applicant]
US 20060054007A1 · Lu · 2006 [cited by examiner]
US 20060143647A1 · Bill · 2006 [cited by examiner]
US 20060170945A1 · Bill · 2006 [cited by examiner]
US 20060277171A1 · Ellis et al. · 2006 [cited by applicant]
US 20070089057A1 · Kindig · 2007 [cited by examiner]
US 20070131096A1 · Lu · 2007 [cited by examiner]
US 20070174274A1 · Kim et al. · 2007 [cited by applicant]
US 20080188964A1 · Bech et al. · 2008 [cited by applicant]
US 20080215979A1 · Clifton et al. · 2008 [cited by applicant]
US 20080281590A1 · Breebaart et al. · 2008 [cited by applicant]
US 20090006397A1 · Lehtiniemi · 2009 [cited by applicant]
US 20090063414A1 · White · 2009 [cited by examiner]
US 20090063971A1 · White · 2009 [cited by examiner]
US 20090069914A1 · Kemp · 2009 [cited by examiner]
US 20090071316A1 · Oppenheimer · 2009 [cited by applicant]
US 20090139389A1 · Bowen · 2009 [cited by applicant]
US 20090182736A1 · Ghatak · 2009 [cited by examiner]
US 20090217804A1 · Lu · 2009 [cited by examiner]
US 20090249945A1 · Yamashia et al. · 2009 [cited by applicant]
US 20100138427A1 · Van De Par et al. · 2010 [cited by applicant]
US 20100191733A1 · Park · 2010 [cited by examiner]
US 20100282045A1 · Chen · 2010 [cited by examiner]
US 20100321519A1 · Bill · 2010 [cited by examiner]
US 20110239137A1 · Bill · 2011 [cited by examiner]
US 20130067348A1 · Kast · 2013 [cited by applicant]
US 20130243245A1 · Bill · 2013 [cited by examiner]
US 20130287214A1 · Resch · 2013 [cited by examiner]
US 20130298044A1 · Bill · 2013 [cited by examiner]
US 20140149468A1 · Lu · 2014 [cited by examiner]
US 20140330848A1 · Chen · 2014 [cited by examiner]
US 20180049688A1 · Knight · 2018 [cited by examiner]
US 20180075039A1 · Chen · 2018 [cited by examiner]
US 20190171409A1 · Boulanger · 2019 [cited by examiner]
US 20200081897A1 · Chen · 2020 [cited by examiner]
US 20220067057A1 · Chen · 2022 [cited by examiner]
US 20230129425A1 · Chen · 2023 [cited by examiner]
CN 107851100 · 2022 [cited by applicant]
CN 107851100B · 2022 [cited by examiner]
EP 1197020B1 · 2007 [cited by applicant]
JP 7206288 · 2023 [cited by applicant]
JP 7206288B2 · 2023 [cited by examiner]
WO 2002084640A2 · 2002 [cited by applicant]
WO 2005106877A1 · 2005 [cited by applicant]
WO 2010129693A1 · 2010 [cited by applicant]
WO 2010151421A1 · 2010 [cited by applicant]
The effect of induced mood states on performance profile areas of perceived need, Jo et al., (Year: 1999). [cited by examiner]
The Measurement and Conceptualization of Mood Awareness: Monitoring and Labeling One's Mood States, Alan et al., vol. 21, No. 9 (Year: 1995). [cited by examiner]
Chen, et al., “Improving Perceived Tempo Estimation by Statistical Modeling of Higer-Level Musical Descriptors,” Audio Enginering Society Convention Paper, 8 pages, 2009. [cited by applicant]
Davies, et al., “Exploring the effect of rhythmic style classification on automatic tempo estimation, Proc European Signal Processing Conf,” Retrieved from the Internet: URL:http://www.elec.qmul.ac.uk/markp/2008/DaviesP… [cited by applicant]
International Application Serial No. PCT/US2010/033753, International Preliminary Report on Patentability mailed Nov. 17, 2011, 9 pages. [cited by applicant]
International Application Serial No. PCT/US2010/033753, Search Report mailed Oct. 15, 2010, 8 pages. [cited by applicant]
International Application Serial No. PCT/US2010/033753, Written Opinion mailed Oct. 15, 2010, 7 pages. [cited by applicant]
International Application Serial No. PCT/US2010/037665, International Preliminary Report on Patentability mailed Jan. 12, 2012, 7 pages. [cited by applicant]
International Application Serial No. PCT/US2010/037665, Search Report mailed Sep. 15, 2010, 18 pages. [cited by applicant]
International Application Serial No. PCT/US2010/037665, Written Opinion mailed Sep. 15, 2010, 18 pages. [cited by applicant]
Laurier, et al., Multimodal Music Mood Classification using Audio and Lyrics, Seventh International Conference on Machine Learning and Applications, Conference Paper, pp. 689-693 2008. [cited by applicant]
Linxing, et al., “Using Astatistic Model to Capture the Association Between Timbre and Perceived Tempo,” Retrieved from the Inernet; URL:http://ismir2008.ismir.net/papers/ISMI R2008_231.pdf, pp. 659-662, 2008. [cited by applicant]
Schuller, et al., “Tango or Waltz? Putting Ballroom Dance Style into Tempo Detection,” Eurasip Journal on Audio Speech and Music Processing, ISSN 1687-4714 001: 001: 10.115/2008/846135, 12 pages, 2008. [cited by applicant]
U.S. Appl. No. 12/436,718, Non-final Office Action mailed Apr. 13, 2011, 8 pages. [cited by applicant]
U.S. Appl. No. 12/436,718, Notice of Allowance mailed Jul. 25, 2011, 8 pages. [cited by applicant]
U.S. Appl. No. 12/436,718, Notice of Allowance mailed Aug. 25, 2011, 9 pages. [cited by applicant]
U.S. Appl. No. 12/436,718, Response filed Jun. 29, 2011 to Non-final Office Action dated Apr. 13, 2011, 10 pages. [cited by applicant]
U.S. Appl. No. 12/436,718, Response to Restriction Requirement mailed Jan. 20, 2011, 6 pages. [cited by applicant]
U.S. Appl. No. 12/436,718, Restriction Requirement mailed Jan. 20, 2011, 6 pages. [cited by applicant]
U.S. Appl. No. 12/489,861, Amendment Under 37 C.F.R. filed Jun. 26, 2014 to Notice of Allowance mailed Mar. 28, 2014, 10 pages. [cited by applicant]
U.S. Appl. No. 12/489,861, Final Office Action Mailed Jan. 23, 2012, 33 pages. [cited by applicant]
U.S. Appl. No. 12/489,861, Non-final Office Action mailed Jul. 13, 2011, 22 pages. [cited by applicant]
U.S. Appl. No. 12/489,861, Notice of Allowance mailed Mar. 28, 2014, 18 pages. [cited by applicant]
U.S. Appl. No. 12/489,861, PTO Response to Rule 312 Communication mailed Jul. 15, 2014, 2 pages. [cited by applicant]
U.S. Appl. No. 12/489,861, Response filed Apr. 10, 2012 to Final Office Action mailed Jan. 23, 2012, 18 pages. [cited by applicant]
U.S. Appl. No. 12/489,861, Response filed Oct. 6, 2011 to Non-Final Office Action mailed Jul. 13, 2011, 21 pages. [cited by applicant]