IP Library › Granted Patent US 12,499,884
Granted Patent B2
US 12,499,884 · App. 18/497,362 · Granted Dec 16, 2025

Modification of electronic system operation based on acoustic ambience classification

Inventors: Suresh Jeyachandran (Alameda, CA); Vadim Brenner (San Francisco, CA); Markus K. Cremer (Orinda, CA)
Assignee: Gracenote, Inc.
G10L15/20G10L25/48H03G3/002H03G3/3005H03G3/3089G10L2015/225
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,499,884
App. No.
18/497,362
Granted
Dec 16, 2025
Kind
B2
Abstract

Methods and systems for modification of electronic system operation based on acoustic ambience classification are presented. In an example method, at least one audio signal present in a physical environment of a user is detected. The at least one audio signal is analyzed to extract at least one audio feature from the audio signal. The audio signal is classified based on the audio feature to produce at least one classification of the audio signal. Operation of an electronic system interacting with the user in the physical environment is modified based on the classification of the audio signal.

Claims (38)

1 . A non-transitory computer readable storage medium comprising instructions that, when executed, cause one or more processors to perform a set of operations comprising:

extracting audio information from a first audio signal, wherein the first audio signal detected by a sensor during presentation of media content in an environment, and wherein the audio information corresponds to the media content;

extracting supplemental audio information from a second audio signal, wherein the second audio signal is detected by the sensor during the presentation of the media content in the environment, and wherein the supplemental audio information indicates an action by one or more users in the environment;

assigning a first classification to the first audio signal based on the audio information;

assigning a second classification to the second audio signal based on the supplemental audio information; and

causing a media player to modify the media content based on the second classification of the second audio signal.

2 . The non-transitory computer readable storage medium of claim 1 , wherein the second audio signal corresponds to the media content.

3 . The non-transitory computer readable storage medium of claim 1 , wherein a classification model is used to extract at least one of the audio information and the supplemental audio information.

4 . The non-transitory computer readable storage medium of claim 1 , wherein the first audio signal is a filtered audio signal and the second audio signal is an unfiltered audio signal.

5 . The non-transitory computer readable storage medium of claim 1 , wherein the audio information includes one or more of volume, pitch, energy, bandwidth, or zero crossing rates associated with the first audio signal.

6 . The non-transitory computer readable storage medium of claim 1 , wherein the supplemental audio information includes one or more of volume, pitch, energy, bandwidth, or zero crossing rates associated with the second audio signal.

7 . The non-transitory computer readable storage medium of claim 1 , wherein supplemental audio information further indicates environmental noise not associated with the one or more users.

8 . The non-transitory computer readable storage medium of claim 7 , wherein the environmental noise is based on a location of the environment.

9 . The non-transitory computer readable storage medium of claim 7 , wherein the set of operations further comprise generating an instruction to cause the media player to adjust a volume of the media content based on a level of the environmental noise relative to a threshold.

10 . The non-transitory computer readable storage medium of claim 1 , wherein the action of the one or more users includes a user input at the media player.

11 . The non-transitory computer readable storage medium of claim 1 , wherein the action of the one or more users comprises one or more of talking or singing by the one or more users during the presentation of the media content.

12 . The non-transitory computer readable storage medium of claim 1 , wherein the second classification indicates a reaction of the one or more users to the media content.

13 . The non-transitory computer readable storage medium of claim 1 , wherein causing the media player to modify the media content comprises adjusting one or more of a volume of (i) the media content and (ii) a playlist including the media content.

14 . A computer-implemented method comprising:

extracting audio information from a first audio signal, wherein the first audio signal is detected by a sensor during presentation of media content in an environment, and wherein the audio information corresponds to the media content;

extracting supplemental audio information from a second audio signal, wherein the second audio signal is detected by the sensor during the presentation of the media content in the environment, and wherein the supplemental audio information indicates an action by one or more users in the environment;

assigning a first classification to the first audio signal based on the audio information;

assigning a second classification to the second audio signal based on the supplemental audio information; and

causing a media player to modify the media content based on the second classification of the second audio signal.

15 . The computer-implemented method of claim 14 , wherein the second audio signal corresponds to the media content.

16 . The computer-implemented method of claim 14 , wherein a classification model is used to extract at least one of the audio information and the supplemental audio information.

17 . The computer-implemented method of claim 14 , wherein the first audio signal is a filtered audio signal and the second audio signal is an unfiltered audio signal.

18 . The computer-implemented method of claim 14 , wherein the audio information includes one or more of volume, pitch, energy, bandwidth, or zero crossing rates associated with the first audio signal.

19 . The computer-implemented method of claim 14 , wherein supplemental audio information further indicates environmental noise not associated with the one or more users.

20 . A computing system comprising:

one or more sensors;

one or more processors;

a non-transitory computer readable storage medium comprising instructions that, when executed, cause the one or more processors to perform a set of operations comprising:

extracting audio information from a first audio signal, wherein the first audio signal is detected by the one or more sensors during presentation of media content in an environment, and wherein the audio information corresponds to the media content;

extracting supplemental audio information from a second audio signal, wherein the second audio signal is detected by the one or more sensors during the presentation of the media content in the environment, and wherein the supplemental audio information indicates an action by one or more users in the environment;

assigning a first classification to the first audio signal based on the audio information;

assigning a second classification to the second audio signal based on the supplemental audio information; and

causing a media player to modify the media content based on the second classification of the second audio signal.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: JEYACHANDRAN, SURESH; BRENNER, VADIM; CREMER, MARKUS K.
To: GRACENOTE, INC.
Reel/Frame 065671/0606 →
Continuity (4)
Continuation 17333932 · May 28, 2021
Continuation 16530236 · Aug 2, 2019
Continuation 14147366 · Jan 3, 2014
Related Publication 20240062754A1 · Feb 22, 2024
References Cited (88)
US 6098040A · Petroni et al. · 2000 [cited by applicant]
US 6888457B2 · Wilkinson et al. · 2005 [cited by applicant]
US 7117149B1 · Zakarauskas · 2006 [cited by applicant]
US 7333618B2 · Shuttleworth et al. · 2008 [cited by applicant]
US 7903825B1 · Melanson · 2011 [cited by applicant]
US 8045726B2 · Kimura · 2011 [cited by applicant]
US 8607295B2 · Bhatia et al. · 2013 [cited by applicant]
US 10373611B2 · Jeyachandran et al. · 2019 [cited by applicant]
US 11024301B2 · Jeyachandran et al. · 2021 [cited by applicant]
US 20030018475A1 · Basu et al. · 2003 [cited by applicant]
US 20030044021A1 · Wilkinson et al. · 2003 [cited by applicant]
US 20030069727A1 · Krasny et al. · 2003 [cited by applicant]
US 20040064315A1 · Deisher et al. · 2004 [cited by applicant]
US 20040218768A1 · Zhurin et al. · 2004 [cited by applicant]
US 20050137861A1 · Diderkisen et al. · 2005 [cited by applicant]
US 20050175970A1 · Dunlap et al. · 2005 [cited by applicant]
US 20060178877A1 · Jiang · 2006 [cited by examiner]
US 20070005206A1 · Zhang et al. · 2007 [cited by applicant]
US 20070233743A1 · Rosenberg · 2007 [cited by applicant]
US 20080103781A1 · Wasson et al. · 2008 [cited by applicant]
US 20080190267A1 · Rechsteiner et al. · 2008 [cited by applicant]
US 20080269926A1 · Xiang et al. · 2008 [cited by applicant]
US 20090063975A1 · Bull et al. · 2009 [cited by applicant]
US 20090094630A1 · Brown · 2009 [cited by applicant]
US 20090172146A1 · Bates et al. · 2009 [cited by applicant]
US 20090248403A1 · Kinoshita et al. · 2009 [cited by applicant]
US 20100112991A1 · Hannaby · 2010 [cited by applicant]
US 20100114344A1 · Hannaby · 2010 [cited by applicant]
US 20110066438A1 · Lindahl et al. · 2011 [cited by applicant]
US 20130014136A1 · Bhatia et al. · 2013 [cited by applicant]
US 20130166042A1 · Sharma et al. · 2013 [cited by applicant]
US 20130179168A1 · Bae et al. · 2013 [cited by applicant]
US 20130211826A1 · Mannby · 2013 [cited by applicant]
US 20130243245A1 · Bill · 2013 [cited by applicant]
US 20130279457A1 · Takano · 2013 [cited by examiner]
US 20130282388A1 · Engdegard · 2013 [cited by applicant]
US 20130337889A1 · Gagner et al. · 2013 [cited by applicant]
US 20140194151A1 · Bengtsson · 2014 [cited by applicant]
US 20140288932A1 · Yeracaris et al. · 2014 [cited by applicant]
US 20140334644A1 · Selig et al. · 2014 [cited by applicant]
US 20150012267A1 · Rill et al. · 2015 [cited by applicant]
US 20150106404A1 · Boulter et al. · 2015 [cited by applicant]
US 20150147045A1 · Birnkrant · 2015 [cited by applicant]
US 20150170665A1 · Gundeti · 2015 [cited by examiner]
US 20190355350A1 · Jeyachandran et al. · 2019 [cited by applicant]
AT 414283 · 2006 [cited by applicant]
EP 2613313 · 2013 [cited by applicant]
WO 2015102921 · 2015 [cited by applicant]
European Patent Office, “Extended European Search Report,” issued in connection with European Patent Application No. 21177796.6, dated Jan. 19, 2022, 9 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Office Action,” issued in connection with U.S. Appl. No. 14/147,366, on Nov. 28, 2017, 10 pages. [cited by applicant]
International Searching Authority, “International Search Report,” issued in connection with application No. PCT/US14/71105, mailed on Mar. 27, 2015, 2 pages. [cited by applicant]
International Searching Authority, “Written Opinion,” issued in connection with application No. PCT/US14/71105, mailed on Mar. 27, 2015, 6 pages. [cited by applicant]
“European Application Serial No. 14876688.4, Response filed Feb. 20, 2017 to Communication purusant to Rules 161(2) and 162 EPC mailed Aug. 12, 2016”, 16 pages. [cited by applicant]
Foote, Jonathan, “An Overview of Audio Information Retrieval,” Institute of Systems Science of the National University of Singapore, Dec. 18, 1997, 18 pages. [cited by applicant]
European Patent Office, “Communication pursuant to Rules 161(2) and 162 EPC,” issued in connection with European Patent Application No. 148766884.4, dated Aug. 12, 2016, 3 pages. [cited by applicant]
International Bureau, “International Preliminary Report on Patentability,” issued in connection with International Patent Application No. PCT/US2014/071105, dated Jul. 14, 2016, 8 pages. [cited by applicant]
Shridhar, et al., “Text-independent speaker recognition: a review and some new results.” Speech Communication, Elsevier Science Publishers, Amsterdam, NL, vol. 1, No. 3-4, Dec. 1, 1982, 11 pages. [cited by applicant]
Susanne Schotz, “Acoustic Analysis of Adult Speaker Age,” Network and Parallel Computing, vol. 4343, Jan. 1, 2007, pp. 88-107, 21 pages. [cited by applicant]
Bishop, et al., “Perception of pitch location within a speaker's range: Fundamental frequency, voice quality and speaker sex,” The Journal of the Acoustical Society of America, American Institute of Physicis for the Aco… [cited by applicant]
Marc Zissman, “Automatic language identification using Gaussian mixture and hidden Markov models,” 2022 IEEE International Conference on Acoustics, Speech and Signal Processing, Proceedings. (ICASSP). Orlando, FL, May 1… [cited by applicant]
Gouyon, et al., “On the Use of Zero-Crossing Rate for an Application of Classification of Percussive Sounds,” Proceedings of the Cost G-6 Conference on Digital Audio Effects (DAFX-00), Verona, Italy, Dec. 9, 2000. 7 pag… [cited by applicant]
Won, et al., “Humming control interface for hand-held devices,” Computers and Accessibility, ACM, 2 Penn Plaza, Suite 701 New York NY 10121-0701, Oct. 15, 2007, 2 pages. [cited by applicant]
Adam J. Sporka, “Non-speech Sounds for User Interface Control,” Thesis submitted to the Faculty of Electrical Engineering, Czech Technical University in Prague, Apr. 1, 2008, 145 pages. [cited by applicant]
European Patent Office, “Extended Search Report”, issued in connection with European Patent Application No. 1487666.4, dated Aug. 10, 2017, 16 pages. [cited by applicant]
IP Australia, “Examination Report No. 1 for Standard Patent Application,” issued in connection with Australian Application No. 2014374183, dated Jun. 18, 2019, 3 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 14/147,366, on May 14, 2015, 14 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 14/147,366, on Jan. 22, 2016, 18 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 14/147,366, on Dec. 21, 2016, 19 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 14/147,366, on Aug. 4, 2017, 6 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 14/147,366, on Nov. 20, 2018, 18 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance” issued in connection with U.S. Appl. No. 14/147,366, on Apr. 28, 2017, 8 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance” issued in connection with U.S. Appl. No. 14/147,366, on Jul. 30, 2018, 10 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance” issued in connection with U.S. Appl. No. 14/147,366, on Mar. 25, 2019, 8 pages. [cited by applicant]
United States Patent and Trademark Office, Notice of Allowability issued in connection with U.S. Appl. No. 14/147,366, on Jul. 2, 2019, 3 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Office Action” issued in connection with U.S. Appl. No. 14/147,366, on Sep. 10, 2015, 23 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Office Action” issued in connection with U.S. Appl. No. 14/147,366, on Jun. 10, 2016, 19 pages. [cited by applicant]
IP Australia, “Notice of Acceptance,” issued in connection with application No. 2014374183, on Dec. 23, 2019, 3 pages. [cited by applicant]
European Patent Office, “Communication pursuant to Article 94(3) EPC,” issued in connection with European Patent Application No. 14876688.4, dated Apr. 1, 2020, 5 pages. [cited by applicant]
IP Australia, “Notice of Grant,” issued in connection with application No. 2014374183, on Apr. 30, 2020, 1 page. [cited by applicant]
IP Australia, “Examination Report” issued in connection with application No. 202020415, on Feb. 5, 2021, 3 pages. [cited by applicant]
European Patent Office, “Communication pursuant to Article 71(3) EPC,” issued in connection with European Patent Application No. 14876688.4, dated Jan. 26, 2021, 7 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action” issued in connection with U.S. Appl. No. 16/530,236, dated Oct. 13, 2020, 9 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance” issued in connection with U.S. Appl. No. 16/530,236, dated Jan. 27, 2021, 8 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance” issued in connection with U.S. Appl. No. 16/530,236, dated Feb. 18, 2021, 3 pages. [cited by applicant]
United States Patent and Trademark Office, Notice of Allowance issued in connection with U.S. Appl. No. 16/530,236, dated Feb. 23, 2021, 3 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance” issued in connection with U.S. Appl. No. 16/530,236, dated Apr. 30, 2021, 2 pages. [cited by applicant]
IP Australia, “Notice of Acceptance for patent application” issued in connection with application No. 202020415, on Jun. 21, 2021, 3 pages. [cited by applicant]
European Patent Office, “Invitation pursuant to Rule 63(1) EPC,” issued in connection with European Patent Application No. 21177796.6 dated Sep. 14, 2021, 4 pages. [cited by applicant]