IP Library Granted Patent US 11,086,591
Granted Patent B2
US 11,086,591 · App. 16/563,717 · Granted Aug 10, 2021

Methods and apparatus for dynamic volume adjustment via audio classification

Inventors: Markus Cremer (Orinda, CA); Robert Coover (Orinda, CA); Steven D. Scherf (Oakland, CA); Cameron Aubrey Summers (Oakland, CA)
Assignee: GRACENOTE, INC.
G06F3/165G10L19/00G10L25/30G10L25/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,086,591
App. No.
16/563,717
Granted
Aug 10, 2021
Kind
B2
Abstract

Methods, apparatus, systems and articles of manufacture are disclosed for dynamic volume adjustment via audio classification. Examples methods include analyzing, with a neural network trained model, a parameter of an audio signal associated with a first volume level to determine a classification group associated with the audio signal, determining an input volume of the audio signal, the selection based on the classification group associated with the audio signal, applying a gain value to the audio signal, the gain value based on the classification group and the input volume, the gain value to modify the first volume level to a second volume level, and applying a compression value to the audio signal, the compression value to modify the second volume level to a third volume level that satisfies a target volume threshold.

Claims (46)

1. An apparatus, comprising:

an audio signal classifier to analyze, with a neural network, a parameter of an audio signal associated with a first volume level to determine a classification group associated with the audio signal;

an input volume detector to determine an input volume of the audio signal;

a volume adjuster to:

determine a classification gain value based on the classification group;

determine an intermediate gain value as an intermediate between the input volume and the classification gain value by applying a first weight to the input volume and a second weight to the classification gain value; and

apply the intermediate gain value to the audio signal, the intermediate gain value to modify the first volume level to a second volume level; and

a dynamic range compressor to apply a compression value to the audio signal, the compression value to modify the second volume level to a third volume level that satisfies a target volume threshold.

2. The apparatus of claim 1 , further including a source change determiner to determine if a source of the audio signal has changed.

3. The apparatus of claim 2 , wherein the source change determiner determines if the source of the audio signal has changed is based on at least one of (1) a comparison of a current compressor gain associated with the audio signal to a previous compressor gain associated with the audio signal, (2) a comparison of a RMS power associated with the audio signal to a previous RMS power associated with the audio signal, or (3) a comparison of a current audio sample value associated with the audio signal to a previous audio sample value associated with the audio signal.

4. The apparatus of claim 2 , wherein the volume adjuster is further to, in response to determining the source of the audio signal has changed, reset the intermediate gain value of the audio signal.

5. The apparatus of claim 1 , wherein the classification group is associated with at least one of a (1) a genre of music represented by the audio signal, (2) a time period of the music represented by the audio signal, or (3) a presence of an instrument in the music represented by the audio signal.

6. The apparatus of claim 1 , wherein:

the input volume detector is further to determine a fourth volume level over a first time period does not fall within the target volume threshold, the first time period occurring after a second time period, the third volume level associated with the second time period; and

the dynamic range compressor is further to adjust the compression value to fifth volume level, the adjusted compression value to modify the fourth volume level to a fifth volume level that satisfies the target volume threshold.

7. The apparatus of claim 1 , wherein the target volume threshold is within five decibels relative to full scale (dBFS) to twenty-one dBFS.

8. The apparatus of claim 1 , wherein the first weight is greater than the second weight.

9. A non-transitory computer readable storage medium comprising instructions which, when executed, cause a processor to at least:

determine if a source of an audio signal has changed based on at least one of (1) a comparison of a current compressor gain associated with the audio signal to a previous compressor gain associated with the audio signal, or (2) a comparison of a RMS power associated with the audio signal to a previous RMS power associated with the audio signal; and

in response to the processor determining the source has changed:

analyze, with a neural network, a parameter of the audio signal associated with a first volume level to determine a classification group associated with the audio signal;

determine an input volume of the audio signal;

apply a gain value to the audio signal, the gain value based on the classification group and the input volume, the gain value to modify the first volume level to a second volume level; and

apply a compression value to the audio signal, the compression value to modify the second volume level to a third volume level that satisfies a target volume threshold.

10. The non-transitory computer readable storage medium of claim 9 , wherein the instructions, when executed, cause the processor to, in response to determining the source of the audio signal has changed, reset the gain value of the audio signal.

11. The non-transitory computer readable storage medium of claim 10 , wherein the classification group is associated with at least one of a (1) a genre of music represented by the audio signal, (2) a time period of the music represented by the audio signal, or (3) a presence of an instrument in the music represented by the audio signal.

12. The non-transitory computer readable storage medium of claim 9 , wherein the instructions, when executed, cause the processor to:

determine a fourth volume level over a first time period does not fall within the target volume threshold, the first time period occurring after a second time period, the third volume level associated with the second time period; and

adjust the compression value to fifth volume level, the adjusted compression value to modify the fourth volume level to a fifth volume level that satisfies the target volume threshold.

13. The non-transitory computer readable storage medium of claim 9 , wherein the target volume threshold is within five decibels relative to full scale (dBFS) to twenty-one dBFS.

14. The non-transitory computer readable storage medium of claim 9 , wherein the audio signal is an input to a media unit and the source changing corresponds to the input changing from a first media device to a second media device.

15. The non-transitory computer readable storage medium of claim 14 , wherein the media unit is a component of a car radio head unit.

16. A method, comprising:

analyzing, with a neural network, a parameter of an audio signal associated with a first volume level to determine a classification group associated with the audio signal, the audio signal from a source;

determining an input volume of the audio signal;

performing a volume adjustment to modify the first volume level to a second volume level, the volume adjustment based on the source;

applying a gain value to the audio signal, the gain value based on the classification group and the input volume, the gain value to modify the second volume level to a third volume level;

applying a compression value to the audio signal, the compression value to modify the third volume level to a fourth volume level that satisfies a target volume threshold;

determining if the source of the audio signal has changed; and

in response to determining the source of the audio signal has changed, resetting the gain value of the audio signal to zero.

17. The method of claim 16 , wherein the determining if the source of the audio signal has changed is based on at least one of (1) a comparison of a current compressor gain associated with the audio signal to a previous compressor gain associated with the audio signal, (2) a comparison of a RMS power associated with the audio signal to a previous RMS power associated with the audio signal, or (3) a comparison of a current audio sample value associated with the audio signal to a previous audio sample value associated with the audio signal.

18. The method of claim 16 , wherein the classification group is associated with at least one of a (1) a genre of music represented by the audio signal, (2) a time period of the music represented by the audio signal, or (3) a presence of an instrument in the music represented by the audio signal.

19. The method of claim 16 , further including:

determining a fifth volume level over a first time period does not fall within the target volume threshold, the first time period occurring after a second time period, the fourth volume level associated with the second time period; and

adjusting the compression value to modify the fifth volume level to a sixth volume level that satisfies the target volume threshold.

20. The method of claim 16 , wherein the audio signal is an input to a media unit and the source changing corresponds to input changing from a first media device to a second media device.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 5, 2021
From: CREMER, MARKUS; SCHERF, STEVEN D.; SUMMERS, CAMERON AUBREY; COOVER, ROBERT
To: GRACENOTE, INC.
Reel/Frame 056145/0596 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
Continuity (3)
Provisional Application 62728677 · Sep 7, 2018
Provisional Application 62745148 · Oct 12, 2018
Related Publication 20200081683A1 · Mar 12, 2020
Cited By (1)
US 12,609,110