Dynamic range and peak control in audio using nonlinear filters
An audio encoding device is described herein. The audio encoding device includes a compressor that is based on a nonlinear filter. In particular, the nonlinear filter may be selected from the class of edge-preserving smoothing filters, which avoids common artifacts of conventional compressors. Edge-preserving smoothing filters have been used in image processing algorithms for their de-noising properties while preserving edges in the image. These properties are useful for audio compression because macro-dynamic loudness changes can be tracked precisely while micro-dynamic loudness changes can be ignored for the compression. Due to these advantages, more aggressive compression can be achieved with less distortion.
1. A method for encoding an audio signal, comprising:
processing samples of the audio signal using a loudness model to determine the instantaneous loudness of the audio signal for a set of the samples;
processing the instantaneous loudness using a primary nonlinear filter to generate a smoothed loudness wherein the primary nonlinear filter performs a time domain-only filter process upon a continuous sequence of instantaneous loudness samples to generate the smooth loudness;
mapping the smoothed loudness to a dynamic range compression gain; and
combining the dynamic range compression gain with the audio signal to generate a compressed audio signal.
2. The method of claim 1 , wherein the primary nonlinear filter is an edge-preserving smoothing filter.
3. The method of claim 1 , wherein the primary nonlinear filter is one of (1) a median filter; (2) a bilateral filter; (3) a guided filter; (4) a weighted least squares filter; and (5) an anisotropic diffusion filter.
4. The method of claim 1 , wherein when the smoothed loudness is above a threshold value, the dynamic range compression gain is set at a first level and when the smoothed loudness is below the threshold value, the dynamic range compression gain is set at a second level, wherein the first level is below the second level.
5. The method of claim 1 , further comprising:
(1) limiting the instantaneous loudness to a maximum loudness threshold value and (2) distorting the limited instantaneous loudness using a mapping function wherein processing the instantaneous loudness using the primary nonlinear filter comprises processing the distorted and limited instantaneous loudness.
6. The method of claim 1 , further comprising:
down-sampling the instantaneous loudness prior to processing by the primary nonlinear filter; and
up-sampling the dynamic range compression gain prior to combination with the audio signal.
7. The method of claim 1 , wherein the primary nonlinear filter is a median filter, the method further comprising:
evaluating a local extremum of a time-derivative of the smoothed loudness to classify the audio signal into separate content segments, wherein the extremum indicates where a maximum slope steepness is reached, wherein a boundary between two segments is indicated when a magnitude of a steepness of the time-derivative of the smoothed loudness exceeds a steepness threshold.
8. The method of claim 1 , further comprising:
processing the instantaneous loudness using a secondary nonlinear filter and an adaptation controller to determine a duration of a peaks or a valley in the instantaneous loudness;
reducing a size of the primary nonlinear filter when the determined duration is below a filter threshold; and
slowing decay to the instantaneous loudness when the determined duration is below a decay threshold.
9. The method of claim 8 , wherein the secondary nonlinear filter is a median filter.
10. An audio encoding device for encoding an audio signal, comprising:
a loudness model to process samples of the audio signal to determine the instantaneous loudness of the audio signal for a set of the samples;
a primary nonlinear filter to process the instantaneous loudness to generate a smoothed loudness wherein the primary nonlinear filter performs a time domain-only filter process upon a continuous sequence of instantaneous loudness samples to generate the smooth loudness; and
a primary mapping unit to map the smoothed loudness to a dynamic range compression gain.
11. The audio encoding device of claim 10 , further comprising:
a multiplier to combine the dynamic range compression gain with the audio signal to generate a compressed audio signal.
12. The audio encoding device of claim 10 , wherein the dynamic range compression gain is added as metadata to the audio signal before being transmitted to an audio playback device.
13. The audio encoding device of claim 10 , wherein the primary nonlinear filter is an edge-preserving smoothing filter.
14. The audio encoding device of claim 10 , wherein the primary nonlinear filter is one of (1) a median filter; (2) a bilateral filter; (3) a guided filter; (4) a weighted least squares filter; and (5) an anisotropic diffusion filter.
15. The audio encoding device of claim 10 , wherein when the smoothed loudness is above a threshold value, the dynamic range compression gain is set at a first level and when the smoothed loudness is below the threshold value, the dynamic range compression gain is set at a second level, wherein the first level is below the second level.
16. The audio encoding device of claim 11 , further comprising:
a maximum value unit to reduce peaks in the compressed audio signal by limiting the instantaneous loudness to a maximum loudness threshold value.
17. The audio encoding device of claim 10 , further comprising:
a down-sampling unit to down-sample the instantaneous loudness prior to processing by the primary nonlinear filter; and
an up-sampling unit to up-sample the output of the primary mapping unit that produces the dynamic range compression gain.
18. The audio encoding device of claim 10 , further comprising:
a secondary nonlinear filter and an adaptation controller to (1) process the instantaneous loudness to determine a duration of a peak or a valley in the instantaneous loudness and (2) reduce size of the primary nonlinear filter when the determined duration is below a filter threshold; and
a decay generator to slow decay to the instantaneous loudness when the determined duration is below a decay threshold.
19. The audio encoding device of claim 18 , wherein the secondary nonlinear filter is a median filter.
20. A non-transitory computer readable medium that stores instructions that when executed by a processor of an audio encoding device cause the audio encoding device to:
process samples of an audio signal using a loudness model to determine the instantaneous loudness of the audio signal for a set of the samples;
process the instantaneous loudness using a primary nonlinear filter to generate a smoothed loudness wherein the primary nonlinear filter performs a time domain-only filter process upon a continuous sequence of instantaneous loudness sample to generate the smooth loudness;
map the smoothed loudness to a dynamic range compression gain; and
combine the dynamic range compression gain with the audio signal to generate a compressed audio signal.
21. The non-transitory computer readable medium of claim 20 , wherein the computer readable medium stores further instructions that when executed by the processor of the audio encoding device cause the audio encoding device to:
process the instantaneous loudness using a secondary nonlinear filter and an adaptation controller to determine a duration of a peak or a valley in the instantaneous loudness; and
slow decay to the instantaneous loudness when the determined duration is below a decay threshold.
22. The non-transitory computer readable medium of claim 20 , wherein the computer readable medium stores further instructions that when executed by the processor of the audio encoding device cause the audio encoding device to:
add the dynamic range compression gain as metadata to the audio signal.
23. The non-transitory computer readable medium of claim 20 , wherein the nonlinear filter is an edge-preserving smoothing filter.
24. The non-transitory computer readable medium of claim 20 , wherein the nonlinear filter is one of (1) a median filter; (2) a bilateral filter; (3) a guided filter; (4) a weighted least squares filter; and (5) an anisotropic diffusion filter.