Virtual bass synthesis using harmonic transposition
In some embodiments, a virtual bass generation method including steps of: performing harmonic transposition on low frequency components of an input audio signal (typically, bass frequency components expected to be inaudible during playback of the input audio signal using an expected speaker or speaker set) to generate transposed data indicative of harmonics (which are expected to be audible during playback, using the expected speaker(s), of an enhanced version of the input audio which includes the harmonics); generating an enhancement signal in response to the transposed data; and generating an enhanced audio signal by combining (e.g., mixing) the enhancement signal with the input audio signal. Other aspects are systems (e.g., programmed processors) and devices (e.g., devices having physically-limited bass reproduction capabilities, such as, for example, a notebook, tablet, mobile phone, or other device with small speakers) configured to perform any embodiment of the method.
1. A virtual bass generation method, including a step of preprocessing samples of a sub-banded, CQMF (complex-valued quadrature mirror filter) input audio signal to generate critically sampled audio indicative of content of a set of low frequency sub-bands of the input audio signal, and including steps of:
(a) performing harmonic transposition on the critically sampled audio to generate transposed data indicative of harmonics, wherein the harmonics are expected to be audible during playback of an enhanced version of the input audio which includes said harmonics;
(b) generating an enhancement signal in response to the transposed data; and
(c) generating an enhanced audio signal by combining the enhancement signal with the input audio signal,
wherein the harmonic transposition performed in step (a) employs combined transposition such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and such that all of the harmonics are generated in response to frequency-domain values determined by a single, common time-to-frequency domain transform stage, and a subsequent inverse transform determined by a single, common frequency-to-time domain transform stage is performed.
2. The method of claim 1 , wherein the input audio signal is indicative of low frequency audio content in a range from 0 to B Hz, where B is a number less than 500, and the critically sampled audio is an at least substantially critically sampled signal indicative of the low frequency audio content.
3. The method of claim 1 , also including the step of generating the critically sampled audio by performing a frequency domain oversampled transform on the input audio signal, by generating windowed, zero-padded samples, and performing a time-to-frequency domain transform on the windowed, zero-padded samples to generate said critically sampled audio.
4. The method of claim 1 , wherein the enhanced audio signal provides an increased perceived level of bass content during playback of said enhanced audio signal by at least one loudspeaker that cannot physically reproduce the low frequency components.
5. The method of claim 1 , also including a step of playback of the enhanced audio signal by loudspeakers that cannot physically reproduce the low frequency components.
6. The method of claim 1 , wherein the low frequency components of the input audio signal are bass frequency components expected to be inaudible during playback of the input audio signal using an expected speaker or speaker set.
7. The method of claim 1 , wherein the transposed data are indicative of amplitude modified versions of said harmonics.
8. The method of claim 7 , wherein the transposed data are amplitude modified versions of the harmonics whose values are determined at least approximately by Equal Loudness Contours (ELCs).
9. The method of claim 1 , wherein step (a) includes a step of attenuating the harmonics in a manner determined by a tonality metric to determine the transposed data.
10. The method of claim 1 , wherein at least one of steps (a) and (b) includes a step of attenuating data indicative of the harmonics in accordance with a control function, wherein the control function determines a gain to be applied to each frequency sub-band of the transposed data.
11. A virtual bass generation method, including a step of preprocessing samples of a sub-banded, CQMF (complex-valued quadrature mirror filter) input audio signal to generate critically sampled audio indicative of content of a set of low frequency sub-bands of the input audio signal, and including steps of:
(a) performing harmonic transposition on the critically sampled audio to generate transposed data indicative of harmonics, wherein the harmonics are expected to be audible during playback of an enhanced version of the input audio which includes said harmonics;
(b) generating an enhancement signal in response to the transposed data; and
(c) generating an enhanced audio signal by combining the enhancement signal with the input audio signal,
wherein the harmonic transposition performed in step (a) employs combined transposition such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and such that all of the harmonics are generated in response to frequency-domain values determined by a single, common time-to-frequency domain transform stage, and a subsequent inverse transform determined by a single, common frequency-to-time domain transform stage is performed,
and wherein the critically sampled audio is a CQMF channel 0 signal, and the enhancement signal generated in step (b) includes a CQMF channel 0 enhancement signal and a CQMF channel 1 enhancement signal.
12. A virtual bass generation method, including steps of:
generating low frequency components of an input audio signal by performing a frequency domain oversampled transform on the input audio signal, by generating windowed, zero-padded samples, and performing a time-to-frequency domain transform on the windowed, zero-padded samples to generate said low frequency components;
(a) performing harmonic transposition on the low frequency components of the input audio signal to generate transposed data indicative of harmonics, wherein the harmonics are expected to be audible during playback of an enhanced version of the input audio which includes said harmonics;
(b) generating an enhancement signal in response to the transposed data; and
(c) generating an enhanced audio signal by combining the enhancement signal with the input audio signal,
wherein the harmonic transposition performed in step (a) employs combined transposition such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and such that all of the harmonics are generated in response to frequency-domain values determined by a single, common time-to-frequency domain transform stage, and a subsequent inverse transform determined by a single, common frequency-to-time domain transform stage is performed, and wherein step (b) includes a step of splitting processed frequency components into a first set of frequency components in a first frequency band and a second set of frequency components in a second frequency band, and performing a first frequency-to-time domain transform on the first set of frequency components and a second frequency-to-time domain transform on the second set of frequency components, wherein each of the first frequency-to-time domain transform and the second frequency-to-time domain transform has block size smaller than does the time-to-frequency domain transform.
13. The method of claim 12 , wherein the first frequency band is the frequency band of CQMF channel 0 , and the second frequency band is the frequency band of CQMF channel 1 .
14. The method of claim 13 , wherein the first set of frequency components and the second set of frequency components are magnitude compensated to account for CQMF channel 0 and CQMF channel 1 frequency responses, respectively.
15. A virtual bass generation method, including steps of:
(a) performing harmonic transposition on low frequency components of an input audio signal to generate transposed data indicative of harmonics, wherein the harmonics are expected to be audible during playback of an enhanced version of the input audio which includes said harmonics;
(b) generating an enhancement signal in response to the transposed data; and
(c) generating an enhanced audio signal by combining the enhancement signal with the input audio signal,
wherein the harmonic transposition performed in step (a) employs combined transposition such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and such that all of the harmonics are generated in response to frequency-domain values determined by a single, common time-to-frequency domain transform stage, and a subsequent inverse transform determined by a single, common frequency-to-time domain transform stage is performed, and wherein the time-to-frequency domain transform and the inverse transform use asymmetric analysis and synthesis windows.
16. A virtual bass generation method, including steps of:
(a) performing harmonic transposition on low frequency components of an input audio signal to generate transposed data indicative of harmonics, wherein the harmonics are expected to be audible during playback of an enhanced version of the input audio which includes said harmonics;
(b) generating an enhancement signal in response to the transposed data; and
(c) generating an enhanced audio signal by combining the enhancement signal with the input audio signal,
wherein the harmonic transposition performed in step (a) employs combined transposition such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and such that all of the harmonics are generated in response to frequency-domain values determined by a single, common time-to-frequency domain transform stage, and a subsequent inverse transform determined by a single, common frequency-to-time domain transform stage is performed, wherein at least one of steps (a) and (b) includes a step of attenuating data indicative of the harmonics in accordance with a control function, wherein the control function determines a gain to be applied to each frequency sub-band of the transposed data, and
wherein the control function determines a gain, g(b), to be applied to harmonic coefficients in frequency sub-band b, and has form: g(b)=H[(G·nrg orig (b)−nrg vb (b))/(G·nrg orig (b)+nrg vb (b))]+B,
where H, G and B are constants, nrg orig (b) is indicative of energy of the input audio signal in the sub-band b, and nrg vb (b) is indicative of energy of the transposed data or the enhancement signal in the sub-band b.
17. A virtual bass generation system, including:
a preprocessing stage coupled to receive an input audio signal, and configured to generate critically sampled audio indicative of low frequency components of the input audio signal;
a harmonic transposition stage coupled and configured to perform harmonic transposition on the critically sampled audio to generate transposed data indicative of harmonics, wherein the harmonics are expected to be audible during playback of an enhanced version of the input audio which includes said harmonics;
an enhancement signal generation stage coupled and configured to generate an enhancement signal in response to the transposed data; and
an enhanced audio signal generation stage coupled and configured to generate an enhanced audio signal by combining the enhancement signal with the input audio signal,
wherein the harmonic transposition stage includes a single time-to-frequency domain transform stage and a single frequency-to-time domain transform stage, and is configured to perform the harmonic transposition by employing combined transposition such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and all of the harmonics are generated in response to frequency-domain values determined by the time-to-frequency domain transform stage, wherein the input audio signal is a sub-banded, CQMF (complex-valued quadrature mirror filter) signal, and the critically sampled audio is indicative of content of a set of low frequency sub-bands of the CQMF signal.
18. The system of claim 17 , wherein the input audio signal is indicative of low frequency audio content in a range from 0 to B Hz, where B is a number less than 500), and the critically sampled audio is an at least substantially critically sampled signal indicative of the low frequency audio content.
19. The system of claim 17 , wherein the
preprocessing stage is coupled and configured to perform frequency domain oversampling on the input audio signal, including by generating windowed, zero-padded samples, and performing a time-to-frequency domain transform stage on the windowed, zero-padded samples to generate said critically sampled audio.
20. The system of claim 17 , wherein the enhanced audio signal provides an increased perceived level of bass content during playback of said enhanced audio signal by at least one loudspeaker that cannot physically reproduce the low frequency components.
21. The system of claim 17 , also including:
a playback subsystem including at least one loudspeaker that cannot physically reproduce the low frequency components, wherein the playback subsystem is coupled and configured to generate at least one speaker feed for the at least one loudspeaker in response to the enhanced audio signal.
22. The system of claim 17 , wherein the transposed data are indicative of amplitude modified versions of said harmonics.
23. The system of claim 22 , wherein the transposed data are amplitude modified versions of the harmonics whose values are determined at least approximately by Equal Loudness Contours (ELCs).
24. The system of claim 17 , wherein the harmonic transposition stage is configured to attenuate the harmonics in a manner determined by a tonality metric to determine the transposed data.
25. The system of claim 17 , wherein at least one stage of said system is configured to attenuate data indicative of the harmonics in accordance with a control function, wherein the control function determines a gain to be applied to each frequency sub-band of the transposed data.
26. The system of claim 17 , wherein said system is a processor programmed to implement the harmonic transposition stage, the enhancement signal generation stage, and the enhanced audio signal generation stage.
27. The system of claim 17 , wherein said system includes a processor programmed to implement the harmonic transposition stage, the enhancement signal generation stage, and the enhanced audio signal generation stage.
28. The system of claim 17 , wherein said system is a digital signal processor configured to implement the harmonic transposition stage, the enhancement signal generation stage, and the enhanced audio signal generation stage.
29. The system of claim 17 , wherein said system includes a digital signal processor configured to implement the harmonic transposition stage, the enhancement signal generation stage, and the enhanced audio signal generation stage.
30. The system of claim 17 , including a processing subsystem configured to implement the harmonic transposition stage, the enhancement signal generation stage, and the enhanced audio signal generation stage, and also including:
a playback subsystem including at least one loudspeaker that cannot physically reproduce the low frequency components, wherein the playback subsystem is coupled and configured to generate at least one speaker feed for the at least one loudspeaker in response to the enhanced audio signal.
31. A virtual bass generation system, including:
a harmonic transposition stage coupled and configured to perform harmonic transposition on low frequency components of an input audio signal to generate transposed data indicative of harmonics, wherein the harmonics are expected to be audible during playback of an enhanced version of the input audio which includes said harmonics;
an enhancement signal generation stage coupled and configured to generate an enhancement signal in response to the transposed data; and
an enhanced audio signal generation stage coupled and configured to generate an enhanced audio signal by combining the enhancement signal with the input audio signal,
wherein the harmonic transposition stage includes a single time-to-frequency domain transform stage and a single frequency-to-time domain transform stage, and is configured to perform the harmonic transposition by employing combined transposition such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and all of the harmonics are generated in response to frequency-domain values determined by the time-to-frequency domain transform stage, and
wherein one of the harmonic transposition stage and the enhancement signal generation stage includes a frequency-to-time domain transform stage, and said time-to-frequency domain transform stage and said frequency-to-time domain transform stage use asymmetric analysis and synthesis windows.
32. A virtual bass generation system, including:
a harmonic transposition stage coupled and configured to perform harmonic transposition on low frequency components of an input audio signal to generate transposed data indicative of harmonics, wherein the harmonics are expected to be audible during playback of an enhanced version of the input audio which includes said harmonics;
an enhancement signal generation stage coupled and configured to generate an enhancement signal in response to the transposed data; and
an enhanced audio signal generation stage coupled and configured to generate an enhanced audio signal by combining the enhancement signal with the input audio signal,
wherein the harmonic transposition stage includes a single time-to-frequency domain transform stage and a single frequency-to-time domain transform stage, and is configured to perform the harmonic transposition by employing combined transposition such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and all of the harmonics are generated in response to frequency-domain values determined by the time-to-frequency domain transform stage, and
wherein the low frequency components of the input audio signal are determined by a CQMF channel 0 signal, and the enhancement signal includes a CQMF channel 0 enhancement signal and CQMF channel 1 enhancement signal.
33. A virtual bass generation system, including:
a harmonic transposition stage coupled and configured to perform harmonic transposition on low frequency components of an input audio signal to generate transposed data indicative of harmonics, wherein the harmonics are expected to be audible during playback of an enhanced version of the input audio which includes said harmonics;
an enhancement signal generation stage coupled and configured to generate an enhancement signal in response to the transposed data;
an enhanced audio signal generation stage coupled and configured to generate an enhanced audio signal by combining the enhancement signal with the input audio signal; and
a frequency domain oversampled transform stage, coupled and configured to perform frequency domain oversampling on the input audio signal, by generating windowed, zero-padded samples, and performing a time-to-frequency domain transform stage on the windowed, zero-padded samples to generate said low frequency components,
wherein the harmonic transposition stage includes a single time-to-frequency domain transform stage and a single frequency-to-time domain transform stage, and is configured to perform the harmonic transposition by employing combined transposition such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and all of the harmonics are generated in response to frequency-domain values determined by the time-to-frequency domain transform stage, and
wherein the enhancement signal generation stage is configured to split processed frequency components into a first set of frequency components in a first frequency band and a second set of frequency components in a second frequency band, and to perform a first frequency-to-time domain transform on the first set of frequency components and a second frequency-to-time domain transform on the second set of frequency components, wherein each of the first frequency-to-time domain transform and the second frequency-to-time domain transform has block size smaller than does the time-to-frequency domain transform.
34. The system of claim 33 , wherein the first frequency band is the frequency band of CQMF channel 0 , and the second frequency band is the frequency band of CQMF channel 1 .
35. The system of claim 34 , wherein the enhancement signal generation stage is configured to perform magnitude compensation on the first set of frequency components and the second set of frequency components to account for CQMF channel 0 and CQMF channel 1 frequency responses, respectively.
36. A virtual bass generation system, including:
a harmonic transposition stage coupled and configured to perform harmonic transposition on low frequency components of an input audio signal to generate transposed data indicative of harmonics, wherein the harmonics are expected to be audible during playback of an enhanced version of the input audio which includes said harmonics;
an enhancement signal generation stage coupled and configured to generate an enhancement signal in response to the transposed data; and
an enhanced audio signal generation stage coupled and configured to generate an enhanced audio signal by combining the enhancement signal with the input audio signal,
wherein the harmonic transposition stage includes a single time-to-frequency domain transform stage and a single frequency-to-time domain transform stage, and is configured to perform the harmonic transposition by employing combined transposition such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and all of the harmonics are generated in response to frequency-domain values determined by the time-to-frequency domain transform stage,
wherein at least one stage of said system is configured to attenuate data indicative of the harmonics in accordance with a control function, wherein the control function determines a gain to be applied to each frequency sub-band of the transposed data, and
wherein the control function determines a gain, g(b), to be applied to harmonic coefficients in frequency sub-band b, and has form:
g ( b )= H [( G·nrg orig ( b )− nrg vb ( b ))/( G·nrg orig ( b )+ nrg vb ( b ))]+ B,
where H, G and B are constants, nrg orig (b) is indicative of energy of the input audio signal in the sub-band b, and nrg vb (b) is indicative of energy of the input audio signal in the sub-band b, and nrgVB(b) is indicative of energy of the transposed data or the enhancement signal in the sub-band b.