Latency reduction in transposer-based virtual bass systems
A latency reduction system in a virtual bass processing system performs harmonic transposition on low frequency components of an audio signal to generate transposed data indicative of harmonics of the audio signal. The system uses a base transposition factor greater than two, and generates the harmonics in response to frequency-domain values determined by forward and inverse transform stages that use asymmetric analysis and synthesis windows. The system combines a virtual bass signal with the delayed wide band audio signal through analysis filter banks having filter coefficient truncated Nyquist filters. The virtual bass signal may lag the delayed wide band audio signal when combining with the audio signal to further reduce the latency caused by the harmonic transposition. The virtual bass input signal may be directly routed from a CQMF analysis filter bank of a preceding Hybrid filter bank stage, in order to avoid the delay associated with a Nyquist filter bank.
1. A method of generating low latency virtual bass, comprising:
receiving an input audio signal;
performing harmonic transposition on low frequency components of the input audio signal to generate transposed data indicative of harmonics of the input audio signal;
generating a virtual bass signal in response to the transposed data; and
generating an enhanced audio signal by combining the virtual bass signal with a delayed version of the input audio signal, wherein the harmonic transposition employs combined transposition using a base transposition order B higher than 2 such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and such that all of the harmonics are generated in response to frequency-domain values determined by a common time-to-frequency domain transform stage using an asymmetric analysis window, and a subsequent inverse transform determined by a common frequency-to-time domain transform stage using an asymmetric synthesis window.
2. The method of claim 1 wherein the base transposition factor B is an integer value selected from the group consisting of: 4, 8, 16 or 32.
3. The method of claim 1 wherein the input audio signal is a sub-band complex-valued quadrature mirror filter (CQMF) signal indicative of critically sampled or close to critically sampled low frequency audio from a set of CQMF sub-band signals.
4. The method of claim 3 wherein the critically sampled or close to critically sampled low frequency input audio is a CQMF channel 0 signal indicative of the lowest frequency band from a set of CQMF sub-band signals.
5. The method of claim 4 further comprising:
generating transposed data from low frequency components by performing a frequency domain oversampled transform on the input audio signal by generating asymmetrically windowed, zero-padded samples, and performing a time-to-frequency domain transform on the asymmetrically windowed, zero-padded samples, and subsequently performing a non-linear operation on the output from the time-to-frequency domain transform to generate the transposed data from the low frequency components;
generating two sets of frequency components from the frequency components processed by the non-linear operation by splitting into a first set of frequency components in a first frequency band and a second set of frequency components in a second frequency band; and
further performing a first frequency-to-time domain transform on the first set of frequency components and a second frequency-to-time domain transform on the second set of frequency components, wherein each of the first frequency-to-time domain transform and the second frequency-to-time domain transform have transform sizes B times smaller than the time-to-frequency domain transform; and
further applying asymmetric zero-padded windows to the samples from the frequency-to-time domain transforms, wherein the asymmetric zero-padded windows are B times shorter than the asymmetrically windowed, zero-padded samples generated from the input audio signal, thus forming two sets of transposed data.
6. The method of claim 5 wherein the first frequency band is the frequency band of CQMF channel 0 , and the second frequency band is the frequency band of CQMF channel 1 from a set of CQMF sub-band signals.
7. The method of claim 6 wherein generating a virtual bass signal in response to the transposed data comprises an analysis filter bank applied to one or both of the two sets of transposed data, wherein the analysis filter bank comprises a truncated version of a symmetric filter.
8. The method of claim 7 wherein the analysis filter bank is a Nyquist filter bank and the truncated version of a symmetric filter is a filter where one of the symmetric halves of the filter is removed.
9. The method of claim 8 wherein the analysis filter bank comprises one of an eight-channel Nyquist filter bank or a four-channel Nyquist filter bank, and wherein one of the removed symmetric halves of the filter comprises six coefficients.
10. The method of claim 1 wherein the delayed version of the input audio signal is delayed a pre-defined time period shorter than the latency of the virtual bass signal and the enhanced audio signal is indicative of a time lagged virtual bass signal.
11. The method of claim 10 wherein the pre-defined time period is a value selected from the range of 0 samples to 1000 samples.
12. The method of claim 4 wherein the input audio CQMF channel 0 is received directly from the analysis CQMF bank output of a pre-processing Hybrid filter bank stage, bypassing the Nyquist analysis filter bank of the pre-processing Hybrid filter bank stage.
13. An apparatus for generating low latency virtual bass, comprising:
a first component receiving an input audio signal and performing harmonic transposition on low frequency components of the input audio signal to generate transposed data indicative of harmonics of the input audio signal; and
a second component generating a virtual bass signal in response to the transposed data and combining the virtual bass signal with a delayed version of the input audio signal to generate an enhanced audio signal, wherein the harmonic transposition employs combined transposition using a base transposition order B higher than 2 such that the harmonics include a second order harmonic and at least one higher order harmonic of each of the low frequency components, and such that all of the harmonics are generated in response to frequency-domain values determined by a common time-to-frequency domain transform stage using an asymmetric analysis window, and a subsequent inverse transform determined by a common frequency-to-time domain transform stage using an asymmetric synthesis window.
14. The apparatus of claim 13 wherein the base transposition factor B is an integer value selected from the group consisting of: 4, 8, 16 or 32.
15. The apparatus of claim 13 wherein the input audio signal is a sub-band complex-valued quadrature minor filter (CQMF) signal indicative of critically sampled or close to critically sampled low frequency audio from a set of CQMF sub-band signals.
16. The apparatus of claim 15 wherein the critically sampled or close to critically sampled low frequency audio is a CQMF channel 0 signal indicative of the lowest frequency band from a set of CQMF sub-band signals.
17. The apparatus of claim 16 further comprising:
a third component generating transposed data from low frequency components by performing a frequency domain oversampled transform on the input audio signal by generating asymmetrically windowed, zero-padded samples, and performing a time-to-frequency domain transform on the asymmetrically windowed, zero-padded samples, and subsequently performing a non-linear operation on the output from the time-to-frequency domain transform to generate the transposed data from the low frequency components;
a fourth component generating two sets of frequency components from the frequency components processed by the non-linear operation by splitting into a first set of frequency components in a first frequency band and a second set of frequency components in a second frequency band;
a fifth component further performing a first frequency-to-time domain transform on the first set of frequency components and a second frequency-to-time domain transform on the second set of frequency components, wherein each of the first frequency-to-time domain transform and the second frequency-to-time domain transform have transform sizes B times smaller than the time-to-frequency domain transform; and
a sixth component applying asymmetric zero-padded windows to the samples from the frequency-to-time domain transforms, wherein the asymmetric zero-padded windows are B times shorter than the asymmetrically windowed, zero-padded samples generated from the input audio signal, thus forming two sets of transposed data.
18. The apparatus of claim 17 wherein the first frequency band is the frequency band of CQMF channel 0 , and the second frequency band is the frequency band of CQMF channel 1 from a set of CQMF sub-band signals, and wherein generating a virtual bass signal in response to the transposed data comprises an analysis filter bank applied to one or both of the two sets of transposed data, wherein the analysis filter bank comprise a truncated version of a symmetric filter.
19. The apparatus of claim 18 wherein the analysis filter bank is a Nyquist filter bank and the truncated version of a symmetric filter is a filter where one of the symmetric halves of the filter is removed.
20. The apparatus of claim 19 wherein the analysis filter bank comprises one of an eight-channel Nyquist filter bank or a four-channel Nyquist filter bank, and wherein one of the removed symmetric halves of the filter comprises six coefficients.
21. The apparatus of claim 13 further comprising:
a timing component generating a version of the input audio signal delayed a pre-defined time period shorter than the latency of the virtual bass signal; and
a mixing component combining the virtual bass signal with the delayed input audio signal to generate an enhanced audio signal indicative of a time lagged virtual bass signal.
22. The apparatus of claim 16 further comprising an interface component receiving the CQMF channel 0 directly from the analysis CQMF bank output of a pre-processing Hybrid filter bank, bypassing the Nyquist analysis filter bank of the pre-processing Hybrid filter bank stage.