Methods and devices for audio upmixing
View Patent ↗At least one exemplary embodiment is directed to a new spatial audio enhancing system including a novel Adaptive Sound Upmixing System (ASUS). In some specific embodiments the ASUS provided converts a two-channel recording into an audio signal including four channels that can be played over four different loudspeakers. In other specific embodiments the ASUS provided converts a two-channel recording into an audio signal including five channels that can be played over five different loudspeakersn even other specific embodiments the ASUS provided converts a five-channel recording (such as those for DVD's) into an audio signal including eight channels that can be played over eight different loudspeakers. More generally, in view of this disclosure those skilled in the art will be able to adapt the ASUS to process and provide an arbitrary number of audio channels both at the input and the output.
1. A method of up-mixing audio signals comprising:
determining a level of panning between first and second audio signals, wherein the level of panning is considered hard if the first and second audio signals are essentially uncorrelated;
introducing cross-talk between the first and second audio signals thereby generating two new first and second audio signals;
filtering the first new audio signal, where the filtering includes use of a first set of filtering coefficients generating a filtered first audio signal;
time-shifting the second new audio signal, where the second new audio signal is time-shifted with respect to the filtered first audio signal generating a shifted second audio signal;
determining a first difference between the filtered first audio signal and the shifted second audio signal, where the first difference is an up-mixed audio signal;
adjusting the first set of filtering coefficients so that the first difference is essentially orthogonal to the first audio signal; and
wherein introducing cross-talk comprises adjusting the introduced cross-talk based on the determined level of panning to improve up-mixing quality.
2. A method according to claim 1 wherein each of the first and second audio signals includes a source image component and a reverberance image component and wherein at least two of the respective source image components are correlated with one another.
3. A method according to claim 2 wherein the first and second audio signals include a left front channel and a right front channel and the first difference corresponds to a left rear channel including some portion of the respective reverberance image of the left front and right front channels.
4. A method according to claim 1 further comprising:
filtering the second new audio signal, where the filtering includes use of a second set of filtering coefficients generating a filtered second audio signal;
time-shifting the first new audio signal, where the first new audio signal is time-shifted with respect to the filtered second audio signal generating a shifted first audio signal;
determining a second difference between the filtered second audio signal and the shifted first audio signal; and
adjusting the second set of filtering coefficients so that the second difference is essentially orthogonal to the second audio signal.
5. A method according to claim 1 wherein the first and second audio signals are adjacent audio channels.
6. A method according to claim 1 wherein the time-shifting includes one of delaying or advancing the first audio signal with respect to the second audio signal.
7. A method according to claim 6 wherein a time-shift value is in the range of about 2 ms to about 10 ms.
8. A method according to claim 1 wherein the filtering of the first audio signal includes equalizing the first audio signal so that the first difference is minimized.
9. A method according to claim 8 wherein the first set of filtering coefficients is adjusted according to one of the Least Means Squares (LMS) method or Normalized LMS (NLMS) method.
10. A device configured to up-mix audio signals comprising:
at least one audio signal input;
a hard panning detector;
a cross-talk inducer; and
a computer-readable medium including a computer program, the computer program comprising:
employing the hard panning detector for determining a level of panning between first and second audio signals, wherein the level of panning is considered hard if the first and second audio signals are essentially uncorrelated;
employing the cross-talk inducer to iniect cross-talk into at least one of the first and second audio signals thereby generating two new first and second audio signals;
filtering the first new audio signal, where the filtering includes using a first set of filtering coefficients to generate a filtered first audio signal;
time-shifting the second new audio signal, where the second new audio signal is time shifted with respect to the filtered first audio signal generating shifted second audio signal;
determining a first difference between the filtered first audio signal and the shifted second audio signal, wherein the first difference is an up-mixed audio signal; and
adjusting the first set of filtering coefficients so that the first difference is essentially orthogonal to the first audio signal, where the first and second audio signals are obtained through the at least one audio signal input, and where the at least one audio signal input and the computer-readable medium are operatively connected;
wherein the cross-talk inducer adjusts the introduced cross-talk to improve upmixing quality based on the level of panning between first and second audio signals determined by the hard panning detector.
11. The device according to claim 10 wherein the first and second audio signals include a left front channel and a right front channel and the first difference corresponds to a left rear channel including some portion of the reverberance image of the left front and right front channels.
12. The device according to claim 10 wherein the computer program further includes:
filtering the second new audio signal, where the filtering includes using a second set of filtering coefficients generating a filtered second audio signal;
time-shifting the first new audio signal, where the first new audio signal is time-shifted with respect to the filtered second audio signal generating a shifted first audio signal;
determining a second difference between the filtered second audio signal and the shifted first audio signal; and
adjusting the second set of filtering coefficients so that the second difference is essentially orthogonal to the first audio signal.
13. The device according to claim 10 further comprising:
at least one audio output, where the at least one audio output can carry a plurality of output audio signals, where at least one output audio signal includes a combination of the first and second audio signals and at least one reverberance channel signal.
14. The device according to claim 10 further comprising a data storage device configured to store a plurality of output audio signals where at least one output audio signal includes a combination of the first and second audio signals and at least one reverberance channel signal.
15. The method according to claim 1 , where the step of adjusting the first set of filtering coefficients includes adjusting the first set of filter coefficients using a time domain or frequency domain implementation of at least one of the LMS algorithm, the NLMS algorithm, and the affine projection algorithm.
16. The device according to claim 10 , where the step of adjusting the first set of filtering coefficients includes adjusting the first set of filter coefficients using a time domain or frequency domain implementation of at least one of the LMS algorithm, the NLMS algorithm, and the affine projection algorithm.
17. A method according to claim 1 wherein each of the first and second audio signals includes a source right image component and a source left image component and wherein at least two of the respective source image components are correlated with one another.
18. The device according to claim 10 wherein each of the first and second audio signals includes a source right image component and a source left image component and wherein at least two of the respective source image components are correlated with one another.