Audio spatial environment down-mixer
An audio spatial environment engine is provided for converting from an N channel audio system to an M channel audio system, such as in a dynamic down-mixer where N and M are integers and N is greater than M. The dynamic down-mix methodology consists of a static down-mix system utilizing an intelligent analysis and correction loop. The original N-channel audio signals are provided to a static down-mix process which produces a down-mixed M-channel audio signal. That M-channel audio signal is provided to an up-mix process which generates a subsequent N-channel audio signal. Any spectral, temporal, or spatial inaccuracies between the original N-channel audio and the subsequent up-mixed N-channel audio are then identified and corrected in the down-mixed M-channel audio signal over a plurality of frequency bands generating the final down-mixed M-channel audio signal. The corrections performed on the down-mixed M-channel audio signal consist of modifications to the relevant inter-channel spatial cues such as inter-channel level difference (ICLD) and inter-channel coherence (ICC) per frequency band.
1 . An audio spatial environment engine for converting from an N channel audio system to an M channel audio system, where N and M are integers and N is greater than M, comprising:
a time domain to frequency domain conversion stage receiving the M channels of audio data and generating a plurality of sub-bands of audio spatial image data;
a filter generator receiving the M channels of the plurality of sub-bands of audio spatial image data and generating N′channels of a plurality of sub-bands of audio spatial image data; and
a summation stage coupled to the filter generator and receiving the M channels of the plurality of sub-bands of audio spatial image data and the N′channels of the plurality of sub-bands of audio spatial image data and generating scaled N′channels of the plurality of sub-bands of audio spatial image data.
2 . The audio spatial environment engine of claim 1 further comprising a frequency domain to time domain conversion stage receiving the scaled N′channels of the plurality of sub-bands of audio spatial image data and generating the N′channels of audio data.
3 . The audio spatial environment engine of claim 1 further comprising:
a smoothing stage coupled to the filter generator, the smoothing stage receiving the N′channels of the plurality of sub-bands of audio spatial image data and averaging each sub-band with one or more adjacent sub-bands; and
the summation stage coupled to the smoothing stage and receiving the M channels of the plurality of sub-bands of audio spatial image data and the smoothed N′channels of the plurality of sub-bands of audio spatial image data and generating scaled N′channels of the plurality of sub-bands of audio spatial image data.
4 . The audio spatial environment engine of claim 1 wherein the summation stage further comprises a left channel summation stage multiplying each of a plurality of sub-bands of a left channel of the M channels times each of a corresponding plurality of sub-bands of audio spatial image data of a left channel of the N′channels.
5 . The audio spatial environment engine of claim 1 wherein the summation stage further comprises a right channel summation stage multiplying each of a plurality of sub-bands of a right channel of the M channels times each of a corresponding plurality of sub-bands of audio spatial image data of a right channel of the N′channels.
6 . The audio spatial environment engine of claim 1 wherein the summation stage further comprises a center channel summation stage satisfying for each sub-band an equation:
( G C ( f )* L ( f )+((1 −G C ( F ))* R ( f ))* H C ( f )
where
G C (f)=a center channel sub-band scaling factor;
L(f)=a left channel sub-band of the M channels;
R(f)=a right channel sub-band of the M channels; and
H C (F)=a filtered center channel sub-band of the N′channels.
7 . The audio spatial environment engine of claim 1 wherein the summation stage further comprises a left surround channel summation stage satisfying for each sub-band an equation:
( G LS ( F )* L ( f )−((1 −G LS ( F ))* R ( f ))* H LS ( F )
where
G LS (F)=a left surround channel sub-band scaling factor;
L(f)=a left channel sub-band of the M channels;
R(f)=a right channel sub-band of the M channels; and
H LS (F)=a filtered left surround channel sub-band of the N′channels.
8 . The audio spatial environment engine of claim 1 wherein the summation stage further comprises a right surround channel summation stage satisfying for each sub-band an equation:
((1 −G S ( f ))* R ( f ))+( G S ( f ))* L ( f ))* H S ( f )
where
G RS (f)=a right surround channel sub-band scaling factor;
L(f)=a left channel sub-band of the M channels;
R(f) a right channel sub-band of the M channels; and
H RS (f)=a filtered right surround channel sub-band of the N′channels.
9 . A method for converting from an M channel audio system to an N channel audio system, where M and N are integers and N is greater than M, comprising:
receiving the M channels of audio data;
generating a plurality of sub-bands of audio spatial image data for each channel of the M channels;
filtering the M channels of the plurality of sub-bands of audio spatial image data to generate N′channels of a plurality of sub-bands of audio spatial image data; and
multiplying the M channels of the plurality of sub-bands of audio spatial image data by the N′channels of the plurality of sub-bands of audio spatial image data to generate scaled N′channels of the plurality of sub-bands of audio spatial image data.
10 . The method of claim 9 wherein multiplying the M channels of the plurality of sub-bands of audio spatial image data by the N′channels of the plurality of sub-bands of audio spatial image data further comprises:
multiplying one or more of the M channels of the plurality of sub-bands of audio spatial image data by a sub-band scaling factor; and
multiplying the scaled M channels of the plurality of sub-bands of audio spatial image data by the N′channels of the plurality of sub-bands of audio spatial image data.
11 . The method of claim 9 wherein multiplying the M channels of the plurality of sub-bands of audio spatial image data by the N′channels of the plurality of sub-bands of audio spatial image data further comprises multiplying each of the plurality of sub-bands of the M channels by a corresponding sub-band of audio spatial image data of the N′channels.
12 . The method of claim 9 wherein multiplying the M channels of the plurality of sub-bands of audio spatial image data by the N′channels of the plurality of sub-bands of audio spatial image data comprises multiplying each of a plurality of sub-bands of a left channel of the M channels times each of a corresponding plurality of sub-bands of audio spatial image data of a left channel of the N′channels.
13 . The method of claim 9 wherein multiplying the M channels of the plurality of sub-bands of audio spatial image data by the N′channels of the plurality of sub-bands of audio spatial image data comprises multiplying each of a plurality of sub-bands of a right channel of the M channels times each of a corresponding plurality of sub-bands of audio spatial image data of a right channel of the N′channels.
14 . The method of claim 9 wherein multiplying the M channels of the plurality of sub-bands of audio spatial image data by the N′channels of the plurality of sub-bands of audio spatial image data comprises satisfying for each sub-band an equation:
( G C ( f )* L ( f )+((1 −G C ( F ))* R ( f ))* H C ( F )
where
G C (f)=a center channel sub-band scaling factor;
L(f)=a left channel sub-band;
R(f)=a right channel sub-band; and
H C (F)=a filtered center channel sub-band.
15 . The method of claim 9 wherein multiplying the M channels of the plurality of sub-bands of audio spatial image data by the N′channels of the plurality of sub-bands of audio spatial image data comprises satisfying for each sub-band an equation:
( G LS ( f )* L ( f )−((1 −G LS ( F ))* R ( f ))* H LS ( F )
where
G LS (F)=a left surround channel sub-band scaling factor;
L(f) a left channel sub-band;
R(f)=a right channel sub-band; and
H LS (F)=a filtered left surround channel sub-band.
16 . The method of claim 9 wherein multiplying the M channels of the plurality of sub-bands of audio spatial image data by the N′channels of the plurality of sub-bands of audio spatial image data comprises satisfying for each sub-band an equation:
((1 −G RS ( f ))* R ( f ))+( G S ( f ))* L ( f ))* H RS ( f )
where
G RS (F)=a right surround channel sub-band scaling factor;
L(f)=a left channel sub-band;
R(f)=a right channel sub-band; and
H RS (f)=a filtered right surround channel sub-band.
17 . An audio spatial environment engine for converting from an M channel audio system to an N channel audio system, where M and N are integers and N is greater than M, comprising:
time domain to frequency domain conversion means for receiving the M channels of audio data and generating a plurality of sub-bands of audio spatial image data;
filter generator means for receiving the M channels of the plurality of sub-bands of audio spatial image data and generating N′channels of a plurality of sub-bands of audio spatial image data; and
summation stage means for receiving the M channels of the plurality of sub-bands of audio spatial image data and the N′channels of the plurality of sub-bands of audio spatial image data and generating scaled N′channels of the plurality of sub-bands of audio spatial image data.
18 . The audio spatial environment engine of claim 17 further comprising frequency domain to time domain conversion stage means for receiving the scaled N′channels of the plurality of sub-bands of audio spatial image data and generating the N′channels of audio data.
19 . The audio spatial environment engine of claim 17 further comprising:
smoothing stage means for receiving the N′channels of the plurality of sub-bands of audio spatial image data and averaging each sub-band with one or more adjacent sub-bands; and
wherein the summation stage means receives the M channels of the plurality of sub-bands of audio spatial image data and the smoothed N′channels of the plurality of sub-bands of audio spatial image data and generates scaled N′channels of the plurality of sub-bands of audio spatial image data.
20 . The audio spatial environment engine of claim 17 wherein the summation stage means further comprises left channel summation stage means for multiplying each of a plurality of sub-bands of a left channel of the M channels times each of a corresponding plurality of sub-bands of audio spatial image data of a left channel of the N′channels.