Audio spatial environment engine
View Patent ↗An audio spatial environment engine for flexible and scalable up-mixing from an M channel audio system to an N channel audio system, where M and N are integers and N is greater than M, is provided. The input M channel audio is provided to an analysis filter bank which converts the time domain signals into frequency domain signals. Relevant inter-channel spatial cues are extracted from the frequency domain signals on a sub-band basis and are used as parameters to generate adaptive N channel filters which control the spatial placement of a frequency band element in the up-mixed sound field. The N channel filters are smoothed across both time and frequency to limit filter variability which could cause annoying fluctuation effects. The smoothed N channel filters are then applied to adaptive combinations of the frequency domain input signals and are provided to a synthesis filter bank which generates the N channel time domain output signals.
1. A system for converting between an N channel audio system and an M channel audio system, where M and N are integers and N is greater than M, comprising:
a reference down-mixer receiving one or more of N channels of audio data and converting the one or more N channels of audio data to one or more M channels of audio data;
a reference up-mixer receiving the one or more M channels of audio data and converting the one or more M channels of audio data to one or more N′ channels of audio data;
a receiver receiving one or more corrected M channels of audio data and converting the one or more corrected M channels of audio data to one or more N′ channels of audio data; and
a correction system receiving the one or more M channels of audio data, the one or more N channels of audio data, and the one or more N′ channels of audio data and correcting the one or more M channels of audio data based on differences between the one or more N channels of audio data and the one or more N′ channels of audio data.
2. The system of claim 1 wherein the correction system further comprises:
a first sub-band vector calculation unit receiving the one or more N channels of audio data and generating one or more first sub-bands of audio spatial image data;
a second sub-band vector calculation unit receiving the one or more N′ channels of audio data and generating one or more second sub-bands of audio spatial image data; and
the correction system receiving the one or more first sub-bands of audio spatial image data and the one or more second sub-bands of audio spatial image data and correcting the one or more M channels of audio data based on differences between the one or more first sub-bands of audio spatial image data and the one or more second sub-bands of audio spatial image data.
3. The system of claim 2 wherein each of the one or more first sub-bands of audio spatial image data and the one or more second sub-bands of audio spatial image data has an associated energy value and position value.
4. The system of claim 3 wherein each of the position values represents the apparent location of the associated sub-band of audio spatial image data in two-dimensional space, where a coordinate of the location is determined by a vector sum of an energy value associated with one or more of N sound sources and a coordinate of one or more of the N sound sources.
5. The system of claim 1 wherein the reference down-mixer further comprises one or more phase shift stages, each receiving one of the N channels of audio data and applying a predetermined phase shift to the associated channel of audio data.
6. The system of claim 5 wherein the reference down-mixer further comprises one or more summation stages coupled to one or more of the phase shift stages and combining the output from the one or more phase shift stages in a predetermined manner.
7. The system of claim 1 wherein the reference up-mixer further comprises:
a time domain to frequency domain conversion stage receiving one or more of the M channels of audio data and generating one or more sub-bands of audio spatial image data;
a filter generator receiving one or more of the M channels of the one or more sub-bands of audio spatial image data and generating two or more filters;
a smoothing stage receiving the two or more filters and averaging two or more of the filters.
8. A system for converting between an N channel audio system and an M channel audio system, where M and N are integers and N is greater than M, comprising:
a reference down-mixer receiving one or more of N channels of audio data and converting the one or more N channels of audio data to one or more M channels of audio data;
a correction system receiving the one or more M channels of audio data, the one or more N channels of audio data, and one or more N′ channels of audio data and correcting the one or more M channels of audio data based on differences between the one or more N channels of audio data and the one or more N′ channels of audio data; and
a receiver receiving the corrected one or more M channels of audio data and generating one or more N″ channels of audio data.
9. The system of claim 8 further comprising a reference up-mixer receiving the one or more M channels of audio data and converting the one or more M channels of audio data to the one or more N′ channels of audio data.
10. The system of claim 8 wherein the correction system further comprises:
a first sub-band vector calculation unit receiving the one or more N channels of audio data and generating one or more first sub-bands of audio spatial image data;
a second sub-band vector calculation unit receiving the one or more N′ channels of audio data and generating one or more second sub-bands of audio spatial image data; and
the correction system receiving the one or more first sub-bands of audio spatial image data and the one or more second sub-bands of audio spatial image data and correcting the one or more M channels of audio data based on differences between the one or more first sub-bands of audio spatial image data and the one or more second sub-bands of audio spatial image data.
11. The system of claim 10 wherein each of the one or more first sub-bands of audio spatial image data and the one or more second sub-bands of audio spatial image data has an associated energy value and position value.
12. The system of claim 11 wherein each of the position values represents the apparent location of the associated sub-band of audio spatial image data in two-dimensional space, where a coordinate of the location is determined by a vector sum of an energy value associated with one or more of N sound sources and a coordinate of one or more of the N sound sources.
13. The system of claim 8 wherein the reference down-mixer further comprises one or more phase shift stages, each receiving one of the N channels of audio data and applying a predetermined phase shift to the associated channel of audio data.
14. The system of claim 13 wherein the reference down-mixer further comprises one or more summation stages coupled to one or more of the phase shift stages and combining the output from the one or more phase shift stages in a predetermined manner.
15. The system of claim 9 wherein the reference up-mixer further comprises:
a time domain to frequency domain conversion stage receiving one or more of the M channels of audio data and generating one or more sub-bands of audio spatial image data;
a filter generator receiving one or more of the M channels of the one or more sub-bands of audio spatial image data and generating two or more filters;
a smoothing stage receiving the one or more filters and averaging two or more of the filters.
16. A method for converting between an N channel audio system and an M channel audio system, where N and M are integers and N is greater than M, comprising:
converting one or more of N channels of audio data to one or more of M channels of audio data using an audio data processing system;
correcting the one or more M channels of audio data based on differences between one or more of the N channels of audio data and one or more N′ channels of audio data using an audio data processing system; and
receiving the corrected M channels of audio data and converting the corrected M channels of audio data into N″ channels of audio data.
17. The method of claim 16 further comprising converting one or more of the M channels of audio data to one or more of the N′ channels of audio data.
18. The method of claim 16 wherein converting one or more of the N channels of audio data to one or more of the M channels of audio data comprises processing one or more of the N channels of audio data with a phase shift function to apply a predetermined phase shift to the associated channel of audio data.
19. The method of claim 17 wherein converting one or more of the M channels of audio data to one or more of the N′ channels of audio data comprises:
converting one or more of the M channels of audio data from a time domain to one or more sub-bands in frequency domain;
generating one or more filters using the one or more of the sub-bands of the M channels; and
averaging two or more of the filters.
20. The method of claim 16 wherein correcting one or more of the M channels of audio data based on differences between one or more of the N channels of audio data and one or more of the N′ channels of audio data comprises determining an energy and position vector for each of one or more sub-bands of one or more of the N channels of audio data.
21. The method of claim 16 wherein correcting one or more sub-bands of the M channels of audio data comprises adjusting an energy and a position vector for one or more of the sub-bands of one or more of the M channels of audio data such that the adjusted sub-bands of one or more of the M channels of audio data are converted into one or more N″ channels of audio data having one or more sub-band energy and position vectors that are closer to the energy and the position vectors of one or more of the sub-bands of one or more of the N channels of audio data than a corresponding energy and position vector for one or more of the sub-bands of one or more of the N′ channels of audio data.