Method, apparatus, and computer program product for categorical spatial analysis-synthesis on spectrum of multichannel audio signals
View Patent ↗A method, apparatus and computer program product are therefore provided according to an example embodiment of the present invention in order to perform categorical analysis and synthesis of a multichannel signal to synthesize binaural signals and extract, separate, and manipulate components within the audio scene of the multichannel signal that were captured through multichannel audio means. In the context of a method, a multichannel signal is received. The method may include computing the spectrum for the multichannel signal, determining tonality of bands within the spectrum, and generating a band structure for the spectrum. The method may also include performing spatial analysis of the bands, performing source filtering using the bands, performing synthesis on the filtered band components, and generating an output signal. A corresponding apparatus and a computer program product are also provided.
1. A method comprising:
receiving a multichannel signal;
computing the spectrum for the multichannel signal;
determining tonality of bands within the spectrum;
generating a band structure for the spectrum;
performing spatial analysis of the bands;
performing source filtering using the bands;
performing synthesis on the filtered band components; and
generating a binaural signal,
wherein the tonality determination comprises comparing a normalized spectral component distribution in a band to an expected spectral component distribution.
2. A method according to claim 1 wherein determining the tonality of bands within the spectrum is performed on at least one channel in the multichannel signal.
3. A method according to claim 1 wherein determining the tonality of bands within the spectrum comprises determining if the band is tonal or non-tonal.
4. A method according to claim 1 wherein the width of the bands is variable.
5. A method according to claim 1 wherein the tonality determination of bands in the spectrum is based on statistical goodness of fit tests.
6. An apparatus comprising at least one processor and at least one memory including computer program instructions, the at least one memory and the computer program instructions configured to, with the at least one processor, cause the apparatus at least to:
receive a multichannel signal;
compute the spectrum for the multichannel signal;
determine tonality of bands within the spectrum;
generate a band structure for the spectrum;
perform spatial analysis of the bands;
perform source filtering using the bands;
perform synthesis on the filtered band components; and
generate a binaural signal,
wherein the apparatus is caused to generate a band structure for the spectrum by categorizing bands as tonal or non-tonal and computing upper and lower limits of tonal and non-tonal bands.
7. An apparatus according to claim 6 wherein determining the tonality of bands within the spectrum is performed on at least one channel in the multichannel signal.
8. An apparatus according to claim 6 wherein determining the tonality of bands within the spectrum comprises determining if the band is tonal or non-tonal.
9. An apparatus according to claim 6 wherein the width of the bands is variable.
10. An apparatus according to claim 6 wherein the tonality determination of bands in the spectrum is based on statistical goodness of fit tests.
11. An apparatus according to claim 6 wherein the tonality determination comprises comparing a normalized spectral component distribution in a band to an expected spectral component distribution.
12. An apparatus according to claim 11 wherein the expected spectral component distribution is generated by an ideal sinusoid.
13. An apparatus according to claim 11 wherein comparison of the spectral component distributions comprises using a test of goodness of fit.
14. An apparatus according to claim 6 wherein generating a band structure for the spectrum further comprises consolidating multiple continuous tonal bands into a single band.
15. An apparatus according to claim 6 wherein performing source filtering of the bands comprises processing the bands with head related transfer function (HRTF) filters.
16. An apparatus according to claim 6 wherein performing synthesis on the filtered band components comprises applying an inverse Discrete Fourier transform and applying add and overlap synthesis.
17. A computer program product comprising at least one non-transitory computer-readable storage medium bearing computer program instructions embodied therein for use with a computer, the computer program instructions comprising program instructions configured to:
receive a multichannel signal;
compute the spectrum for the multichannel signal;
determine tonality of bands within the spectrum;
generate a band structure for the spectrum;
perform spatial analysis of the bands;
perform source filtering using the bands;
perform synthesis on the filtered band components; and
generate a binaural signal,
wherein the program instructions configured to generate a band structure for the spectrum comprise program instructions configured to categorize bands as tonal or non-tonal and compute upper and lower limits of tonal and non-tonal bands.
18. An apparatus comprising:
means for receiving a multichannel signal;
means for computing the spectrum for the multichannel signal;
means for determining tonality of bands within the spectrum;
means for generating a band structure for the spectrum;
means for performing spatial analysis of the bands;
means for performing source filtering using the bands;
means for performing synthesis on the filtered band components; and
means for generating a binaural signal,
wherein the tonality determination comprises comparing a normalized spectral component distribution in a band to an expected spectral component distribution.
19. A method comprising:
receiving a multichannel signal;
computing the spectrum for the multichannel signal;
determining tonality of bands within the spectrum;
generating a band structure for the spectrum;
performing spatial analysis of the bands;
performing source filtering using the bands;
performing synthesis on the filtered band components; and
generating a binaural signal,
wherein generating a band structure for the spectrum comprises categorizing bands as tonal or non-tonal and computing upper and lower limits of tonal and non-tonal bands.
20. An apparatus comprising at least one processor and at least one memory including computer program instructions, the at least one memory and the computer program instructions configured to, with the at least one processor, cause the apparatus at least to:
receive a multichannel signal;
compute the spectrum for the multichannel signal;
determine tonality of bands within the spectrum;
generate a band structure for the spectrum;
perform spatial analysis of the bands;
perform source filtering using the bands;
perform synthesis on the filtered band components; and
generate a binaural signal,
wherein the tonality determination comprises comparing a normalized spectral component distribution in a band to an expected spectral component distribution.