Method for rendering multi-channel audio signals for L1 channels to a different number L2 of loudspeaker channels and apparatus for rendering multi-channel audio signals for L1 channels to a different number L2 of loudspeaker channels
Multi-channel audio content is mixed for a particular loudspeaker setup. However, a consumer's audio setup is very likely to use a different placement of speakers. The present invention provides a method of rendering multi-channel audio that assures replay of the spatial signal components with equal loudness of the signal. A method for obtaining an energy preserving mixing matrix (G) for mixing L 1 input audio channels to L 2 output channels comprises steps of obtaining (s 711 ) a first mixing matrix G, performing (s 712 ) a singular value decomposition on the first mixing matrix Ĝ to obtain a singularity matrix S, processing (s 713 ) the singularity matrix S to obtain a processed singularity matrix Ŝ, determining (s 715 ) a scaling factor a, and calculating (s 716 ) an improved mixing matrix G according to G=a U Ŝ V T . The perceived sound, loudness, timbre and spatial impression of multi-channel audio replayed on an arbitrary loudspeaker setup practically equals that of the original speaker setup.
1. A method for rendering L 1 channel-based input audio signals to L 2 loudspeaker channels, where L 1 is different from L 2 , the method comprising steps of
determining a mix type of the L 1 input audio signals, wherein the mix type specifies a coordinate system used for defining speaker positions and wherein possible mix types include at least one of spherical, cylindrical and rectangular;
performing a first delay and gain compensation on the L 1 input audio signals according to the determined mix type, wherein a delay and gain compensated input audio signal with L 1 channels and with a defined mix type is obtained;
mixing the delay and gain compensated input audio signal for L 2 audio channels, wherein a remixed audio signal for L 2 audio channels is obtained;
clipping the remixed audio signal, wherein a clipped remixed audio signal for L 2 audio channels is obtained; and
performing a second delay and gain compensation on the clipped remixed audio signal for L 2 audio channels, wherein L 2 loudspeaker channels are obtained;
wherein the mixing uses an energy preserving mixing matrix G that is obtained by
obtaining a first mixing matrix Ĝ from virtual source directions and target speaker directions using a panning method;
performing a singular value decomposition on the first mixing matrix Ĝ according to Ĝ=U S V T , wherein Uε L 2 ×L 2 and Vε L 1 ×L 2 are orthogonal matrices and Sε L 2 ×L 2 is a singularity matrix and has s first diagonal elements being the singular values of G in descending order and all other elements of S are zero;
processing the singularity matrix S, wherein a quantized singularity matrix Ŝ is obtained with diagonal elements that are above a threshold set to one and diagonal elements that are below a threshold set to zero;
determining a number m of diagonal elements that are set to one in the quantized singularity matrix Ŝ;
determining a scaling factor a according to
a
=
L
1
for
(
L
2
≤
L
1
)
or
a
=
L
2
for
(
L
2
>
L
1
)
;
and
calculating the energy preserving mixing matrix G according to G=a U Ŝ V T .
2. The method according to claim 1 , further comprising a step of filtering the delay and gain compensated input audio signal with L 1 channels, wherein a filtered delay and gain compensated input audio signal is obtained, and wherein the mixing uses the filtered delay and gain compensated input audio signal.
3. The method according to claim 2 , wherein the filtering of the delay and gain compensated input audio signal with L 1 channels uses an equalizer filter with different types of filters for the channels, wherein at least one channel uses a high-pass filter and at least one channel uses a low-pass filter.
4. The method according to claim 1 , wherein the defined mix type is spherical.
5. The method according to claim 1 , wherein the input signal is optimized for L 1 regular loudspeaker positions and the rendering is optimized for L 2 arbitrary loudspeaker positions, wherein at least one of the arbitrary loudspeaker positions is different from the regular loudspeaker positions.
6. A computer-implemented method for generating an energy preserving mixing matrix G for mixing input channel-based audio signals for L 1 audio channels to L 2 loudspeaker channels, the method comprising steps executed by the computer of
Obtaining a first mixing matrix Ĝ from virtual source directions and target speaker directions wherein a panning method is used;
Performing a singular value decomposition on the first mixing matrix Ĝ according to Ĝ=U S V T , wherein Uε L 2 ×L 2 and Vε L 2 ×L 2 are orthogonal matrices and Sε L 2 ×L 2 is a singularity matrix and has s first diagonal elements being the singular values of G in descending order and all other elements of S are zero;
processing the singularity matrix S, wherein a quantized singularity matrix Ŝ is obtained with diagonal elements that are above a threshold set to one and diagonal elements that are below a threshold set to zero;
determining a number m of diagonal elements that are set to one in the quantized singularity matrix Ŝ;
determining a scaling factor a according to
a
=
L
1
for
(
L
2
≤
L
1
)
or
a
=
L
2
for
(
L
2
>
L
1
)
;
and
calculating the energy preserving mixing matrix G according to G=a U Ŝ V T .
7. An apparatus for rendering L 1 channel-based input audio signals to L 2 loudspeaker channels, where L 1 is different from L 2 , the apparatus comprising at least one processor comprising at least one of each
a determining unit for determining a mix type of the L 1 input audio signals, wherein the mix type specifies a coordinate system used for defining speaker positions and wherein possible mix types include at least one of spherical, cylindrical and rectangular;
a first delay and gain compensation unit for performing a first delay and gain compensation on the L 1 input audio signals according to the determined mix type, wherein a delay and gain compensated input audio signal with L 1 channels and with a defined mix type is obtained;
a mixer unit for mixing the delay and gain compensated input audio signal for L 2 audio channels, wherein a remixed audio signal for L 2 audio channels is obtained;
a clipping unit for clipping the remixed audio signal, wherein a clipped remixed audio signal for L 2 audio channels is obtained; and
a second delay and gain compensation unit for performing a second delay and gain compensation on the clipped remixed audio signal for L 2 audio channels, wherein L 2 loudspeaker channels are obtained;
wherein the mixer unit mixes the delay and gain compensated input audio signal for L 2 audio channels uses an energy preserving mixing matrix G that is obtained by a mixing matrix generation unit that comprises one or more processors for implementing
a first calculating module for obtaining a first mixing matrix Ĝ from virtual source directions and target speaker directions using a panning method;
a singular value decomposition module for performing a singular value decomposition on the first mixing matrix Ĝ according Ĝ=USV T , wherein Uε L 2 ×L 2 and Vε L 2 ×L 2 are orthogonal matrices and Sε L 2 ×L 2 is a singularity matrix and has s first diagonal elements being the singular values of G in descending order and all other elements of S are zero;
a processing module for processing the singularity matrix S, wherein a quantized singularity matrix Ŝ is obtained with diagonal elements that are above a threshold set to one and diagonal elements that are below a threshold set to zero;
a counting module for determining a number m of diagonal elements that are set to one in the quantized singularity matrix Ŝ;
a second calculating module for determining a scaling factor a according to
a
=
L
1
for
(
L
2
≤
L
1
)
or
a
=
L
2
for
(
L
2
>
L
1
)
;
and
a third calculating module for calculating a mixing matrix G according to G=a U Ŝ V T .
8. The apparatus according to claim 7 , further comprising an equalization filter for filtering the delay and gain compensated input audio signal with L 1 channels, wherein a filtered delay and gain compensated input audio signal is obtained.
9. The apparatus according to claim 8 , wherein the equalization filter comprises different types of filters that are used for the channels, wherein at least one channel uses a high-pass filter and at least one channel uses a low-pass filter.
10. The apparatus according to claim 7 , wherein the defined mix type is spherical.
11. The apparatus according to claim 7 , wherein the input signal is optimized for L 1 regular loudspeaker positions and the rendering is optimized for L 2 arbitrary loudspeaker positions, wherein at least one of the arbitrary loudspeaker positions is different from the regular loudspeaker positions.
12. An apparatus for obtaining an energy preserving mixing matrix G for mixing input channel-based audio signals for L 1 audio channels to L 2 loudspeaker channels, comprising at least one processor comprising at least one processing element for implementing
a first calculation module for obtaining a first mixing matrix Ĝ from virtual source directions and target speaker directions wherein a panning method is used;
a singular value decomposition module for performing a singular value decomposition on the first mixing matrix Ĝ according to Ĝ=U S V T , wherein Uε L 2 ×L 2 and Vε L 2 ×L 2 are orthogonal matrices and Sε L 2 ×L 2 is a singularity matrix and has s first diagonal elements being the singular values of G in descending order and all other elements of S are zero;
a processing module processing the singularity matrix S, wherein a quantized singularity matrix Ŝ is obtained with diagonal elements that are above a threshold set to one and diagonal elements that are below a threshold set to zero;
a counting module for determining a number m of diagonal elements that are set to one in the quantized singularity matrix Ŝ;
a second calculation module for determining a scaling factor α according to
a
=
L
1
for
(
L
2
≤
L
1
)
or
a
=
L
2
for
(
L
2
>
L
1
)
;
and
a third calculation module for calculating the energy preserving mixing matrix G according to G=a U Ŝ V T .
13. A non-transitory computer readable storage medium having stored thereon instructions that when executed on a computer cause the computer to perform a method according to claim 1 .