System and method for capturing, encoding, distributing, and decoding immersive audio
View Patent ↗A sound field coding system and method that provides flexible capture, distribution, and reproduction of immersive audio recordings encoded in a generic digital audio format compatible with standard two-channel or multi-channel reproduction systems. This end-to-end system and method mitigates any impractical need for standard multi-channel microphone array configurations in consumer mobile devices such as smart phones or cameras. The system and method capture and spatially encode two-channel or multi-channel immersive audio signals that are compatible with legacy playback systems from flexible multi-channel microphone array configurations.
1. A method for processing a plurality of capture microphone signals, comprising:
selecting a capture microphone configuration having a plurality of capture microphones for capturing sound from at least one audio source, the capture microphone configuration defining a microphone directivity for each of the plurality of capture microphones relative to a reference direction;
selecting a virtual microphone configuration having a plurality of virtual microphones for encoding spatial information about a position of the at least one audio source relative to the reference direction, the virtual microphone configuration defining a virtual microphone directivity for each of the plurality of virtual microphones relative to the reference direction;
calculating spatial encoding coefficients based on the capture microphone configuration and on the virtual microphone configuration;
converting the plurality of capture microphone signals into a Spatially Encoded Signal (SES) including virtual microphone signals; and
defining at least one of the capture or virtual microphone directivities as a complex amplitude scaling factor that is dependent on the position of the at least one audio source and contains a non-zero phase component;
wherein each of the virtual microphone signals is obtained by combining the capture microphone signals using the spatial encoding coefficients.
2. The method of claim 1 , wherein the spatial information includes inter-channel phase differences between at least two of the virtual microphone signals.
3. The method of claim 2 , wherein the Spatially-Encoded Signal further comprises a two-channel phase-amplitude Spatially-Encoded Signal.
4. The method of claim 1 , wherein the plurality of capture microphone signals are A-format microphone signals, further comprising converting the A-format capture microphone signals into B-format microphone signals.
5. The method of claim 3 , further comprising reproducing the two-channel phase-amplitude Spatially-Encoded Signal over stereo loudspeakers or headphones.
6. The method of claim 4 , further comprising using the following phase-amplitude spatial encoding equations to obtain the virtual microphone signals:
L T =aV L +jbV S ; R T =aV R −jbV S
V L =p √{square root over (2)} W +(1− p )( X cos θ L +Y sin θ L )
V R =p √{square root over (2)} W +(1− p )( X cos θ R +Y sin θ R )
V S =p √{square root over (2)} W +(1− p )( X cos θ S +Y sin θ S )
where L T denotes a left-channel virtual microphone signal, R T denotes a right-channel virtual microphone signal, j denotes a substantially frequency-independent phase shift, a and b are 3:2 matrix encoding weights, θ L , θ K , θ S , and p are design parameters, W is an omnidirectional pressure signal in the B-format, X is a front-back figure-eight signal in the B-format, Y is a left-right figure-eight signal in the B-format, V L is a virtual left microphone signal in a horizontal plane, V R is a virtual right microphone signal corresponding to a supercardioid in the horizontal plane, and V S is a virtual surround microphone signal corresponding to a supercardioid in the horizontal plane, wherein the spatial information includes inter-channel phase differences between at least two of the virtual microphone signals, and wherein the Spatially-Encoded Signal further comprises a two-channel phase-amplitude Spatially-Encoded Signal.
7. The method of claim 6 , further comprising:
setting the 3:2 encoding weights to approximately a=1 and b=√{square root over ( 2 )}/3;
setting the design parameters to approximately θ L =−π/3, θ R =π/3, θ s =π; and
setting the design parameter p in accordance with a desired directivity of the virtual microphone signals.
8. The method of claim 4 , further comprising using the following phase-amplitude spatial encoding equations to obtain the virtual microphone signals:
L T =a 1 L+a 2 R+a 3 C+ja 4 L S −ja 5 R S
R T =a 2 L+a 1 R+a 3 C−ja 5 L S +ja 4 R S
where L T denotes the left-channel virtual microphone signal, R T denotes the right-channel virtual microphone signal, j denotes a substantially frequency-independent phase shift, {a 1 . . . a 5 } are 5:2 matrix encoding weights, and the B-format signals are converted into 5-channel surround-sound signals (L, R, C, L S , R S ), wherein the spatial information includes inter-channel phase differences between at least two of the virtual microphone signals, and wherein the Spatially-Encoded Signal further comprises a two-channel phase-amplitude Spatially-Encoded Signal.
9. A method for processing a plurality of capture microphone signals, comprising:
selecting a capture microphone configuration having a plurality of capture microphones for capturing sound from at least one audio source, the capture microphone configuration defining a microphone directivity for each of the plurality of capture microphones relative to a reference direction;
selecting a virtual microphone configuration having a plurality of virtual microphones for encoding spatial information about a position of the at least one audio source relative to the reference direction, the virtual microphone configuration defining a virtual microphone directivity for each of the plurality of virtual microphones relative to the reference direction;
calculating spatial encoding coefficients based on the capture microphone configuration and on the virtual microphone configuration; and
converting the plurality of capture microphone signals into a Spatially Encoded Signal (SES) including virtual microphone signals;
defining at least one of the capture microphone directivities as a frequency-dependent amplitude scaling factor that depends on the position of the at least one audio source; and
wherein each of the virtual microphone signals is obtained by combining the capture microphone signals using the spatial encoding coefficients.
10. The method of claim 9 , further comprising defining at least one of the capture microphone directivities as a complex amplitude scaling factor that is dependent on the position of the at least one audio source and contains a non-zero phase component.
11. The method of claim 9 , wherein the capture microphone directivities are estimated.
12. The method of claim 9 , wherein the capture microphone directivities are measured.
13. The method of claim 9 , further comprising defining at least one of the virtual microphone directivities as a complex amplitude scaling factor that is dependent on the position of the at least one audio source and contains a non-zero phase component.
14. The method of claim 13 , wherein the virtual microphone directivities are estimated.
15. The method of claim 13 , wherein the virtual microphone directivities are measured.