System for and method of controlling a three-dimensional audio engine
A system for and a method of controlling generation of a 3D audio stream are disclosed. The method comprises accessing an audio stream; determining a value of a feature associated with the audio stream; selecting one or more 3D control parameters from a set of 3D control parameters, the selecting being based on the value of the feature associated with the audio stream; and generating the 3D audio stream based on the selected one or more 3D control parameters. In some embodiments, the feature is a metric associated with a frequency distribution of correlations of the audio stream.
1. A method of controlling generation of a 3D audio stream, the method comprising:
accessing an audio stream;
generating a feature vector associated with the audio stream by a Deep Neural Network (DNN) which takes for input the audio stream in either time or frequency domain and which outputs the feature vector;
selecting one or more 3D control parameters from a set of 3D control parameters, the selecting being based on the feature vector associated with the audio stream; and
generating the 3D audio stream based on the selected one or more 3D control parameters.
2. The method of claim 1 , wherein the feature vector comprises a metric associated with a frequency distribution of correlations of left and right channels of the audio stream.
3. The method of claim 2 , wherein the selecting the one or more 3D control parameters comprises accessing a database of 3D productions, the database comprising associations between values of frequency distributions of correlations and 3D productions, each 3D production being associated with 3D control parameters specifically customized for a unique audio stream.
4. The method of claim 2 , wherein the selecting the one or more 3D control parameters comprises accessing a database of 3D presets, the database comprising associations between values of frequency distributions of correlations and 3D presets, the 3D presets being generic 3D control parameters.
5. The method of claim 2 , further comprising determining the frequency distribution of correlations in real-time, and wherein the selecting of the one or more 3D control parameters is done dynamically.
6. The method of claim 2 , wherein the frequency distribution of correlations is calculated based on the following equations:
ϕ
=
tan
-
1
(
x
l
x
r
)
x
=
x
l
2
+
x
r
2
corr
(
x
l
,
x
r
)
=
ϕ
x
wherein xl and xr are left and right channels sample values at a given time of the audio stream, Phi is the phase between the left and right channels, ∥x∥ is a vector length of the feature vector and the correlation (corr) is a product of the phase and the vector length.
7. The method of claim 6 , wherein the selecting the one or more 3D control parameters comprises minimizing the KLD.
8. The method of claim 2 , wherein the selecting the one or more 3D control parameters from the set of 3D control parameters comprises calculating a Kullback Leibler Divergence (KLD) comparing the frequency distribution of correlations of left and right channels of the audio stream and reference frequency distributions of correlations associated with the 3D parameters.
9. The method of claim 1 , wherein the one or more 3D control parameters comprises a list of positional impulse responses (PIRs), a list of identifiers allowing to retrieve PIRs, a number of PIRs, a list of positions of PIRs, and/or a list of identifiers allowing to retrieve positions of PIRs.
10. The method of claim 1 , wherein the one or more 3D control parameters comprises a list of positions of positional impulse responses (PIRs), and wherein the positions of PIRs are defined by one or more of an azimuth, an elevation and/or a distance.
11. The method of claim 1 , wherein the one or more 3D control parameters comprise a gain, a delay, and/or a filter to be applied to a given virtual wave front associated with a given positional impulse response (PIR) generated by an audio engine.
12. The method of claim 1 , wherein generating the 3D audio stream comprises generating three virtual wave fronts by selecting and/or positioning three positional impulse responses (PIRs) identified and/or positioned based on the one or more 3D control parameters.
13. The method of claim 1 , further comprising creating the one or more 3D control parameters based on the audio stream.
14. The method of claim 13 , wherein the creating the one or more 3D control parameters comprises selecting and adapting pre-existing 3D control parameters.
15. The method of claim 1 , wherein the selecting the one or more 3D control parameters is based at least in part on a user input.
16. A computer-implemented system, the system comprising at least one processor and memory storing a plurality of executable instructions which, when executed by the at least one processor, cause the system to:
access an audio stream;
generate a feature vector associated with the audio stream by a Deep Neural Network (DNN) which takes for input the audio stream and which outputs the feature vector;
select one or more 3D control parameters from a set of 3D control parameters, the selecting being based on the feature vector associated with the audio stream; and
generate the 3D audio stream based on the selected one or more 3D control parameters.
17. A non-transitory computer-readable medium storing program instructions which upon being executed by a processor cause to perform:
accessing an audio stream;
generating a feature vector associated with the audio stream by a Deep Neural Network (DNN) which takes for input the audio stream and which outputs the feature vector;
selecting one or more 3D control parameters from a set of 3D control parameters, the selecting being based on the feature vector associated with the audio stream; and
generating the 3D audio stream based on the selected one or more 3D control parameters.
18. A method of controlling generation of a 3D audio stream, the method comprising:
accessing an audio stream;
generating a feature vector associated with the audio stream, wherein the feature vector comprises a metric associated with a frequency distribution of correlations of left and right channels of the audio stream;
selecting, based on the feature vector associated with the audio stream, one or more 3D control parameters from a set of 3D control parameters; and
generating the 3D audio stream based on the one or more 3D control parameters, wherein the frequency distribution of correlations of left and right channels of the audio stream is calculated based on the following equations:
ϕ
=
tan
-
1
(
x
l
x
r
)
x
=
x
l
2
+
x
r
2
corr
(
x
l
,
x
r
)
=
ϕ
x
wherein xl and xr are left and right channels sample values at a given time of the audio stream, Phi is the phase between the left and right channels, ∥x∥ is a vector length of the feature vector and the correlation (corr) is a product of the phase and the vector length.
19. The method of claim 18 , wherein the selecting the one or more 3D control parameters comprises accessing a database of 3D productions, each 3D production being associated with 3D control parameters specifically customized for a unique audio stream.
20. A computer-implemented system, the system comprising at least one processor and memory storing a plurality of executable instructions which, when executed by the at least one processor, cause the system to:
generate a feature vector associated with an audio stream, wherein the feature vector comprises a metric associated with a frequency distribution of correlations of left and right channels of the audio stream;
select, based on the feature vector associated with the audio stream, one or more 3D control parameters from a set of 3D control parameters; and
generate a 3D audio stream based on the selected one or more 3D control parameters, wherein the frequency distribution of correlations of left and right channels of the audio stream is calculated based on the following equations:
ϕ
=
tan
-
1
(
x
l
x
r
)
x
=
x
l
2
+
x
r
2
corr
(
x
l
,
x
r
)
=
ϕ
x
wherein xl and xr are left and right channels sample values at a given time of the audio stream, Phi is the phase between the left and right channels, ∥x∥ is a vector length of the feature vector and the correlation (corr) is a product of the phase and the vector length.