Objective quality metrics for ambisonic spatial audio
A computing device includes a processor and a memory. The processor is configured to generate spectrograms, for example, using short-time Fourier transform, for a plurality of channels of reference and test ambisonic signals. In some implementations, the test ambisonic signal may be generated by decoding an encoded version of the reference ambisonic signal. The processor is further configured to compare, for each of the plurality of channels of a reference ambisonic signal, at least a patch associated with a channel of the reference ambisonic signal with at least a corresponding patch of a corresponding channel of the test ambisonic signal and determine a localization accuracy of the test ambisonic signal based on the comparison. In some implementations, the comparing may be based on phaseograms of the reference and test ambisonic signals.
1. A computer-implemented method of determining quality of experience (QoE) of ambisonic spatial audio signals, comprising:
comparing, for each of a plurality of channels of a reference ambisonic signal, at least a patch associated with a channel of the reference ambisonic signal with at least a corresponding patch of a corresponding channel of a test ambisonic signal, the test ambisonic signal generated by decoding an encoded version of the reference ambisonic signal; and
determining a localization accuracy of the test ambisonic signal based on the comparison.
2. The method of claim 1 , further comprising:
aligning, prior to the comparing, the patch associated with the channel of the reference ambisonic signal with the corresponding patch of the corresponding channel of the test ambisonic signal.
3. The method of claim 1 , wherein the comparing is based, at least in part, on spectrograms, phaseograms, or a combination thereof, of the reference ambisonic signal and the test ambisonic signal.
4. The method of claim 1 , further comprising:
generating spectrograms of the plurality of channels of the reference ambisonic signal and the test ambisonic signal, the spectrograms generated using short-time Fourier transform (STFT).
5. The method of claim 1 , further comprising:
determining a listening quality of the test ambisonic signal based on the comparison.
6. The method of claim 5 , wherein the comparing is based on a neurogram similarity index measure (NSIM),
wherein the comparing further comprises comparing a patch associated with an omni-directional channel of the reference ambisonic signal with a corresponding patch of an omni-directional channel of the test ambisonic signal, and
wherein the determining the listening quality further comprises determining an aggregated similarity score based on the comparing of the omni-directional channel of the reference ambisonic signal and the omni-directional channel of the test ambisonic signal.
7. The method of claim 1 , herein the comparing is based on a neurogram similarity index measure (NSIM),
wherein the comparing further comprises comparing a patch associated with each multi-directional channel of the reference ambisonic signal with a corresponding patch of a corresponding multi-directional channel of the test ambisonic signal, and
wherein the determining the localization accuracy further comprises determining an aggregated similarity score that is based on weighted sum of similarity scores between corresponding multi-directional channels of the test ambisonic signal and the reference ambisonic signal.
8. The method of claim 7 , further comprising:
assigning different weights to vertical and horizontal components of the multi-directional channels.
9. A computing device for determining quality of experience (QoE) of Ambisonic spatial audio signals, comprising:
a processor; and
a memory, the memory including instructions configured to cause the processor to:
compare, for each of a plurality of channels of a reference ambisonic signal, at least a patch associated with a channel of the reference ambisonic signal with at least a corresponding patch of a corresponding channel of a test ambisonic signal, the test ambisonic signal generated by decoding an encoded version of the reference ambisonic signal; and
determine a localization accuracy of the test ambisonic signal based on the comparison.
10. The computing device of claim 9 , wherein the processor is further configured to:
align, prior to the comparing, the patch associated with the channel of the reference ambisonic signal with the corresponding patch of the corresponding channel of the test ambisonic signal.
11. The computing device of claim 9 , wherein the processor is further configured to:
compare based, at least in part, on spectrograms, phaseograms, or a combination thereof, of the reference ambisonic signal and the test ambisonic signal.
12. The computing device of claim 9 , wherein the processor is further configured to:
determine a listening quality of the test ambisonic signal based on the comparison.
13. The computing device of claim 12 , wherein the comparison is based on a neurogram similarity index measure (NSIM), and wherein the processor is further configured to:
compare a patch associated with an omni-directional channel of the reference ambisonic signal with a corresponding patch of an omni-directional channel of the test ambisonic signal, and
determine the listening quality further comprises determining an aggregated similarity score based on the comparing of the omni-directional channel of the reference ambisonic signal and the omni-directional channel of the test ambisonic signal.
14. The computing device of claim 9 , wherein the comparing is based on a neurogram similarity index measure (NSIM), wherein the processor is further configured to:
compare a patch associated with each multi-directional channel of the reference ambisonic signal with a corresponding patch of a corresponding multi-directional channel of the test ambisonic signal, and
determine the localization accuracy further comprises determining an aggregated similarity score that is based on weighted sum of similarity scores between corresponding multi-directional channels of the test ambisonic signal and the reference ambisonic signal.
15. A non-transitory computer-readable storage medium having stored thereon computer executable program code which, when executed on a computer system, causes the computer system to perform a method of determining quality of experience (QoE) of ambisonic spatial audio signals comprising:
comparing, for each of a plurality of channels of a reference ambisonic signal, at least a patch associated with a channel of the reference ambisonic signal with at least a corresponding patch of a corresponding channel of a test ambisonic signal, the test ambisonic signal generated by decoding an encoded version of the reference ambisonic signal; and
determining a localization accuracy of the test ambisonic signal based on the comparison.
16. The computer-readable storage medium of claim 15 , further comprising code for:
aligning, prior to the comparing, the patch associated with the channel of the reference ambisonic signal with the corresponding patch of the corresponding channel of the test ambisonic signal.
17. The computer-readable storage medium of claim 15 , further comprising code for:
comparing being based, at least in part, on spectrograms, phaseograms, or a combination thereof, of the reference ambisonic signal and the test ambisonic signal,
generating spectrograms of the plurality of channels of the reference ambisonic signal and the test ambisonic signal, the spectrograms generated using short-time Fourier transform (STFT).
18. The computer-readable storage medium of claim 15 , further comprising code for:
determining a listening quality of the test ambisonic signal based on the comparison.
19. The computer-readable storage medium of claim 18 , wherein the comparing is based on a neurogram similarity index measure (NSIM),
wherein the comparing further comprises comparing a patch associated with an omni-directional channel of the reference ambisonic signal with a corresponding patch of an omni-directional channel of the test ambisonic signal, and
wherein the determining the listening quality further comprises determining an aggregated similarity score based on the comparing of the omni-directional channel of the reference ambisonic signal and the omni-directional channel of the test ambisonic signal.
20. The computer-readable storage medium of claim 15 , wherein the comparing is based on a neurogram similarity index measure (NSIM),
wherein the comparing further comprises comparing a patch associated with each multi-directional channel of the reference ambisonic signal with a corresponding patch of a corresponding multi-directional channel of the test ambisonic signal, and
wherein the determining the localization accuracy further comprises determining an aggregated similarity score that is based on weighted sum of similarity scores between corresponding multi-directional channels of the test ambisonic signal and the reference ambisonic signal.