Immersive voice and audio services (IVAS) with adaptive downmix strategies
View Patent ↗Disclosed is an audio signal encoding/decoding method that uses an encoding downmix strategy applied at an encoder that is different than a decoding re-mix/upmix strategy applied at a decoder. Based on the type of downmix coding scheme, the method comprises: computing input downmixing gains to be applied to the input audio signal to construct a primary downmix channel; determining downmix scaling gains to scale the primary downmix channel; generating prediction gains based on the input audio signal, the input downmixing gains and the downmix scaling gains; determining residual channel(s) from the side channels by using the primary downmix channel and the prediction gains to generate side channel predictions and subtracting the side channel predictions from the side channels; determining decorrelation gains based on energy in the residual channels; encoding the primary downmix channel, the residual channel(s), the prediction gains and the decorrelation gains; and sending the bitstream to a decoder.
1. An audio signal encoding method comprising:
obtaining, with at least one processor, an input audio signal, the input audio signal representing an input audio scene and comprising a primary input audio channel and side channels;
determining, with the at least one processor, a type of downmix coding scheme based on the input audio signal;
based on the type of downmix coding scheme:
computing, with the at least one processor, one or more input downmixing gains to be applied to the input audio signal to construct a primary downmix channel, wherein the input downmixing gains are determined to minimize an overall prediction error on the side channels;
determining, with the at least one processor, one or more downmix scaling gains to scale the primary downmix channel, wherein the downmix scaling gains are determined by minimizing an energy difference between a reconstructed representation of the input audio scene from the primary downmix channel and the input audio signal;
generating, with the at least one processor, prediction gains based on the input audio signal, the input downmixing gains and the downmix scaling gains;
determining, with the at least one processor, one or more residual channels from the side channels in the input audio signal by using the primary downmix channel and the prediction gains to generate side channel predictions and then subtracting the side channel predictions from the side channels;
determining, with the at least one processor, decorrelation gains based on energy in the residual channels;
encoding, with the at least one processor, the primary downmix channel, zero or more of the residual channels and side information into a bitstream, the side information comprising the prediction gains and the decorrelation gains; and
outputting, with the at least one processor, the bitstream.