Automatic generation of metadata for audio dominance effects

1. A method, comprising:

receiving a first audio stream and a second audio stream;

determining whether a first energy level for the first audio stream individually meets or exceeds an energy level threshold;

determining whether a second energy level for the second audio stream individually meets or exceeds the energy level threshold;

in response to determining that the first energy level for the first audio stream individually meets or exceeds the energy level threshold and determining that the second energy level for the second audio stream individually meets or exceeds the energy level threshold, computing a set of gains for at least one of the first audio stream and the second audio stream to create a dominance effect of the second audio stream over the first audio stream;

generating metadata comprising the set of gains for at least one of the first audio stream and the second audio stream;

generating an output audio signal comprising the first audio stream, the second audio stream, and the metadata comprising the set of gains;

wherein the method is performed by a computing device, which comprises a processor.

2. The method as recited in claim 1 , wherein computing the set of gains includes:

receiving a dominance criterion of the second audio stream over the first audio stream;

wherein the set of gains is computed based on the dominance criterion of the second audio stream over the first audio stream.

3. The method as recited in claim 2 , wherein the dominance criterion comprises a loudness ratio between the first audio stream and the second audio stream.

4. The method as recited in claim 2 , wherein the dominance criterion comprises a power ratio between the first audio stream and the second audio stream.

5. The method as recited in claim 1 , wherein the set of gains is derived under the constraint that the loudness of the combined first audio stream and second audio stream, after application of said set of gains, does not exceed the larger of the loudness values of the first audio stream and the second audio stream.

6. The method as recited in claim 1 , wherein computing the set of gains includes:

receiving an intelligibility criterion of the second audio;

wherein the set of gains is computed such that the intelligibility of the second audio stream is urged above the intelligibility criterion.

7. The method as recited in claim 6 , wherein the computation of the set of gains comprises deriving a measure of speech intelligibility of speech in the second audio stream.

8. The method as recited in claim 7 , wherein the measure of speech intelligibility is a speech intelligibility index.

9. The method as recited in claim 1 , further comprising:

determining whether the first audio stream comprises speech;

wherein the set of gains is based at least on the result of said determining.

10. The method as recited in claim 1 , further comprising:

determining whether the first audio stream comprises speech or non-speech content during an interval when both the first audio stream and the second audio stream are active;

wherein the set of gains for the first interval is based at least on whether the first audio stream comprises speech or non-speech content during the interval.

11. The method as recited in claim 1 , further comprising:

determining a confidence level that the first audio stream comprises speech during an interval when both the first audio stream and the second audio stream are active;

wherein the set of gains for the interval is based at least on the confidence level that the first audio stream comprises speech during the interval.

12. The method as recited in claim 1 , wherein the set of gains is calculated based on frequency sub-bands, wherein a first set of gains associated with a first frequency sub-band is different than a second set of gains associated with a second frequency sub-band.

13. The method as recited in claim 12 , further comprising:

receiving a first dominance criterion for the first frequency sub-band, wherein the first set of gains is computed based on the first dominance criterion;

receiving a second dominance criterion for the second frequency sub-band, wherein the second set of gains is computed based on the second dominance criterion.

14. The method as recited in claim 1 , further comprising one or more of:

transmitting the first audio stream, the second audio stream, and the metadata; or

mixing the first audio stream and the second audio stream based on the metadata.

15. The method as recited in claim 1 , wherein the set of gains is computed when the first audio stream and the second audio stream are active.

16. The method as recited in claim 1 , further comprising detecting overlapping signal time intervals when both the first audio stream and the second audio stream are active.

17. The method as recited in claim 1 , wherein the first audio stream comprises primary audio associated with media content, and wherein the second audio stream comprises descriptive audio associated with the media content.

18. The method as recited in claim 1 , wherein the first audio stream comprises a first set of one or more channels in a multi-channel program, wherein the second audio stream comprises a second set of one or more channels in the multi-channel program.

19. A method for processing an encoded audio signal generated according to the method of claim 1 , comprising:

receiving the encoded audio signal;

extracting, from the encoded audio signal, (a) the first audio stream, (b) the second audio stream, and (c) the metadata comprising the set of gains;

decoding the first audio stream and the second audio stream; and

mixing the first audio stream and the second audio stream based on the set of gains to create an output audio signal in which one of the first and second audio streams dominates the other of the first and second audio streams.

20. A non-transitory computer readable storage medium, comprising a set of instructions, which when executed by a processing or computing device cause, control or program the device to execute or perform a process, wherein the process comprises the steps of:

receiving a first audio stream and a second audio stream;

determining whether a first energy level for the first audio stream individually meets or exceeds an energy level threshold;

determining whether a second energy level for the second audio stream individually meets or exceeds the energy level threshold;

in response to determining that the first energy level for the first audio stream individually meets or exceeds the energy level threshold and determining that the second energy level for the second audio stream individually meets or exceeds the energy level threshold, computing a set of gains for at least one of the first audio stream and the second audio stream to create a dominance effect of the second audio stream over the first audio stream;

generating metadata comprising the set of gains for at least one of the first audio stream and the second audio stream;

generating an output audio signal comprising the first audio stream, the second audio stream, and the metadata comprising the set of gains.

21. An apparatus comprising:

a processor; and

a non-transitory computer readable storage medium, comprising a set of instructions, which when executed by the processor cause, control or program the apparatus, or the processor thereof, to perform a process that comprises the steps of:

receiving a first audio stream and a second audio stream;

determining whether a first energy level for the first audio stream individually meets or exceeds an energy level threshold;

determining whether a second energy level for the second audio stream individually meets or exceeds the energy level threshold;

in response to determining that the first energy level for the first audio stream individually meets or exceeds the energy level threshold and determining that the second energy level for the second audio stream individually meets or exceeds the energy level threshold, computing a set of gains for at least one of the first audio stream and the second audio stream to create a dominance effect of the second audio stream over the first audio stream;

generating metadata comprising the set of gains for at least one of the first audio stream and the second audio stream;

generating an output audio signal comprising the first audio stream, the second audio stream, and the metadata comprising the set of gains.

22. A system, comprising:

means for receiving a first audio stream and a second audio stream;

means for determining whether a first energy level for the first audio stream individually meets or exceeds an energy level threshold;

means for determining whether a second energy level for the second audio stream individually meets or exceeds the energy level threshold;

means for computing a set of gains for at least one of the first audio stream and the second audio stream to create a dominance effect of the second audio stream over the first audio stream in response to determining that the first energy level for the first audio stream individually meets or exceeds the energy level threshold and determining that the second energy level for the second audio stream individually meets or exceeds the energy level threshold;

means for generating metadata comprising the set of gains for at least one of the first audio stream and the second audio stream;

means for generating an output audio signal comprising the first audio stream, the second audio stream, and the metadata comprising the set of gains.

23. An integrated circuit (IC) device, comprising:

a die;

an array of active devices disposed within the die; and

a non-transitory storage medium, which is readable by one or more of the active components in the array and disposed within the die, wherein the component-readable non-transitory medium comprises a set of instructions, which when executed by the one or more active components cause, control, program or configure the IC device, the one or more active components in the array therein, or another set of active components of the array, to perform a process that comprises the steps of:

receiving a first audio stream and a second audio stream;

determining whether a first energy level for the first audio stream individually meets or exceeds an energy level threshold;

determining whether a second energy level for the second audio stream individually meets or exceeds the energy level threshold;

in response to determining that the first energy level for the first audio stream individually meets or exceeds the energy level threshold and determining that the second energy level for the second audio stream individually meets or exceeds the energy level threshold, computing a set of gains for at least one of the first audio stream and the second audio stream to create a dominance effect of the second audio stream over the first audio stream;

generating metadata comprising the set of gains for at least one of the first audio stream and the second audio stream;

generating an output audio signal comprising the first audio stream, the second audio stream, and the metadata comprising the set of gains.

24. The IC device as recited in claim 23 , wherein the IC device comprises one or more of:

a digital signal processor (DSP);

an audio processor;

a field programmable gate array (FPGA);

an application specific IC (ASIC);

a microcontroller; or

an IC device that comprises:

a processor, which comprises the active devices array; and

the component readable non-transitory storage medium.