Individual channel shaping for BCC schemes and the like
View Patent ↗At an audio encoder, cue codes are generated for one or more audio channels, wherein an envelope cue code is generated by characterizing a temporal envelope in an audio channel. At an audio decoder, E transmitted audio channel(s) are decoded to generate C playback audio channels, where C>E≧1. Received cue codes include an envelope cue code corresponding to a characterized temporal envelope of an audio channel corresponding to the transmitted channel(s). One or more transmitted channel(s) are upmixed to generate one or more upmixed channels. One or more playback channels are synthesized by applying the cue codes to the one or more upmixed channels, wherein the envelope cue code is applied to an upmixed channel or a synthesized signal to adjust a temporal envelope of the synthesized signal based on the characterized temporal envelope such that the adjusted temporal envelope substantially matches the characterized temporal envelope.
1. An encoder-implemented method for encoding audio channels, the method comprising:
an encoder generating one or more cue codes for one or more audio channels, wherein at least one cue code is an envelope cue code generated by characterizing a temporal envelope in one of the one or more audio channels; and
the encoder transmitting the one or more cue codes, wherein:
the one or more cue codes further comprise one or more of inter-channel correlation (ICC) codes, inter-channel level difference (ICLD) codes, and inter-channel time difference (ICTD) codes; and
a first time resolution associated with the envelope cue code is finer than a second time resolution associated with the other cue code(s).
2. The invention of claim 1 , further comprising transmitting E transmitted audio channel(s) corresponding to the one or more audio channels, where E≧1.
3. The invention of claim 2 , wherein:
the one or more audio channels comprise C input audio channels, where C>E; and
the C input channels are downmixed to generate the E transmitted channel(s).
4. The invention of claim 1 , wherein the one or more cue codes are transmitted to enable a decoder to perform envelope shaping during decoding of E transmitted channel(s) based on the one or more cue codes, wherein the E transmitted audio channel(s) correspond to the one or more audio channels, where E≧1.
5. The invention of claim 4 , wherein the envelope shaping adjusts a temporal envelope of a synthesized signal generated by the decoder to substantially match the characterized temporal envelope.
6. The invention of claim 1 , wherein the temporal envelope is characterized only for specified frequencies of the corresponding audio channel.
7. The invention of claim 6 , wherein the temporal envelope is characterized only for frequencies of the corresponding audio channel above a specified cutoff frequency.
8. The invention of claim 1 , wherein the temporal envelope is characterized for the corresponding audio channel in a frequency domain.
9. The invention of claim 8 , wherein temporal envelopes are characterized individually for different signal subbands in the corresponding audio channel.
10. The invention of claim 8 , wherein the frequency domain corresponds to a fast Fourier transform (FFT).
11. The invention of claim 8 , wherein the frequency domain corresponds to a quadrature mirror filter (QMF).
12. The invention of claim 1 , wherein the temporal envelope is characterized for the corresponding audio channel in a time domain.
13. The invention of claim 1 , further comprising determining whether to enable or disable the characterizing.
14. The invention of claim 13 , further comprising generating and transmitting an enable/disable flag based on the determining to instruct a decoder whether or not to implement envelope shaping during decoding of E transmitted channel(s) corresponding to the one or more audio channels, where E≧1.
15. The invention of claim 13 , wherein the determining is based on analyzing an audio channel to detect transients in the audio channel such that the characterizing is enabled if occurrence of a transient is detected.
16. Apparatus for encoding audio channels, the apparatus comprising:
means for generating one or more cue codes for one or more audio channels, wherein at least one cue code is an envelope cue code generated by characterizing a temporal envelope in one of the one or more audio channels; and
means for transmitting the one or more cue codes, wherein:
the one or more cue codes further comprise one or more of inter-channel correlation (ICC) codes, inter-channel level difference (ICLD) codes, and inter-channel time difference (ICTD) codes; and
a first time resolution associated with the envelope cue code is finer than a second time resolution associated with the other cue code(s).
17. Apparatus for encoding C input audio channels to generate E transmitted audio channel(s), the apparatus comprising:
an envelope analyzer adapted to characterize an input temporal envelope of at least one of the C input channels;
a code estimator adapted to generate cue codes for two or more of the C input channels; and
a downmixer adapted to downmix the C input channels to generate the E transmitted channel(s), where C>E≧1, wherein the apparatus is adapted to transmit information about the cue codes and the characterized input temporal envelope to enable a decoder to perform synthesis and envelope shaping during decoding of the E transmitted channel(s), wherein:
the cue codes further comprise one or more of inter-channel correlation (ICC) codes, inter-channel level difference (ICLD) codes, and inter-channel time difference (ICTD) codes; and
a first time resolution associated with the envelope cue code is finer than a second time resolution associated with the other cue code(s).
18. The invention of claim 17 , wherein:
the apparatus is a system selected from the group consisting of a digital video recorder, a digital audio recorder, a computer, a satellite transmitter, a cable transmitter, a terrestrial broadcast transmitter, a home entertainment system, and a movie theater system; and
the system comprises the envelope analyzer, the code estimator, and the downmixer.
19. A machine-readable storage medium, having encoded thereon program code, wherein, when the program code is executed by a machine, the machine implements a method for encoding audio channels, the method comprising:
generating one or more cue codes for one or more audio channels, wherein at least one cue code is an envelope cue code generated by characterizing a temporal envelope in one of the one or more audio channels; and
transmitting the one or more cue codes, wherein:
the one or more cue codes further comprise one or more of inter-channel correlation (ICC) codes, inter-channel level difference (ICLD) codes, and inter-channel time difference (ICTD) codes; and
a first time resolution associated with the envelope cue code is finer than a second time resolution associated with the other cue code(s).
20. A machine-readable storage medium, having encoded thereon an encoded audio bitstream generated by encoding audio channels, wherein:
one or more cue codes are generated for one or more audio channels, wherein at least one cue code is an envelope cue code generated by characterizing a temporal envelope in one of the one or more audio channels;
the one or more cue codes and E transmitted audio channel(s) corresponding to the one or more audio channels, where E≧1, are encoded onto the machine-readable medium as part of the encoded audio bitstream;
the one or more cue codes further comprise one or more of inter-channel correlation (ICC) codes, inter-channel level difference (ICLD) codes, and inter-channel time difference (ICTD) codes; and
a first time resolution associated with the envelope cue code is finer than a second time resolution associated with the other cue code(s).
21. A machine-readable storage medium, having encoded thereon an encoded audio bitstream comprising one or more cue codes and E transmitted audio channel(s), wherein:
the one or more cue codes are generated for one or more audio channels, wherein at least one cue code is an envelope cue code generated by characterizing a temporal envelope in one of the one or more audio channels;
the E transmitted audio channel(s) correspond to the one or more audio channels;
the one or more cue codes further comprise one or more of inter-channel correlation (ICC) codes, inter-channel level difference (ICLD) codes, and inter-channel time difference (ICTD) codes; and
a first time resolution associated with the envelope cue code is finer than a second time resolution associated with the other cue code(s).
22. A decoder-implemented method for decoding E transmitted audio channel(s) to generate C playback audio channels, where C>E≧1, the method comprising:
a decoder receiving cue codes corresponding to the E transmitted channel(s), wherein the cue codes comprise an envelope cue code corresponding to a characterized temporal envelope of an audio channel corresponding to the E transmitted channel(s);
the decoder upmixing one or more of the E transmitted channel(s) to generate one or more upmixed channels; and
the decoder synthesizing one or more of the C playback channels by applying the cue codes to the one or more upmixed channels, wherein the envelope cue code is applied to an upmixed channel or a synthesized signal to adjust a temporal envelope of the synthesized signal based on the characterized temporal envelope such that the adjusted temporal envelope substantially matches the characterized temporal envelope, wherein:
the cue codes further comprise one or more of inter-channel correlation (ICC) codes, inter-channel level difference (ICLD) codes, and inter-channel time difference (ICTD) codes; and
a first time resolution associated with the envelope cue code is finer than a second time resolution associated with the other cue code(s).
23. The invention of claim 22 , wherein the envelope cue code corresponds to a characterized temporal envelope in an original input channel used to generate the E transmitted channel(s).
24. The invention of claim 22 , wherein the synthesis comprises late-reverberation ICC synthesis.
25. The invention of claim 22 , wherein the temporal envelope of the synthesized signal is adjusted prior to ICLD synthesis.
26. The invention of claim 22 , wherein:
the temporal envelope of the synthesized signal is characterized; and
the temporal envelope of the synthesized signal is adjusted based on both the characterized temporal envelope corresponding to the envelope cue code and the characterized temporal envelope of the synthesized signal.
27. The invention of claim 26 , wherein:
a scaling function is generated based on the characterized temporal envelope corresponding to the envelope cue code and the characterized temporal envelope of the synthesized signal; and
the scaling function is applied to the synthesized signal.
28. The invention of claim 22 , further comprising adjusting a transmitted channel based on the characterized temporal envelope to generate a flattened channel, wherein the upmixing and synthesis are applied to the flattened channel to generate a corresponding playback channel.
29. The invention of claim 22 , further comprising adjusting an upmixed channel based on the characterized temporal envelope to generate a flattened channel, wherein the synthesis is applied to the flattened channel to generate a corresponding playback channel.
30. The invention of claim 22 , wherein the temporal envelope of the synthesized signal is adjusted only for specified frequencies.
31. The invention of claim 30 , wherein the temporal envelope of the synthesized signal is adjusted only for frequencies above a specified cutoff frequency.
32. The invention of claim 22 , wherein the temporal envelope of the synthesized signal is adjusted in a frequency domain.
33. The invention of claim 32 , wherein temporal envelopes are adjusted individually for different signal subbands in the synthesized signal.
34. The invention of claim 32 , wherein the frequency domain corresponds to a fast Fourier transform (FFT).
35. The invention of claim 32 , wherein the frequency domain corresponds to a quadrature mirror filter (QMF).
36. The invention of claim 22 , wherein the temporal envelope of the synthesized signal is adjusted in a time domain.
37. The invention of claim 22 , further comprising determining whether to enable or disable the adjusting of the temporal envelope of the synthesized signal.
38. The invention of claim 37 , wherein the determining is based on an enable/disable flag generated by an audio encoder that generated the E transmitted channel(s).
39. The invention of claim 37 , wherein the determining is based on analyzing the E transmitted channel(s) to detect transients such that the adjusting is enabled if occurrence of a transient is detected.
40. The invention of claim 22 , further comprising:
characterizing a temporal envelope of a transmitted channel; and
determining whether to use (1) the characterized temporal envelope corresponding to the envelope cue code or (2) the characterized temporal envelope of the transmitted channel to adjust the temporal envelope of the synthesized signal.
41. The invention of claim 22 , wherein power within a specified window of the synthesized signal after adjusting the temporal envelope is substantially equal to power within a corresponding window of the synthesized signal before the adjusting.
42. The invention of claim 41 , wherein the specified window corresponds to a synthesis window associated with one or more non-envelope cue codes.
43. Apparatus for decoding E transmitted audio channel(s) to generate C playback audio channels, where C>E≧1, the apparatus comprising:
means for receiving cue codes corresponding to the E transmitted channel(s), wherein the cue codes comprise an envelope cue code corresponding to a characterized temporal envelope of an audio channel corresponding to the E transmitted channels;
means for upmixing one or more of the E transmitted channels to generate one or more upmixed channels; and
means for synthesizing one or more of the C playback channels by applying the cue codes to the one or more upmixed channels, wherein the envelope cue code is applied to an upmixed channel or a synthesized signal to adjust a temporal envelope of the synthesized signal based on the characterized temporal envelope such that the adjusted temporal envelope substantially matches the characterized temporal envelope, wherein:
the cue codes further comprise one or more of inter-channel correlation (ICC) codes, inter-channel level difference (ICLD) codes, and inter-channel time difference (ICTD) codes; and
a first time resolution associated with the envelope cue code is finer than a second time resolution associated with the other cue code(s).
44. Apparatus for decoding E transmitted audio channel(s) to generate C playback audio channels, where C>E≧1, the apparatus comprising:
a receiver adapted to receive cue codes corresponding to the E transmitted channel(s), wherein the cue codes comprise an envelope cue code corresponding to a characterized temporal envelope of an audio channel corresponding to the E transmitted channels;
an upmixer adapted to upmix one or more of the E transmitted channels to generate one or more upmixed channels; and
a synthesizer adapted to synthesize one or more of the C playback channels by applying the cue codes to the one or more upmixed channels, wherein the envelope cue code is applied to an upmixed channel or a synthesized signal to adjust a temporal envelope of the synthesized signal based on the characterized temporal envelope such that the adjusted temporal envelope substantially matches the characterized temporal envelope, wherein:
the cue codes further comprise one or more of inter-channel correlation (ICC) codes, inter-channel level difference (ICLD) codes, and inter-channel time difference (ICTD) codes; and
a first time resolution associated with the envelope cue code is finer than a second time resolution associated with the other cue code(s).
45. The invention of claim 44 , wherein:
the apparatus is a system selected from the group consisting of a digital video player, a digital audio player, a computer, a satellite receiver, a cable receiver, a terrestrial broadcast receiver, a home entertainment system, and a movie theater system; and
the system comprises the receiver, the upmixer, the synthesizer, and the envelope adjuster.
46. A machine-readable storage medium, having encoded thereon program code, wherein, when the program code is executed by a machine, the machine implements a method for decoding E transmitted audio channel(s) to generate C playback audio channels, where C>E≧1, the method comprising:
receiving cue codes corresponding to the E transmitted channel(s), wherein the cue codes comprise an envelope cue code corresponding to a characterized temporal envelope of an audio channel corresponding to the E transmitted channel(s);
upmixing one or more of the E transmitted channel(s) to generate one or more upmixed channels; and
synthesizing one or more of the C playback channels by applying the cue codes to the one or more upmixed channels, wherein the envelope cue code is applied to an upmixed channel or a synthesized signal to adjust a temporal envelope of the synthesized signal based on the characterized temporal envelope such that the adjusted temporal envelope substantially matches the characterized temporal envelope, wherein:
the cue codes further comprise one or more of inter-channel correlation (ICC) codes, inter-channel level difference (ICLD) codes, and inter-channel time difference (ICTD) codes; and
a first time resolution associated with the envelope cue code is finer than a second time resolution associated with the other cue code(s).