IP Library › Granted Patent US 11,935,547
Granted Patent B2
US 11,935,547 · App. 17/400,289 · Granted Mar 19, 2024

Method for determining audio coding/decoding mode and related product

Inventors: Bin Wang (Shenzhen, CN); Haiting Li (Beijing, CN); Lei Miao (Beijing, CN)
Assignee: HUAWEI TECHNOLOGIES CO., LTD.
G10L19/008G10L19/22
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,935,547
App. No.
17/400,289
Granted
Mar 19, 2024
Kind
B2
Abstract

A non-transitory computer-readable medium is provided. The non-transitory computer-readable medium having computer instructions stored therein, which when executed by one or more processors, cause the one or more processors to perform operations. The operations comprise: determining a channel combination scheme for a current frame, where the determined channel combination scheme for the current frame is one of a plurality of channel combination schemes; and determining a coding mode of the current frame based on a channel combination scheme for a previous frame and the channel combination scheme for the current frame, where the coding mode of the current frame is one of a plurality of coding modes.

Claims (89)

1. A non-transitory computer-readable medium having computer instructions stored therein, which when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising:

determining a channel combination scheme for a current frame of a multi-channel signal, wherein the multi-channel signal includes at least two channels; and

determining a coding mode of the current frame based on a channel combination scheme for a previous frame and the channel combination scheme for the current frame;

wherein the channel combination scheme for the current frame is one of a plurality of channel combination schemes that comprise an anticorrelated signal channel combination scheme and a correlated signal channel combination scheme, the correlated signal channel combination scheme is a channel combination scheme corresponding to a near in phase signal, and the anticorrelated signal channel combination scheme is a channel combination scheme corresponding to a near out of phase signal;

wherein

determining the channel combination scheme for the current frame comprises:

performing an initial channel combination scheme decision for the current frame to determine an initial channel combination scheme for the current frame; and

performing a channel combination scheme modification decision for the current frame based on the initial channel combination scheme for the current frame to determine the channel combination scheme for the current frame;

performing the initial channel combination scheme decision for the current frame comprises:

determining a signal type of in/out of phase of a stereo signal in the current frame by using left and right channel signals in the current frame; and

determining the initial channel combination scheme for the current frame based on the signal type of in/out of phase of the stereo signal in the current frame and the channel combination scheme for the previous frame;

determining the initial channel combination scheme for the current frame based on the signal type of in/out of phase of the stereo signal in the current frame and the channel combination scheme for the previous frame comprises:

when the signal type of in/out of phase of the stereo signal in the current frame is the near in phase signal and the channel combination scheme for the previous frame is the correlated signal channel combination scheme, determining that the initial channel combination scheme for the current frame is the correlated signal channel combination scheme; or

when the signal type of in/out of phase of the stereo signal in the current frame is the near out of phase signal and the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme, determining that the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

when the signal type of in/out of phase of the stereo signal in the current frame is the near in phase signal and the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme,

if signal-to-noise ratios of the left and right channel signals in the current frame are both less than a second threshold, determining that the initial channel combination scheme for the current frame is the correlated signal channel combination scheme; or

if the signal-to-noise ratio of the left channel signal or the signal-to-noise ratio of the right channel signal in the current frame is greater than or equal to the second threshold, determining that the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

when the signal type of in/out of phase of the stereo signal in the current frame is the near out of phase signal and the channel combination scheme for the previous frame is the correlated signal channel combination scheme,

if the signal-to-noise ratios of the left and right channel signals in the current frame are both less than the second threshold, determining that the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

if the signal-to-noise ratio of the left channel signal or the signal-to-noise ratio of the right channel signal in the current frame is greater than or equal to the second threshold, determining that the initial channel combination scheme for the current frame is the correlated signal channel combination scheme.

2. The non-transitory computer-readable medium according to claim 1 , wherein performing the channel combination scheme modification decision for the current frame comprises:

if a channel combination ratio factor modification flag of the previous frame indicates that a channel combination ratio factor needs to be modified, determining that the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

if the channel combination ratio factor modification flag of the previous frame indicates that the channel combination ratio factor does not need to be modified, determining whether the current frame meets a switching condition, and determining the channel combination scheme for the current frame based on a result of determining whether the current frame meets the switching condition.

3. The non-transitory computer-readable medium according to claim 2 , wherein determining the channel combination scheme for the current frame based on the result of determining whether the current frame meets the switching condition comprises:

when the channel combination scheme for the previous frame is different from the initial channel combination scheme for the current frame, determining that the current frame meets the switching condition, the initial channel combination scheme for the current frame is the correlated signal channel combination scheme, and the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme, and determining that the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

when the channel combination scheme for the previous frame is different from the initial channel combination scheme for the current frame, determining that the current frame meets the switching condition, the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and a channel combination ratio factor of the previous frame is less than a first ratio factor threshold, and determining that the channel combination scheme for the current frame is the correlated signal channel combination scheme; or

when the channel combination scheme for the previous frame is different from the initial channel combination scheme for the current frame, determining that the current frame meets the switching condition, the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and the channel combination ratio factor of the previous frame is greater than or equal to the first ratio factor threshold, and determining that the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

when a channel combination scheme for the (P−1) th -to-current frame is different from an initial channel combination scheme for the P th -to-current frame, determining that the P th -to-current frame does not meet the switching condition, the current frame meets the switching condition, the signal type of in/out of phase of the stereo signal in the current frame is the near in phase signal, the initial channel combination scheme for the current frame is the correlated signal channel combination scheme, and the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme, and determining that the channel combination scheme for the current frame is the correlated signal channel combination scheme; or

when the channel combination scheme for the (P−1) th -to-current frame is different from the initial channel combination scheme for the P th -to-current frame, determining that the P th -to-current frame does not meet the switching condition, the current frame meets the switching condition, the signal type of in/out of phase of the stereo signal in the current frame is the near out of phase signal, the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and the channel combination ratio factor of the previous frame is less than a second ratio factor threshold, and determining that the channel combination scheme for the current frame is the correlated signal channel combination scheme; or

when the channel combination scheme for the (P−1) th -to-current frame is different from the initial channel combination scheme for the P th -to-current frame, determining that the P th -to-current frame does not meet the switching condition, the current frame meets the switching condition, the signal type of in/out of phase of the stereo signal in the current frame is the near out of phase signal, the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and the channel combination ratio factor of the previous frame is greater than or equal to the second ratio factor threshold, and determining that the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme.

4. The non-transitory computer-readable medium according to claim 3 , wherein

determining whether the current frame meets the switching condition comprises: determining, based on a frame type of a primary channel signal in the previous frame or a frame type of a secondary channel signal in the previous frame, whether the current frame meets the switching condition.

5. The non-transitory computer-readable medium according to claim 4 , wherein determining, based on the frame type of the primary channel signal in the previous frame or the frame type of the secondary channel signal in the previous frame, whether the current frame meets the switching condition comprises:

when a first condition, a second condition, and a third condition are all met, determining that the current frame meets the switching condition; or

when the second condition, the third condition, a fourth condition, and a fifth condition are all met, determining that the current frame meets the switching condition; or

when a sixth condition is met, determining that the current frame meets the switching condition; wherein

the first condition is: a frame type of a primary channel signal in a previous frame of the previous frame is any one of the following: a VOICED_CLAS frame, an ONSET frame, a SIN_ONSET frame, an INACTIVE_CLAS frame, or an AUDIO_CLAS frame, and the frame type of the primary channel signal in the previous frame is a UNVOICED_CLAS frame or a VOICED_TRANSITION frame; or a frame type of a secondary channel signal in the previous frame of the previous frame is any one of the following: a VOICED_CLAS frame, an ONSET frame, a SIN_ONSET frame, an INACTIVE_CLAS frame, or an AUDIO_CLAS frame, and the frame type of the secondary channel signal in the previous frame is an UNVOICED_CLAS frame or a VOICED_TRANSITION frame;

the second condition is: not of raw coding modes of the primary channel signal and the secondary channel signal in the previous frame is a coding type corresponding to VOICED;

the third condition is: a quantity of consecutive frames before the previous frame that use the channel combination scheme used by the previous frame is greater than a preset frame quantity threshold;

the fourth condition is: the frame type of the primary channel signal in the previous frame is an UNVOICED_CLAS frame, or the frame type of the secondary channel signal in the previous frame is an UNVOICED_CLAS frame;

the fifth condition is: a long-term root mean square energy value of the left and right channel signals in the current frame is less than an energy threshold; and

the sixth condition is: the frame type of the primary channel signal in the previous frame is a music signal, a ratio of energy of a lower frequency band to energy of a higher frequency band of the primary channel signal in the previous frame is greater than a first energy ratio threshold, and a ratio of energy of a lower frequency band to energy of a higher frequency band of the secondary channel signal in the previous frame is greater than a second energy ratio threshold.

6. The non-transitory computer-readable medium according to claim 1 , wherein determining the coding mode of the current frame based on the channel combination scheme for the previous frame and the channel combination scheme for the current frame comprises:

when the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, determining that the coding mode of the current frame is a correlated-to-anticorrelated signal coding switching mode, wherein in the correlated-to-anticorrelated signal coding switching mode, time-domain downmix processing is performed by using a downmix processing method corresponding to a transition from the correlated signal channel combination scheme to the anticorrelated signal channel combination scheme; or

when the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme, and the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, determining that the coding mode of the current frame is the anticorrelated signal coding mode, wherein in the anticorrelated signal coding mode, the time-domain downmix processing is performed by using a downmix processing method corresponding to the anticorrelated signal channel combination scheme; or

when the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme, and the channel combination scheme for the current frame is the correlated signal channel combination scheme, determining that the coding mode of the current frame is an anticorrelated-to-correlated signal coding switching mode, wherein in the anticorrelated-to-correlated signal coding switching mode, the time-domain downmix processing is performed by using a downmix processing method corresponding to a transition from the anticorrelated signal channel combination scheme to the correlated signal channel combination scheme; or

when the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and the channel combination scheme for the current frame is the correlated signal channel combination scheme, determining that the coding mode of the current frame is the correlated signal coding mode, wherein in the correlated signal coding mode, the time-domain downmix processing is performed by using a downmix processing method corresponding to the correlated signal channel combination scheme.

7. An audio signal encoder, comprising:

a computer-readable medium to store instructions, which when executed by one or more processors, cause the audio signal encoder to:

determine a channel combination scheme for a current frame of a multi-channel signal, wherein the multi-channel signal includes at least two channels; and

determine a coding mode of the current frame based on a channel combination scheme for a previous frame and the channel combination scheme for the current frame;

wherein the channel combination scheme for the current frame is one of a plurality of channel combination schemes that comprise an anticorrelated signal channel combination scheme and a correlated signal channel combination scheme, the correlated signal channel combination scheme is a channel combination scheme corresponding to a near in phase signal, and the anticorrelated signal channel combination scheme is a channel combination scheme corresponding to a near out of phase signal;

wherein

to determine the channel combination scheme for the current frame, the instructions, which when executed by the one or more processor, cause the audio signal encoder to perform an initial channel combination scheme decision for the current frame to determine an initial channel combination scheme for the current frame; and to perform a channel combination scheme modification decision for the current frame based on the initial channel combination scheme for the current frame to determine the channel combination scheme for the current frame;

to perform the initial channel combination scheme decision for the current frame, the instructions, which when executed by the one or more processor, cause the audio signal encoder to determine a signal type of in/out of phase of a stereo signal in the current frame by using left and right channel signals in the current frame; and to determine the initial channel combination scheme for the current frame based on the signal type of in/out of phase of the stereo signal in the current frame and the channel combination scheme for the previous frame;

to determine the initial channel combination scheme for the current frame based on the signal type of in/out of phase of the stereo signal in the current frame and the channel combination scheme for the previous frame, the instructions, which when executed by the one or more processor, cause the audio signal encoder to

when the signal type of in/out of phase of the stereo signal in the current frame is the near in phase signal and the channel combination scheme for the previous frame is the correlated signal channel combination scheme, determine that the initial channel combination scheme for the current frame is the correlated signal channel combination scheme; or when the signal type of in/out of phase of the stereo signal in the current frame is the near out of phase signal and the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme, determine that the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

when the signal type of in/out of phase of the stereo signal in the current frame is the near in phase signal and the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme,

if signal-to-noise ratios of the left and right channel signals in the current frame are both less than a second threshold, determine that the initial channel combination scheme for the current frame is the correlated signal channel combination scheme; or

if the signal-to-noise ratio of the left channel signal or the signal-to-noise ratio of the right channel signal in the current frame are/is greater than or equal to the second threshold, determine that the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

when the signal type of in/out of phase of the stereo signal in the current frame is the near out of phase signal and the channel combination scheme for the previous frame is the correlated signal channel combination scheme,

if the signal-to-noise ratios of the left and right channel signals in the current frame are both less than the second threshold, determine that the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

if the signal-to-noise ratio of the left channel signal or the signal-to-noise ratio of the right channel signal in the current frame are/is greater than or equal to the second threshold, determine that the initial channel combination scheme for the current frame is the correlated signal channel combination scheme.

8. The audio signal encoder according to claim 7 , wherein to perform the channel combination scheme modification decision for the current frame based on the initial channel combination scheme for the current frame to determine the channel combination scheme for the current frame, the instructions, which when executed by the one or more processor, cause the audio signal encoder to

if a channel combination ratio factor modification flag of the previous frame indicates that a channel combination ratio factor needs to be modified, determine that the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

if the channel combination ratio factor modification flag of the previous frame indicates that the channel combination ratio factor does not need to be modified, determine whether the current frame meets a switching condition, and determine the channel combination scheme for the current frame based on a result of determining whether the current frame meets the switching condition.

9. The audio signal encoder according to claim 8 , wherein to determine the channel combination scheme for the current frame based on the result of determining whether the current frame meets the switching condition, the instructions, which when executed by the one or more processor, cause the audio signal encoder to

when the channel combination scheme for the previous frame is different from the initial channel combination scheme for the current frame, determine that the current frame meets the switching condition, the initial channel combination scheme for the current frame is the correlated signal channel combination scheme, and the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme, and determine that the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

when the channel combination scheme for the previous frame is different from the initial channel combination scheme for the current frame, determine that the current frame meets the switching condition, the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and a channel combination ratio factor of the previous frame is less than a first ratio factor threshold, and determine that the channel combination scheme for the current frame is the correlated signal channel combination scheme; or

when the channel combination scheme for the previous frame is different from the initial channel combination scheme for the current frame, determine that the current frame meets the switching condition, the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and the channel combination ratio factor of the previous frame is greater than or equal to a first ratio factor threshold, and determine that the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme; or

when a channel combination scheme for the (P−1) th -to-current frame is different from an initial channel combination scheme for the P th -to-current frame, determine that the P th -to-current frame does not meet the switching condition, the current frame meets the switching condition, the signal type of in/out of phase of the stereo signal in the current frame is the near in phase signal, the initial channel combination scheme for the current frame is the correlated signal channel combination scheme, and the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme, and determine that the channel combination scheme for the current frame is the correlated signal channel combination scheme; or

when the channel combination scheme for the (P−1) th -to-current frame is different from the initial channel combination scheme for the P th -to-current frame, determine that the P th -to-current frame does not meet the switching condition, the current frame meets the switching condition, the signal type of in/out of phase of the stereo signal in the current frame is the near out of phase signal, the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and the channel combination ratio factor of the previous frame is less than a second ratio factor threshold, and determine that the channel combination scheme for the current frame is the correlated signal channel combination scheme; or

when the channel combination scheme for the (P−1) th frame is different from the initial channel combination scheme for the P th frame, determine that the P th frame does not meet the switching condition, the current frame meets the switching condition, the signal type of in/out of phase of the stereo signal in the current frame is the near out of phase signal, the initial channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and the channel combination ratio factor of the previous frame is greater than or equal to the second ratio factor threshold, and determine that the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme.

10. The audio signal encoder according to claim 9 , wherein to determine whether the current frame meets the switching condition, the instructions, which when executed by the one or more processor, cause the audio signal encoder to determine, based on a frame type of a primary channel signal in the previous frame or a frame type of a secondary channel signal in the previous frame, whether the current frame meets the switching condition.

11. The audio signal encoder according to claim 10 , wherein to determine, based on the frame type of the primary channel signal in the previous frame or the frame type of the secondary channel signal in the previous frame, whether the current frame meets the switching condition, the instructions, which when executed by the one or more processor, cause the audio signal encoder to

when a first condition, a second condition, and a third condition are all met, determine that the current frame meets the switching condition; or

when the second condition, the third condition, a fourth condition, and a fifth condition are all met, determine that the current frame meets the switching condition; or

when a sixth condition is met, determine that the current frame meets the switching condition; wherein

the first condition is: a frame type of a primary channel signal in a previous frame of the previous frame is any one of the following: a VOICED_CLAS frame, an ONSET frame, a SIN_ONSET frame, an INACTIVE_CLAS frame, or an AUDIO_CLAS frame, and the frame type of the primary channel signal in the previous frame is a UNVOICED_CLAS frame or a VOICED_TRANSITION frame; or a frame type of a secondary channel signal in the previous frame of the previous frame is any one of the following: a VOICED_CLAS frame, an ONSET frame, a SIN_ONSET frame, an INACTIVE_CLAS frame, or an AUDIO_CLAS frame, and the frame type of the secondary channel signal in the previous frame is an UNVOICED_CLAS frame or a VOICED_TRANSITION frame;

the second condition is: not of raw coding modes of the primary channel signal and the secondary channel signal in the previous frame is a coding type corresponding to VOICED;

the third condition is: a quantity of consecutive frames before the previous frame that use the channel combination scheme used by the previous frame is greater than a preset frame quantity threshold;

the fourth condition is: the frame type of the primary channel signal in the previous frame is an UNVOICED_CLAS frame, or the frame type of the secondary channel signal in the previous frame is an UNVOICED_CLAS frame;

the fifth condition is: a long-term root mean square energy value of the left and right channel signals in the current frame is less than an energy threshold; and

the sixth condition is: the frame type of the primary channel signal in the previous frame is a music signal, a ratio of energy of a lower frequency band to energy of a higher frequency band of the primary channel signal in the previous frame is greater than a first energy ratio threshold, and a ratio of energy of a lower frequency band to energy of a higher frequency band of the secondary channel signal in the previous frame is greater than a second energy ratio threshold.

12. The audio signal encoder according to claim 7 , wherein to determine the coding mode of the current frame based on the channel combination scheme for the previous frame and the channel combination scheme for the current frame, the instructions, which when executed by the one or more processor, cause the audio signal encoder to

when the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, determine that the coding mode of the current frame is a correlated-to-anticorrelated signal coding switching mode, wherein in the correlated-to-anticorrelated signal coding switching mode, time-domain downmix processing is performed by using a downmix processing method corresponding to a transition from the correlated signal channel combination scheme to the anticorrelated signal channel combination scheme; or

when the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme, and the channel combination scheme for the current frame is the anticorrelated signal channel combination scheme, determine that the coding mode of the current frame is the anticorrelated signal coding mode, wherein in the anticorrelated signal coding mode, the time-domain downmix processing is performed by using a downmix processing method corresponding to the anticorrelated signal channel combination scheme; or

when the channel combination scheme for the previous frame is the anticorrelated signal channel combination scheme, and the channel combination scheme for the current frame is the correlated signal channel combination scheme, determine that the coding mode of the current frame is an anticorrelated-to-correlated signal coding switching mode, wherein in the anticorrelated-to-correlated signal coding switching mode, the time-domain downmix processing is performed by using a downmix processing method corresponding to a transition from the anticorrelated signal channel combination scheme to the correlated signal channel combination scheme; or

when the channel combination scheme for the previous frame is the correlated signal channel combination scheme, and the channel combination scheme for the current frame is the correlated signal channel combination scheme, determine that the coding mode of the current frame is the correlated signal coding mode, wherein in the correlated signal coding mode, the time-domain downmix processing is performed by using a downmix processing method corresponding to the correlated signal channel combination scheme.

Priority Claims (1)
CN 201710679081.6 · Aug 10, 2017 · national
Continuity (3)
Continuation 16785274 · Feb 7, 2020
Continuation PCTCN2018100100 · Aug 10, 2018
Related Publication 20210375292A1 · Dec 2, 2021