Granted Patent B2

US 12,738,287 · App. 18/643,717 · Granted Sep 15, 2026

Inventors: Yupeng Shi (Shenzhen, CN); Wei Xiao (Shenzhen, CN); Meng Wang (Shenzhen, CN); Yuyong Kang (Shenzhen, CN); Qingbo Huang (Shenzhen, CN)

Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

G10L19/06G10L19/0204G10L21/02G10L19/04

Abstract

Claims (111)

1 . An audio decoding method, executed by an electronic device, and comprising:

obtaining a bitstream of an audio signal;

performing label extraction processing on a predicted value of a feature vector of the audio signal associated with the bitstream to obtain a label information vector, comprising:

performing convolution processing on the predicted value of the feature vector to obtain a first tensor having a same dimension as the predicted value of the feature vector;

performing feature extraction processing on the first tensor to obtain a second tensor having a same dimension as the first tensor;

performing full-connection processing on the second tensor to obtain a third tensor having a same dimension as the second tensor; and

activating the third tensor to obtain the label information vector, a dimension of the label information vector being the same as a dimension of the predicted value of the feature vector;

performing signal reconstruction based on the predicted value of the feature vector and the label information vector; and

identifying a predicted value of the audio signal obtained by the signal reconstruction as a decoding result of the bitstream.

2 . The method according to claim 1 , wherein the performing signal reconstruction based on the predicted value of the feature vector and the label information vector comprises:

splicing the predicted value of the feature vector and the label information vector to obtain a spliced vector; and

compressing the spliced vector to obtain the predicted value of the audio signal.

3 . The method according to claim 1 , further comprising:

decoding the bitstream to obtain an index value of the feature vector of the audio signal; and

querying a quantization table based on the index value to obtain the predicted value of the feature vector of the audio signal.

4 . The method according to claim 2 , wherein the compressing the spliced vector to obtain the predicted value of the audio signal comprises:

performing first convolution processing on the spliced vector to obtain a convolution feature of the audio signal;

upsampling the convolution feature to obtain an upsampled feature of the audio signal;

performing pooling processing on the upsampled feature to obtain a pooled feature of the audio signal; and

performing second convolution processing on the pooled feature to obtain the predicted value of the audio signal.

5 . The method according to claim 4 , wherein

the upsampling process uses a plurality of cascaded decoding layers, and sampling factors of different decoding layers are different; and

the upsampling the convolution feature to obtain an upsampled feature of the audio signal comprises:

upsampling the convolution feature by using the first decoding layer among the plurality of cascaded decoding layers;

outputting an upsampling result of the first decoding layer to a subsequent cascaded decoding layer, and repeating upsampling processing and outputting upsampling result by using the subsequent cascaded decoding layer until an output reaches the last decoding layer; and

identifying an upsampling result output by the last decoding layer as the upsampled feature of the audio signal.

6 . The method according to claim 3 , wherein

the bitstream comprises a low-frequency bitstream and a high-frequency bitstream, the low-frequency bitstream being obtained by coding a low-frequency sub-band signal obtained by decomposing the audio signal, and the high-frequency bitstream being obtained by coding a high-frequency sub-band signal obtained by decomposing the audio signal; and

the decoding the bitstream to obtain a predicted value of a feature vector of the audio signal comprises:

decoding the low-frequency bitstream to obtain a predicted value of a feature vector of the low-frequency sub-band signal; and

decoding the high-frequency bitstream to obtain a predicted value of a feature vector of the high-frequency sub-band signal.

7 . The method according to claim 6 , wherein the performing label extraction processing on the predicted value of the feature vector to obtain a label information vector comprises:

performing label extraction processing on the predicted value of the feature vector of the low-frequency sub-band signal to obtain a first label information vector, a dimension of the first label information vector being the same as a dimension of the predicted value of the feature vector of the low-frequency sub-band signal; and

performing label extraction processing on the predicted value of the feature vector of the high-frequency sub-band signal to obtain a second label information vector, a dimension of the second label information vector being the same as a dimension of the predicted value of the feature vector of the high-frequency sub-band signal.

8 . The method according to claim 6 , wherein the performing label extraction processing on the predicted value of the feature vector of the low-frequency sub-band signal to obtain a first label information vector comprises:

invoking a first enhancement network to perform the following processing:

performing convolution processing on the predicted value of the feature vector of the low-frequency sub-band signal to obtain a fourth tensor having a same dimension as the predicted value of the feature vector of the low-frequency sub-band signal;

performing feature extraction processing on the fourth tensor to obtain a fifth tensor having a same dimension as the fourth tensor;

performing full-connection processing on the fifth tensor to obtain a sixth tensor having a same dimension as the fifth tensor; and

activating the sixth tensor to obtain the first label information vector.

9 . The method according to claim 6 , wherein the performing label extraction processing on the predicted value of the feature vector of the high-frequency sub-band signal to obtain a second label information vector comprises:

invoking a second enhancement network to perform the following processing:

performing convolution processing on the predicted value of the feature vector of the high-frequency sub-band signal to obtain a seventh tensor having a same dimension as the predicted value of the feature vector of the high-frequency sub-band signal;

performing feature extraction processing on the seventh tensor to obtain an eighth tensor having a same dimension as the seventh tensor;

performing full-connection processing on the eighth tensor to obtain a ninth tensor having a same dimension as the eighth tensor; and

activating the ninth tensor to obtain the second label information vector.

10 . The method according to claim 7 , wherein

the predicted value of the feature vector comprises: the predicted value of the feature vector of the low-frequency sub-band signal and the predicted value of the feature vector of the high-frequency sub-band signal; and

the performing signal reconstruction based on the predicted value of the feature vector and the label information vector comprises:

splicing the predicted value of the feature vector of the low-frequency sub-band signal and the first label information vector to obtain a first spliced vector;

invoking, based on the first spliced vector, a first synthesis network for signal reconstruction to obtain a predicted value of the low-frequency sub-band signal;

splicing the predicted value of the feature vector of the high-frequency sub-band signal and the second label information vector to obtain a second spliced vector;

invoking, based on the second spliced vector, a second synthesis network for signal reconstruction to obtain a predicted value of the high-frequency sub-band signal; and

synthesizing the predicted value of the low-frequency sub-band signal and the predicted value of the high-frequency sub-band signal to obtain the predicted value of the audio signal.

11 . The method according to claim 10 , wherein the invoking, based on the first spliced vector, a first synthesis network for signal reconstruction to obtain a predicted value of the low-frequency sub-band signal comprises:

invoking the first synthesis network to perform the following processing:

performing first convolution processing on the first spliced vector to obtain a convolution feature of the low-frequency sub-band signal;

upsampling the convolution feature to obtain an upsampled feature of the low-frequency sub-band signal;

performing pooling processing on the upsampled feature to obtain a pooled feature of the low-frequency sub-band signal; and

performing second convolution processing on the pooled feature to obtain the predicted value of the low-frequency sub-band signal,

the upsampling process being implemented by using a plurality of cascaded decoding layers, and sampling factors of different decoding layers being different.

12 . The method according to claim 10 , wherein the invoking, based on the second spliced vector, a second synthesis network for signal reconstruction to obtain a predicted value of the high-frequency sub-band signal comprises:

invoking the second synthesis network to perform the following processing:

performing first convolution processing on the second spliced vector to obtain a convolution feature of the high-frequency sub-band signal;

upsampling the convolution feature to obtain an upsampled feature of the high-frequency sub-band signal;

performing pooling processing on the upsampled feature to obtain a pooled feature of the high-frequency sub-band signal; and

performing second convolution processing on the pooled feature to obtain the predicted value of the high-frequency sub-band signal,

the upsampling process being implemented by using the plurality of cascaded decoding layers, and sampling factors of different decoding layers being different.

13 . The method according to claim 3 , wherein

the bitstream comprises N sub-bitstreams, the N sub-bitstreams corresponding to different frequency bands and being obtained by coding N sub-band signals obtained by decomposing the audio signal, and N being an integer greater than 2; and

the decoding the bitstream to obtain a predicted value of a feature vector of the audio signal comprises:

decoding the N sub-bitstreams respectively to obtain predicted values of feature vectors corresponding to the N sub-band signals, respectively.

14 . The method according to claim 13 , wherein the performing label extraction processing on the predicted value of the feature vector to obtain a label information vector comprises:

performing label extraction processing on the predicted values of the feature vectors corresponding to the N sub-band signals respectively to obtain N label information vectors, a dimension of each label information vector being the same as a dimension of the predicted value of the feature vector corresponding to the sub-band signal.

15 . The method according to claim 14 , wherein the performing label extraction processing on the predicted values of the feature vectors corresponding to the N sub-band signals respectively to obtain N label information vectors comprises:

invoking, based on a predicted value of a feature vector of an i th sub-band signal, an i th enhancement network for label extraction processing to obtain an i th label information vector,

a value range of i satisfying that i is greater than or equal to 1 and is smaller than or equal to N, and a dimension of the i th label information vector being the same as a dimension of the predicted value of the feature vector of the i th sub-band signal.

16 . The method according to claim 15 , wherein the invoking, based on a predicted value of a feature vector of an i th sub-band signal, an i th enhancement network for label extraction processing to obtain an i th label information vector comprises:

invoking the i th enhancement network to perform the following processing:

performing convolution processing on the predicted value of the feature vector of the i th sub-band signal to obtain a tenth tensor having a same dimension as the predicted value of the feature vector of the i th sub-band signal;

performing feature extraction processing on the tenth tensor to obtain an eleventh tensor having a same dimension as the tenth tensor;

performing full-connection processing on the eleventh tensor to obtain a twelfth tensor having a same dimension as the eleventh tensor; and

activating the twelfth tensor to obtain the i th label information vector.

17 . The method according to claim 14 , wherein the performing signal reconstruction based on the predicted value of the feature vector and the label information vector comprises:

splicing the predicted values of the feature vectors corresponding to the N sub-band signals respectively and the N label information vectors one-to-one to obtain N spliced vectors;

invoking, based on a j th spliced vector, a j th synthesis network for signal reconstruction to obtain a predicted value of a j th sub-band signal, a value range of j satisfying that j is greater than or equal to 1 and is smaller than or equal to N;

performing first convolution processing on the j th spliced vector to obtain a convolution feature of the j th sub-band signal;

upsampling the convolution feature to obtain an upsampled feature of the j th sub-band signal;

performing pooling processing on the upsampled feature to obtain a pooled feature of the j th sub-band signal; and

performing second convolution processing on the pooled feature to obtain the predicted value of the j th sub-band signal, the upsampling process being implemented by using the plurality of cascaded decoding layers, and sampling factors of different decoding layers being different; and

synthesizing predicted values corresponding to the N sub-band signals respectively to obtain the predicted value of the audio signal.

18 . An electronic device, comprising:

a memory, configured to store computer-executable instructions; and

a processor, configured, when executing the computer-executable instructions stored in the memory, to implement:

obtaining a bitstream of an audio signal;

performing label extraction processing on a predicted value of a feature vector of the audio signal associated with the bitstream to obtain a label information vector, comprising:

performing convolution processing on the predicted value of the feature vector to obtain a first tensor having a same dimension as the predicted value of the feature vector;

performing feature extraction processing on the first tensor to obtain a second tensor having a same dimension as the first tensor;

performing full-connection processing on the second tensor to obtain a third tensor having a same dimension as the second tensor; and

activating the third tensor to obtain the label information vector, a dimension of the label information vector being the same as a dimension of the predicted value of the feature vector;

performing signal reconstruction based on the predicted value of the feature vector and the label information vector; and

identifying a predicted value of the audio signal obtained by the signal reconstruction as a decoding result of the bitstream.

19 . A non-transitory computer-readable storage medium, having computer-executable instructions stored thereon, the computer-executable instructions, when being executed by a processor, causing the processor to implement:

obtaining a bitstream of an audio signal;

performing label extraction processing on a predicted value of a feature vector of the audio signal associated with the bitstream to obtain a label information vector, comprising:

performing convolution processing on the predicted value of the feature vector to obtain a first tensor having a same dimension as the predicted value of the feature vector;

performing feature extraction processing on the first tensor to obtain a second tensor having a same dimension as the first tensor;

performing full-connection processing on the second tensor to obtain a third tensor having a same dimension as the second tensor; and

activating the third tensor to obtain the label information vector, a dimension of the label information vector being the same as a dimension of the predicted value of the feature vector;

performing signal reconstruction based on the predicted value of the feature vector and the label information vector; and

identifying a predicted value of the audio signal obtained by the signal reconstruction as a decoding result of the bitstream.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (37)