Granted Patent B2

US 12,217,739 · App. 17/760,609 · Granted Feb 4, 2025

Inventor: Li Fu (Beijing, CN)

Assignee: JINGDONG TECHNOLOGY HOLDING CO., LTD.

G10L15/063G10L19/022G10L25/30

Abstract

Claims (92)

1. A speech recognition method, comprising:

acquiring first linear spectrums corresponding to to-be-trained audios with different sampling rates, wherein an abscissa of the first linear spectrums is a spectrum-sequence serial number, an ordinate of the first linear spectrums is a frequency-domain serial number, and a value of a coordinate point determined by the abscissa and the ordinate is an original amplitude value corresponding to the to-be-trained audios;

determining a maximum sampling rate and other sampling rate than the maximum sampling rate in the different sampling rates;

determining a maximum frequency-domain serial number of the first linear spectrums corresponding to the other sampling rate as a first frequency-domain serial number;

determining a maximum frequency-domain serial number of the first linear spectrums corresponding to the maximum sampling rate as a second frequency-domain serial number;

setting, to zero, amplitude values corresponding to each frequency-domain serial number that is greater than the first frequency-domain serial number and less than or equal to the second frequency-domain serial number, in the first linear spectrums corresponding to the other sampling rate, to obtain second linear spectrums corresponding to the other sampling rate;

determining first speech features of the to-be-trained audios with the maximum sampling rate according to first Mel-spectrum features of the first linear spectrums corresponding to the maximum sampling rate;

determining second speech features of the to-be-trained audios with the other sampling rate according to second Mel-spectrum features of the second linear spectrums corresponding to the other sampling rate; and

training a machine learning model by using the first speech features and the second speech features,

wherein the determining second speech features of the to-be-trained audios with the other sampling rate comprises performing local normalization processing on the second Mel-spectrum features to obtain the second speech features and the local normalization processing comprises:

according to a maximum linear-spectrum frequency corresponding to the to-be-trained audios with the other sampling rate, acquiring a Mel-spectrum frequency corresponding to the maximum linear-spectrum frequency;

calculating a maximum Mel-filter serial number corresponding to the Mel-spectrum frequency;

acquiring first amplitude values corresponding to each other Mel-filter serial number in the second Mel-spectrum features, the other Mel-filter serial number being a Mel-filter serial number less than or equal to the maximum Mel-filter serial number;

respectively calculating a mean and a standard deviation of all first amplitude values as a local mean and a local standard deviation;

calculating a first difference between each of the first amplitude values and the local mean thereof;

calculating a ratio of each first difference to the local standard deviation as a normalized first amplitude value corresponding to each first amplitude value; and

replacing each first amplitude value in the second Mel-spectrum features with the normalized first amplitude value corresponding to each first amplitude value.

2. A speech recognition method, comprising:

acquiring first linear spectrums corresponding to to-be-trained audios with different sampling rates, wherein an abscissa of the first linear spectrums is a spectrum-sequence serial number, an ordinate of the first linear spectrums is a frequency-domain serial number, and a value of a coordinate point determined by the abscissa and the ordinate is an original amplitude value corresponding to the to-be-trained audios;

determining a maximum sampling rate and other sampling rate than the maximum sampling rate in the different sampling rates;

determining a maximum frequency-domain serial number of the first linear spectrums corresponding to the other sampling rate as a first frequency-domain serial number;

determining a maximum frequency-domain serial number of the first linear spectrums corresponding to the maximum sampling rate as a second frequency-domain serial number;

setting, to zero, amplitude values corresponding to each frequency-domain serial number that is greater than the first frequency-domain serial number and less than or equal to the second frequency-domain serial number, in the first linear spectrums corresponding to the other sampling rate, to obtain second linear spectrums corresponding to the other sampling rate;

determining first speech features of the to-be-trained audios with the maximum sampling rate according to first Mel-spectrum features of the first linear spectrums corresponding to the maximum sampling rate;

determining second speech features of the to-be-trained audios with the other sampling rate according to second Mel-spectrum features of the second linear spectrums corresponding to the other sampling rate; and

training a machine learning model by using the first speech features and the second speech features,

wherein the determining first speech features of the to-be-trained audios with the maximum sampling rate comprises performing global normalization processing on the first Mel-spectrum features to obtain the first speech features and the global normalization processing comprises:

acquiring second amplitude values corresponding to each Mel-filter serial number in the first Mel-spectrum features;

calculating a mean and a standard deviation of all second amplitude values as a global mean and a global standard deviation;

calculating a second difference between each of the second amplitude values and the global mean thereof;

calculating a ratio of each second difference to the global standard deviation as a normalized second amplitude value corresponding to each second amplitude value; and

replacing each second amplitude value in the first Mel-spectrum features with the normalized second amplitude value corresponding to each second amplitude value.

3. The speech recognition method according to claim 1 , wherein the acquiring first linear spectrums corresponding to to-be-trained audios with different sampling rates comprises:

respectively acquiring the first linear spectrums corresponding to the to-be-trained audios with the different sampling rates by using short-time Fourier transform.

4. The speech recognition method according to claim 1 , wherein the acquiring first linear spectrums corresponding to to-be-trained audios with different sampling rates comprises:

acquiring speech signal oscillograms of the to-be-trained audios with the different sampling rates;

respectively performing pre-emphasis processing on the speech signal oscillograms of the to-be-trained audios with the different sampling rates; and

acquiring the first linear spectrums corresponding to the to-be-trained audios with the different sampling rates according to the speech signal oscillograms after the pre-emphasis processing.

5. The speech recognition method according to claim 1 , further comprising:

respectively performing Mel-filtering transform on the first linear spectrums corresponding to the maximum sampling rate and the second linear spectrums corresponding to the other sampling rate by using a plurality of unit triangle filters, to obtain the first Mel-spectrum features and the second Mel-spectrum features.

6. The speech recognition method according to claim 1 , wherein the machine learning model comprises a deep neural network (DNN) model.

7. The speech recognition method according to claim 1 , wherein the different sampling rates comprise 16 kHZ and 8 kHZ.

8. The speech recognition method according to claim 1 , further comprising:

acquiring a to-be-recognized audio;

determining a speech feature of the to-be-recognized audio; and

inputting the speech feature of the to-be-recognized audio into the machine learning model, to obtain a speech recognition result.

9. The speech recognition method according to claim 8 , wherein the determining a speech feature of the to-be-recognized audio comprises:

determining a maximum frequency-domain serial number of a first linear spectrum of the to-be-recognized audio as a third frequency-domain serial number;

setting, to zero, amplitude values corresponding to each frequency-domain serial number that is greater than the third frequency-domain serial number and less than or equal to the second frequency-domain serial number in the first linear spectrum of the to-be-recognized audio, to obtain a second linear spectrum of the to-be-recognized audio; and

determining the speech feature of the to-be-recognized audio according to a Mel-spectrum feature of the second linear spectrum of the to-be-recognized audio.

10. A speech recognition apparatus, comprising:

a memory; and

a processor coupled to the memory, which is configured to executed the speech recognition method according to claim 1 .

11. A speech recognition apparatus, comprising:

a memory; and

a processor coupled to the memory, which is configured to executed the speech recognition method according to claim 2 .

12. A non-transitory computer-storable medium having stored thereon computer program instructions which, when executed by a processor, implement a speech recognition method, which comprises:

acquiring first linear spectrums corresponding to to-be-trained audios with different sampling rates, wherein an abscissa of the first linear spectrums is a spectrum-sequence serial number, an ordinate of the first linear spectrums is a frequency-domain serial number, and a value of a coordinate point determined by the abscissa and the ordinate is an original amplitude value corresponding to the to-be-trained audios;

determining a maximum sampling rate and other sampling rate than the maximum sampling rate in the different sampling rates;

determining a maximum frequency-domain serial number of the first linear spectrums corresponding to the other sampling rate as a first frequency-domain serial number;

determining a maximum frequency-domain serial number of the first linear spectrums corresponding to the maximum sampling rate as a second frequency-domain serial number;

setting, to zero, amplitude values corresponding to each frequency-domain serial number that is greater than the first frequency-domain serial number and less than or equal to the second frequency-domain serial number, in the first linear spectrums corresponding to the other sampling rate, to obtain second linear spectrums corresponding to the other sampling rate;

determining first speech features of the to-be-trained audios with the maximum sampling rate according to first Mel-spectrum features of the first linear spectrums corresponding to the maximum sampling rate;

determining second speech features of the to-be-trained audios with the other sampling rate according to second Mel-spectrum features of the second linear spectrums corresponding to the other sampling rate; and

training a machine learning model by using the first speech features and the second speech features,

wherein the determining second speech features of the to-be-trained audios with the other sampling rate comprises performing local normalization processing on the second Mel-spectrum features to obtain the second speech features and the local normalization processing comprises:

according to a maximum linear-spectrum frequency corresponding to the to-be-trained audios with the other sampling rate, acquiring a Mel-spectrum frequency corresponding to the maximum linear-spectrum frequency;

calculating a maximum Mel-filter serial number corresponding to the Mel-spectrum frequency;

acquiring first amplitude values corresponding to each other Mel-filter serial number in the second Mel-spectrum features, the other Mel-filter serial number being a Mel-filter serial number less than or equal to the maximum Mel-filter serial number;

respectively calculating a mean and a standard deviation of all first amplitude values as a local mean and a local standard deviation;

calculating a first difference between each of the first amplitude values and the local mean thereof;

calculating a ratio of each first difference to the local standard deviation as a normalized first amplitude value corresponding to each first amplitude value; and

replacing each first amplitude value in the second Mel-spectrum features with the normalized first amplitude value corresponding to each first amplitude value.

13. A non-transitory computer-storable medium having stored thereon computer program instructions which, when executed by a processor, implement the speech recognition method according to claim 2 .

14. The speech recognition method according to claim 2 , wherein the acquiring first linear spectrums corresponding to to-be-trained audios with different sampling rates comprises:

respectively acquiring the first linear spectrums corresponding to the to-be-trained audios with the different sampling rates by using short-time Fourier transform.

15. The speech recognition method according to claim 2 , wherein the acquiring first linear spectrums corresponding to to-be-trained audios with different sampling rates comprises:

acquiring speech signal oscillograms of the to-be-trained audios with the different sampling rates;

respectively performing pre-emphasis processing on the speech signal oscillograms of the to-be-trained audios with the different sampling rates; and

acquiring the first linear spectrums corresponding to the to-be-trained audios with the different sampling rates according to the speech signal oscillograms after the pre-emphasis processing.

16. The speech recognition method according to claim 2 , further comprising:

respectively performing Mel-filtering transform on the first linear spectrums corresponding to the maximum sampling rate and the second linear spectrums corresponding to the other sampling rate by using a plurality of unit triangle filters, to obtain the first Mel-spectrum features and the second Mel-spectrum features.

17. The speech recognition method according to claim 2 , wherein the machine learning model comprises a deep neural network (DNN) model.

18. The speech recognition method according to claim 2 , wherein the different sampling rates comprise 16 kHZ and 8 kHZ.

19. The speech recognition method according to claim 2 , further comprising:

acquiring a to-be-recognized audio;

determining a speech feature of the to-be-recognized audio; and

inputting the speech feature of the to-be-recognized audio into the machine learning model, to obtain a speech recognition result.

20. The speech recognition method according to claim 19 , wherein the determining a speech feature of the to-be-recognized audio comprises:

determining a maximum frequency-domain serial number of a first linear spectrum of the to-be-recognized audio as a third frequency-domain serial number;

setting, to zero, amplitude values corresponding to each frequency-domain serial number that is greater than the third frequency-domain serial number and less than or equal to the second frequency-domain serial number in the first linear spectrum of the to-be-recognized audio, to obtain a second linear spectrum of the to-be-recognized audio; and

determining the speech feature of the to-be-recognized audio according to a Mel-spectrum feature of the second linear spectrum of the to-be-recognized audio.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (35)