Speech recognition method and apparatus, electronic device, and computer-readable storage medium

1 . An electronic device, comprising: one or more processors, a memory, a communication interface, and a communication bus, wherein the one or more processors, the memory, and the communication interface communicate with each other via the communication bus,

the memory is configured to store at least one executable instruction, and the executable instruction enables the one or more processors to cause the electronic device to perform operations corresponding to perform operations comprising:

acquiring a to-be-recognized speech fragment;

encoding acquired speech data while simultaneously acquiring the to-be-recognized speech fragment, to obtain an encoding result of the to-be-recognized speech fragment; and

decoding the encoding result of the to-be-recognized speech fragment to obtain text data corresponding to the to-be-recognized speech fragment;

wherein encoding the acquired speech data further comprises:

performing feature extraction on an acquired current speech frame to obtain a first feature matrix;

downsampling the first feature matrix to obtain a second feature matrix; and

calculating an encoding result of a current speech frame based on the second feature matrix, wherein the encoding result of the to-be-recognized speech fragment is obtained based on encoding results of a plurality of speech frames comprised in the to-be-recognized speech fragment;

wherein calculating the encoding result of the current speech frame based on the second feature matrix comprises:

multiplying the second feature matrix respectively with a weight matrix W K , a weight matrix W Q , and a weight matrix W V to obtain a third feature matrix K, a third feature matrix Q, and a third feature matrix V;

performing a full connection layer operation on the third feature matrix K to obtain a fourth feature matrix K corresponding to the current speech frame;

performing a full connection layer operation on the third feature matrix V to obtain a fourth feature matrix V corresponding to the current speech frame;

determining the encoding result of the current speech frame based on the third feature matrix Q, the fourth feature matrix K and the fourth feature matrix V that correspond to the current speech frame, and the fourth feature matrix K and the fourth feature matrix V that correspond to a historical speech frame.

2 . The electronic device of claim 1 , wherein the operations further comprise:

loading, from a first cache, the fourth feature matrix K corresponding to the historical speech frame, wherein the first cache caches the fourth feature matrix K corresponding to at least one first speech frame; and

loading, from a second cache, the fourth feature matrix V corresponding to the historical speech frame, wherein the second cache caches a fourth feature matrix V corresponding to at least one second speech frame.

3 . The electronic device of claim 2 , wherein the operations further comprise:

caching the fourth feature matrix K that corresponds to the current speech frame to the first cache; and

caching the fourth feature matrix V that corresponds to the current speech frame to the second cache.

4 . The electronic device of claim 1 , wherein the operation of determining the encoding result of the current speech frame based on the third feature matrix Q, the fourth feature matrix K and the fourth feature matrix V that correspond to the current speech frame, and the fourth feature matrix K and the fourth feature matrix V that correspond to the historical speech frame comprises:

splicing the fourth feature matrix K corresponding to the current speech frame and the fourth feature matrix K corresponding to the historical speech frame to obtain a fifth feature matrix K;

splicing the fourth feature matrix V corresponding to the current speech frame and the fourth feature matrix V corresponding to the historical speech frame to obtain a fifth feature matrix V; and

determining the encoding result of the current speech frame based on the third feature matrix Q, the fifth feature matrix K, and the fifth feature matrix V.

5 . The electronic device of claim 4 , wherein the operations further comprise:

loading, from a first cache, the fourth feature matrix K corresponding to the historical speech frame, wherein the first cache caches the fourth feature matrix K corresponding to at least one first speech frame; and

loading, from a second cache, the fourth feature matrix V corresponding to the historical speech frame, wherein the second cache caches a fourth feature matrix V corresponding to at least one second speech frame.

6 . The electronic device of claim 5 , wherein the operations further comprise:

caching the fourth feature matrix K that corresponds to the current speech frame to the first cache; and

caching the fourth feature matrix V that corresponds to the current speech frame to the second cache.

7 . The electronic device of claim 4 , wherein the operation of determining the encoding result of the current speech frame based on the third feature matrix Q, the fifth feature matrix K, and the fifth feature matrix V comprise:

calculating a product of the third feature matrix Q and a transposed matrix of the fifth feature matrix K to obtain a sixth feature matrix;

performing normalization on the sixth feature matrix to obtain a seventh feature matrix;

processing the seventh feature matrix using an activation function to obtain an eighth feature matrix;

calculating a product of the eighth feature matrix and the fifth feature matrix V to obtain a ninth feature matrix; and

determining the encoding result of the current speech frame based on the ninth feature matrix.

8 . The electronic device of claim 7 , wherein the operations further comprise:

loading, from a first cache, the fourth feature matrix K corresponding to the historical speech frame, wherein the first cache caches the fourth feature matrix K corresponding to at least one first speech frame; and

loading, from a second cache, the fourth feature matrix V corresponding to the historical speech frame, wherein the second cache caches a fourth feature matrix V corresponding to at least one second speech frame.

9 . The electronic device of claim 8 , wherein the operations further comprise:

caching the fourth feature matrix K that corresponds to the current speech frame to the first cache; and

caching the fourth feature matrix V that corresponds to the current speech frame to the second cache.

10 . A speech recognition method, comprising:

acquiring a to-be-recognized speech fragment;

encoding acquired speech data while simultaneously acquiring the to-be-recognized speech fragment, to obtain an encoding result of the to-be-recognized speech fragment; and

decoding the encoding result of the to-be-recognized speech fragment to obtain text data corresponding to the to-be-recognized speech fragment;

wherein encoding the acquired speech data further comprises:

performing feature extraction on an acquired current speech frame to obtain a first feature matrix;

downsampling the first feature matrix to obtain a second feature matrix; and

calculating an encoding result of a current speech frame based on the second feature matrix, wherein the encoding result of the to-be-recognized speech fragment is obtained based on encoding results of a plurality of speech frames comprised in the to-be-recognized speech fragment;

wherein calculating the encoding result of the current speech frame based on the second feature matrix comprise:

multiplying the second feature matrix respectively with a weight matrix W K , a weight matrix W Q , and a weight matrix W V to obtain a third feature matrix K, a third feature matrix Q, and a third feature matrix V;

performing a full connection layer operation on the third feature matrix K to obtain a fourth feature matrix K corresponding to the current speech frame;

performing a full connection layer operation on the third feature matrix V to obtain a fourth feature matrix V corresponding to the current speech frame;

determining the encoding result of the current speech frame based on the third feature matrix Q, the fourth feature matrix K and the fourth feature matrix V that correspond to the current speech frame, and the fourth feature matrix K and the fourth feature matrix V that correspond to a historical speech frame.

11 . The method of claim 10 , further comprising:

loading, from a first cache, the fourth feature matrix K corresponding to the historical speech frame, wherein the first cache caches the fourth feature matrix K corresponding to at least one first speech frame; and

loading, from a second cache, the fourth feature matrix V corresponding to the historical speech frame, wherein the second cache caches a fourth feature matrix V corresponding to at least one second speech frame.

12 . The method of claim 11 , further comprising:

caching the fourth feature matrix K that corresponds to the current speech frame to the first cache; and

caching the fourth feature matrix V that corresponds to the current speech frame to the second cache.

13 . The method of claim 10 , wherein determining the encoding result of the current speech frame based on the third feature matrix Q, the fourth feature matrix K and the fourth feature matrix V that correspond to the current speech frame, and the fourth feature matrix K and the fourth feature matrix V that correspond to the historical speech frame comprises:

splicing the fourth feature matrix K corresponding to the current speech frame and the fourth feature matrix K corresponding to the historical speech frame to obtain a fifth feature matrix K;

splicing the fourth feature matrix V corresponding to the current speech frame and the fourth feature matrix V corresponding to the historical speech frame to obtain a fifth feature matrix V; and

determining the encoding result of the current speech frame based on the third feature matrix Q, the fifth feature matrix K, and the fifth feature matrix V.

14 . The method of claim 13 , wherein determining the encoding result of the current speech frame based on the third feature matrix Q, the fifth feature matrix K, and the fifth feature matrix V comprises:

calculating a product of the third feature matrix Q and a transposed matrix of the fifth feature matrix K to obtain a sixth feature matrix;

performing normalization on the sixth feature matrix to obtain a seventh feature matrix;

processing the seventh feature matrix using an activation function to obtain an eighth feature matrix;

calculating a product of the eighth feature matrix and the fifth feature matrix V to obtain a ninth feature matrix; and

determining the encoding result of the current speech frame based on the ninth feature matrix.

15 . A speech interaction method, applied to a smart speaker, the method comprising:

acquiring a speech interaction instruction issued by a user;

encoding acquired speech data while simultaneously acquiring the speech interaction instruction, to obtain an encoding result of the speech interaction instruction;

decoding the encoding result of the speech interaction instruction to obtain text data corresponding to the speech interaction instruction; and

issuing speech feedback information or performing an interactive action based on the text data corresponding to the speech interaction instruction;

wherein encoding the acquired speech data further comprises:

performing feature extraction on an acquired current speech frame to obtain a first feature matrix;

downsampling the first feature matrix to obtain a second feature matrix; and

calculating an encoding result of a current speech frame based on the second feature matrix, wherein the encoding result of the speech data is obtained based on encoding results of a plurality of speech frames comprised in the speech data;

wherein calculating the encoding result of the current speech frame based on the second feature matrix comprise:

multiplying the second feature matrix respectively with a weight matrix W K , a weight matrix W Q , and a weight matrix W V to obtain a third feature matrix K, a third feature matrix Q, and a third feature matrix V;

performing a full connection layer operation on the third feature matrix K to obtain a fourth feature matrix K corresponding to the current speech frame;

performing a full connection layer operation on the third feature matrix V to obtain a fourth feature matrix V corresponding to the current speech frame;

determining the encoding result of the current speech frame based on the third feature matrix Q, the fourth feature matrix K and the fourth feature matrix V that correspond to the current speech frame, and the fourth feature matrix K and the fourth feature matrix V that correspond to a historical speech frame.

16 . The method of claim 15 , further comprising:

loading, from a first cache, the fourth feature matrix K corresponding to the historical speech frame, wherein the first cache caches the fourth feature matrix K corresponding to at least one first speech frame; and

loading, from a second cache, the fourth feature matrix V corresponding to the historical speech frame, wherein the second cache caches a fourth feature matrix V corresponding to at least one second speech frame.

17 . The method of claim 16 , further comprising:

caching the fourth feature matrix K that corresponds to the current speech frame to the first cache; and

caching the fourth feature matrix V that corresponds to the current speech frame to the second cache.

18 . The method of claim 15 , wherein determining the encoding result of the current speech frame based on the third feature matrix Q, the fourth feature matrix K and the fourth feature matrix V that correspond to the current speech frame, and the fourth feature matrix K and the fourth feature matrix V that correspond to the historical speech frame comprises:

splicing the fourth feature matrix K corresponding to the current speech frame and the fourth feature matrix K corresponding to the historical speech frame to obtain a fifth feature matrix K;

splicing the fourth feature matrix V corresponding to the current speech frame and the fourth feature matrix V corresponding to the historical speech frame to obtain a fifth feature matrix V; and

determining the encoding result of the current speech frame based on the third feature matrix Q, the fifth feature matrix K, and the fifth feature matrix V.

19 . The method of claim 18 , wherein determining the encoding result of the current speech frame based on the third feature matrix Q, the fifth feature matrix K, and the fifth feature matrix V comprises:

calculating a product of the third feature matrix Q and a transposed matrix of the fifth feature matrix K to obtain a sixth feature matrix;

performing normalization on the sixth feature matrix to obtain a seventh feature matrix;

processing the seventh feature matrix using an activation function to obtain an eighth feature matrix;

calculating a product of the eighth feature matrix and the fifth feature matrix V to obtain a ninth feature matrix; and

determining the encoding result of the current speech frame based on the ninth feature matrix.

20 . The method of claim 18 , further comprising:

loading, from a first cache, the fourth feature matrix K corresponding to the historical speech frame, wherein the first cache caches the fourth feature matrix K corresponding to at least one first speech frame; and

loading, from a second cache, the fourth feature matrix V corresponding to the historical speech frame, wherein the second cache caches a fourth feature matrix V corresponding to at least one second speech frame.