Adaptive frame skipping for speech recognition
Systems and techniques are described herein for processing audio signals. For instance, a process can include receiving a first audio frame associated with a first time frame. The process can further include generating a first time frame feature vector based on the first audio frame. The process can include determining a distance between the first time frame feature vector and a second time frame feature vector. The second time frame feature vector may be generated based on a second audio frame associated with a second time frame, where second time frame is being before the first time frame. The process can further include comparing the distance between the first time frame feature vector and the second time frame feature vector to a threshold distance. The process can include determining whether to skip processing of the first audio frame by an application based on the comparison.
1. An apparatus for processing audio signals, comprising:
at least one memory; and
at least one processor coupled to the at least one memory, wherein the at least one processor is configured to:
obtain a first audio frame associated with a first time frame;
generate a first time frame feature vector based on the first audio frame;
determine a distance between the first time frame feature vector and a second time frame feature vector, the second time frame feature vector generated based on a second audio frame associated with a second time frame;
compare the distance between the first time frame feature vector and the second time frame feature vector to a threshold distance;
determine whether to skip processing of the first audio frame by a keyword application based on the comparison;
generate, by a machine learning model, a third time frame score associated with a third time frame, the third time frame being before the second time frame, and wherein the third frame score indicates that a third time frame feature vector, associated with the third frame score, is a representation of a first portion of a keyword;
generate, by the machine learning model, a second frame score associated with the second time frame, wherein the second frame score indicates that the second time frame feature vector is a representation of a second portion of the keyword;
determine that the second frame score is greater than the third frame score; and
determine to process the first audio frame, by the machine learning model based on the determination that the second frame score is greater than the third frame score.
2. The apparatus of claim 1 , wherein the at least one processor is configured to determine not to process the first audio frame based on a determination that the distance between the first time frame feature vector and the second time frame feature vector is less than the threshold distance.
3. The apparatus of claim 1 , wherein the at least one processor is configured to determine to use a machine learning model to process the first time frame feature vector to detect the keyword based on a determination that the distance between the first time frame feature vector and the second time frame feature vector is greater than the threshold distance.
4. The apparatus of claim 1 , wherein the at least one processor is configured to store the second frame score and third frame score in the memory.
5. The apparatus of claim 1 , wherein the at least one processor is configured to determine the distance between the first time frame feature vector and the second time frame feature vector based on one of:
an absolute value of a difference between the first time frame feature vector and the second time frame feature vector;
a Euclidean distance between the first time frame feature vector and the second time frame feature vector; or
a cosine similarity between the first time frame feature vector and the second time frame feature vector.
6. The apparatus of claim 1 , wherein the at least one processor is configured to store the first time frame feature vector in the memory for comparison with a time frame feature vector of a later received audio frame.
7. The apparatus of claim 1 , wherein the first time frame feature vector indicates characteristics of the received first audio frame.
8. The apparatus of claim 1 , wherein the at least one processor is configured to determine whether to skip processing of the first audio frame by a noise suppression application.
9. The apparatus of claim 1 , wherein the second time frame is before the first time frame.
10. The apparatus of claim 1 , wherein the at least one processor is configured to determine whether to skip processing of the first audio frame by the machine learning model based on the comparison.
11. A method for processing audio signals, comprising:
receiving a first audio frame associated with a first time frame;
generating a first time frame feature vector based on the first audio frame;
determining a distance between the first time frame feature vector and a second time frame feature vector, the second time frame feature vector generated based on a second audio frame associated with a second time frame;
comparing the distance between the first time frame feature vector and the second time frame feature vector to a threshold distance;
determining whether to skip processing of the first audio frame by a keyword application based on the comparison;
generating, by a machine learning model, a third time frame score associated with a third time frame, the third time frame being before the second time frame, and wherein the third frame score indicates a probability that a third time frame feature vector, associated with the third frame score, is a presentation of a first portion of a keyword;
generating, by the machine learning model, a second frame score associated with the second time frame wherein the second frame score indicates that the second time frame feature vector is a representation of a second portion of the keyword;
determining that the third frame score is greater than the second frame score; and
determining to not process the first audio frame based on the determination that the third frame score is greater than the second frame score.
12. The method of claim 11 , further comprising determining not to process the first audio frame based on a determination that the distance between the first time frame feature vector and the second time frame feature vector is less than the threshold distance.
13. The method of claim 11 , further comprising determining to use the machine learning model to process the first time frame feature vector to detect the keyword based on a determination that the distance between the first time frame feature vector and the second time frame feature vector is greater than the threshold distance.
14. The method of claim 11 , further comprising storing the second frame score and third frame score in a memory.
15. The method of claim 11 , wherein the distance between the first time frame feature vector and the second time frame feature vector is determined based on one of:
an absolute value of a difference between the first time frame feature vector and the second time frame feature vector;
a Euclidean distance between the first time frame feature vector and the second time frame feature vector; or
a cosine similarity between the first time frame feature vector and the second time frame feature vector.
16. The method of claim 11 , further comprising storing the first time frame feature vector in a memory for comparison with a time frame feature vector of a later received audio frame.
17. The method of claim 11 , wherein the first time frame feature vector indicates characteristics of the received first audio frame.
18. The method of claim 11 , further comprising determining whether to skip processing of the first audio frame by a noise suppression application.
19. The method of claim 11 , wherein the second time frame is before the first time frame.
20. The method of claim 11 , further comprising determining whether to skip processing of the first audio frame by the machine learning model based on the comparison.
21. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the at one or more processors to:
receive a first audio frame associated with a first time frame;
generate a first time frame feature vector based on the first audio frame;
determine a distance between the first time frame feature vector and a second time frame feature vector, the second time frame feature vector generated based on a second audio frame associated with a second time frame;
compare the distance between the first time frame feature vector and the second time frame feature vector to a threshold distance;
determine whether to skip processing of the first audio frame by a keyword application based on the comparison;
generate, by a machine learning model, a third time frame score associated with a third time frame, the third time frame being before the second time frame, and wherein the third frame score indicates that a third time frame feature vector, associated with the third frame score, is a representation of a first portion of a keyword;
generate, by the machine learning model, a second frame score associated with the second time frame, wherein the second frame score indicates that the second time frame feature vector is a representation of a second portion of the keyword;
determine that the second frame score is greater than the third frame score; and
determine to process the first audio frame, my the machine learning model based on the determination that the second frame score is greater than the third frame score.
22. The non-transitory computer-readable medium of claim 21 , wherein the instructions further cause the one or more processors to determine not to process the first audio frame based on a determination that the distance between the first time frame feature vector and the second time frame feature vector is less than the threshold distance.
23. The non-transitory computer-readable medium of claim 21 , wherein the instructions further cause the one or more processors to determine to use a machine learning model to process the first time frame feature vector to detect a keyword based on a determination that the distance between the first time frame feature vector and the second time frame feature vector is greater than the threshold distance.
24. The non-transitory computer-readable medium of claim 21 , wherein the instructions further cause the one or more processors to store the second frame score and the third frame score in memory.
25. The non-transitory computer-readable medium of claim 21 , wherein the distance between the first time frame feature vector and the second time frame feature vector is determined based on one of:
an absolute value of a difference between the first time frame feature vector and the second time frame feature vector;
a Euclidean distance between the first time frame feature vector and the second time frame feature vector; or
a cosine similarity between the first time frame feature vector and the second time frame feature vector.
26. The non-transitory computer-readable medium of claim 21 , wherein the instructions further cause the one or more processors to store the first time frame feature vector in memory for comparison with a time frame feature vector of a later received audio frame.
27. The non-transitory computer-readable medium of claim 21 , wherein the first time frame feature vector indicates characteristics of the received first audio frame.
28. The non-transitory computer-readable medium of claim 21 , wherein the instructions further cause the one or more processors to determine whether to skip processing of the first audio frame by a noise suppression application.
29. The non-transitory computer-readable medium of claim 21 , wherein the second time frame is before the first time frame.
30. The non-transitory computer-readable medium of claim 21 , wherein the instructions further cause the one or more processors to determine whether to skip processing of the first audio frame by a machine learning model based on the comparison.