Score trend analysis for reduced latency automatic speech recognition
Techniques are provided for reducing the latency of automatic speech recognition using hypothesis score trend analysis. A methodology implementing the techniques according to an embodiment includes generating complete-phrase hypotheses and partial-phrase hypotheses, along with associated likelihood scores, based on a segment of speech. The method also includes selecting the complete-phrase hypothesis associated with the highest of the complete-phrase hypotheses likelihood scores, and selecting the partial-phrase hypothesis associated with the highest of the partial-phrase hypotheses likelihood scores. The method further includes calculating a relative likelihood score based on a ratio of the likelihood score associated with the selected complete-phrase hypothesis to the likelihood score associated with the selected partial-phrase hypothesis. The method further includes calculating a trend of the relative likelihood score as a function of time and identifying an endpoint of the speech based on a determination that the trend does not decrease over a selected time period.
1. A processor-implemented method for reduced latency automatic speech recognition (ASR), the method comprising:
generating, by a processor-based system, one or more complete-phrase hypotheses from a segment of speech, each complete-phrase hypothesis associated with a likelihood score;
generating, by the processor-based system, one or more partial-phrase hypotheses from the segment of speech, each partial-phrase hypothesis associated with a likelihood score;
selecting, by the processor-based system, one of the complete-phrase hypotheses associated with a highest of the complete-phrase hypotheses likelihood scores;
selecting, by the processor-based system, one of the partial-phrase hypotheses associated with a highest of the partial-phrase hypotheses likelihood scores;
calculating, by the processor-based system, a relative likelihood score based on a ratio of the likelihood score associated with the selected complete-phrase hypothesis to the likelihood score associated with the selected partial-phrase hypothesis;
calculating, by the processor-based system, a trend of the relative likelihood score as a function of time; and
identifying, by the processor-based system, an endpoint of the speech based on a determination that the trend does not decrease over a selected time period.
2. The method of claim 1 , further comprising providing the selected complete-phrase hypothesis as an ASR result in response to identifying the endpoint of the speech.
3. The method of claim 2 , further comprising generating, by a language interpreter circuit, a response to the ASR result, the response including execution of a requested command associated with the ASR result.
4. The method of claim 1 , wherein the trend calculation further comprises calculating differences between time samples of the relative likelihood score and applying a low pass filter to a time sequence of the calculated differences.
5. The method of claim 1 , wherein the selected time period is chosen to provide a desired ASR error rate.
6. The method of claim 1 , wherein the complete-phrase hypotheses and the partial-phrase hypotheses are generated by a weighted finite state transducer (WFST) decoder.
7. The method of claim 6 , further comprising generating, by a neural network, acoustic scores of phonetic units of the speech, for use by the WFST decoder.
8. A system for reduced latency automatic speech recognition (ASR), the system comprising:
a weighted finite state transducer (WFST) decoder circuit to generate one or more complete-phrase hypotheses from a segment of speech, each complete-phrase hypothesis associated with a likelihood score;
the WFST decoder circuit further to generate one or more partial-phrase hypotheses from the segment of speech, each partial-phrase hypothesis associated with a likelihood score;
a relative likelihood score calculation circuit to select one of the complete-phrase hypotheses associated with a highest of the complete-phrase hypotheses likelihood scores, and to select one of the partial-phrase hypotheses associated with a highest of the partial-phrase hypotheses likelihood scores;
the relative likelihood score calculation circuit further to calculate a relative likelihood score based on a ratio of the likelihood score associated with the selected complete-phrase hypothesis to the likelihood score associated with the selected partial-phrase hypothesis;
a score trend calculation circuit to calculate a trend of the relative likelihood score as a function of time; and
an endpoint decision circuit to identify an endpoint of the speech based on a determination that the trend does not decrease over a selected time period.
9. The system of claim 8 , wherein the WFST decoder circuit is further to provide the selected complete-phrase hypothesis as an ASR result in response to the identification of the endpoint of the speech.
10. The system of claim 9 , further comprising a language interpreter circuit to generate a response to the ASR result, the response including execution of a requested command associated with the ASR result.
11. The system of claim 8 , wherein the trend calculation further comprises calculating differences between time samples of the relative likelihood score and applying a low pass filter to a time sequence of the calculated differences.
12. The system of claim 8 , wherein the selected time period is chosen to provide a desired ASR error rate.
13. The system of claim 8 , further comprising a neural network to generate acoustic scores of phonetic units of the speech for use by the WFST decoder.
14. At least one non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, result in the following operations for reduced latency automatic speech recognition (ASR), the operations comprising:
generating one or more complete-phrase hypotheses from a segment of speech, each complete-phrase hypothesis associated with a likelihood score;
generating one or more partial-phrase hypotheses from the segment of speech, each partial-phrase hypothesis associated with a likelihood score;
selecting one of the complete-phrase hypotheses associated with a highest of the complete-phrase hypotheses likelihood scores;
selecting one of the partial-phrase hypotheses associated with a highest of the partial-phrase hypotheses likelihood scores;
calculating a relative likelihood score based on a ratio of the likelihood score associated with the selected complete-phrase hypothesis to the likelihood score associated with the selected partial-phrase hypothesis;
calculating a trend of the relative likelihood score as a function of time; and
identifying an endpoint of the speech based on a determination that the trend does not decrease over a selected time period.
15. The computer readable storage medium of claim 14 , further comprising the operation of providing the selected complete-phrase hypothesis as an ASR result in response to identifying the endpoint of the speech.
16. The computer readable storage medium of claim 15 , further comprising the operation of generating, by a language interpreter circuit, a response to the ASR result, the response including execution of a requested command associated with the ASR result.
17. The computer readable storage medium of claim 14 , wherein the trend calculation further comprises the operations of calculating differences between time samples of the relative likelihood score and applying a low pass filter to a time sequence of the calculated differences.
18. The computer readable storage medium of claim 14 , wherein the selected time period is chosen to provide a desired ASR error rate.
19. The computer readable storage medium of claim 14 , wherein the complete-phrase hypotheses and the partial-phrase hypotheses are generated by a weighted finite state transducer (WFST) decoder.
20. The method of claim 19 , further comprising the operation of generating, by a neural network, acoustic scores of phonetic units of the speech, for use by the WFST decoder.