Keyword detection without decoding
View Patent ↗Embodiments pertain to automatic speech recognition in mobile devices to establish the presence of a keyword. An audio waveform is received at a mobile device. Front-end feature extraction is performed on the audio waveform, followed by acoustic modeling, high level feature extraction, and output classification to detect the keyword. Acoustic modeling may use a neural network or a vector quantization dictionary and high level feature extraction may use pooling.
1. A computer-implemented method comprising:
receiving, by a key phrase detection system that is trained to detect a presence of an utterance of a particular key phrase in an audio waveform, a plurality of audio frame vectors that each model an audio waveform during a different period of time;
generating, by the key phrase detection system, two or more acoustic event vectors by coding respective ones of two or more audio frame vectors from the plurality of audio frame vectors without decoding using a language model, each of the two or more acoustic event vectors having a predetermined length;
generating, by the key phrase detection system, a pooled event vector by pooling all of the two or more acoustic event vectors, the pooled event vector having the predetermined length;
determining, by the key phrase detection system, whether the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors; and
outputting, by the key phrase detection system, a score that indicates a likelihood of whether or not the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors in response to determining whether the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors.
2. The method of claim 1 , further comprising:
producing, by the key phrase detection system, the plurality of audio frame vectors that each model the audio waveform during a different period of time by performing front-end feature extraction on an acoustic signal.
3. The method of claim 1 , wherein generating, by the key phrase detection system, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors without decoding using a language model comprises generating, by a neural network, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors.
4. The method of claim 1 , wherein generating, by the key phrase detection system, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors without decoding using a language model comprises generating, by a restricted Boltzmann machine, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors.
5. The method of claim 1 , wherein generating, by the key phrase detection system, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors without decoding using a language model comprises generating, by a vector quantization dictionary, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors.
6. The method of claim 1 , wherein generating, by the key phrase detection system, the pooled event vector by pooling all of the two or more acoustic event vectors, the pooled event vector having the predetermined length comprises generating the pooled event vector by pooling all of the two or more acoustic event vectors using root mean square (RMS) pooling, max-pooling, mean-pooling, or a recurrent neural network.
7. A key phrase detection system that is trained to detect a presence of an utterance of a particular key phrase in an audio waveform comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving, by the key phrase detection system, a plurality of audio frame vectors that each model an audio waveform during a different period of time;
generating, by the key phrase detection system, two or more acoustic event vectors by coding respective ones of two or more audio frame vectors from the plurality of audio frame vectors without decoding using a language model, each of the two or more acoustic event vectors having a predetermined length;
generating, by the key phrase detection system, a pooled event vector by pooling all of the two or more acoustic event vectors, the pooled event vector having the predetermined length;
determining, by the key phrase detection system, whether the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors; and
outputting, by the key phrase detection system, a score that indicates a likelihood of whether or not the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors in response to determining whether the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors.
8. The system of claim 7 , the operations further comprising:
producing, by the key phrase detection system, the plurality of audio frame vectors that each model the audio waveform during a different period of time by performing front-end feature extraction on an acoustic signal.
9. The system of claim 7 , wherein generating, by the key phrase detection system, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors without decoding using a language model comprises generating, by a neural network, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors.
10. The system of claim 7 , wherein generating, by the key phrase detection system, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors without decoding using a language model comprises generating, by a restricted Boltzmann machine, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors.
11. The system of claim 7 , wherein generating, by the key phrase detection system, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors without decoding using a language model comprises generating, by a vector quantization dictionary, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors.
12. The system of claim 7 , wherein generating, by the key phrase detection system, the pooled event vector by pooling all of the two or more acoustic event vectors, the pooled event vector having the predetermined length comprises generating the pooled event vector by pooling all of the two or more acoustic event vectors using root mean square (RMS) pooling, max-pooling, mean-pooling, or a recurrent neural network.
13. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
receiving, by a key phrase detection system that is trained to detect a presence of an utterance of a particular key phrase in an audio waveform, a plurality of audio frame vectors that each model an audio waveform during a different period of time;
generating, by the key phrase detection system, two or more acoustic event vectors by coding respective ones of two or more audio frame vectors from the plurality of audio frame vectors without decoding using a language model, each of the two or more acoustic event vectors having a predetermined length;
generating, by the key phrase detection system, a pooled event vector by pooling all of the two or more acoustic event vectors, the pooled event vector having the predetermined length;
determining, by the key phrase detection system, whether the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors; and
outputting, by the key phrase detection system, a score that indicates a likelihood of whether or not the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors in response to determining whether the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors.
14. The medium of claim 13 , the operations further comprising:
producing, by the key phrase detection system, the plurality of audio frame vectors that each model the audio waveform during a different period of time by performing front-end feature extraction on an acoustic signal.
15. The medium of claim 13 , wherein generating, by the key phrase detection system, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors without decoding using a language model comprises generating, by a neural network, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors.
16. The medium of claim 13 , wherein generating, by the key phrase detection system, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors without decoding using a language model comprises generating, by a restricted Boltzmann machine, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors.
17. The medium of claim 13 , wherein generating, by the key phrase detection system, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors without decoding using a language model comprises generating, by a vector quantization dictionary, the two or more acoustic event vectors by coding respective ones of the two or more audio frame vectors.
18. The medium of claim 13 , wherein generating, by the key phrase detection system, the pooled event vector by pooling all of the two or more acoustic event vectors, the pooled event vector having the predetermined length comprises generating the pooled event vector by pooling all of the two or more acoustic event vectors using root mean square (RMS) pooling, max-pooling, mean-pooling, or a recurrent neural network.
19. The method of claim 1 , wherein outputting, by the key phrase detection system, the score that indicates a likelihood of whether or not the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors in response to determining whether the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors comprises outputting, by the key phrase detection system, a binary score that indicates whether or not the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors.
20. The system of claim 7 , wherein outputting, by the key phrase detection system, the score that indicates a likelihood of whether or not the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors in response to determining whether the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors comprises outputting, by the key phrase detection system, a binary score that indicates whether or not the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors.
21. The medium of claim 13 , wherein outputting, by the key phrase detection system, the score that indicates a likelihood of whether or not the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors in response to determining whether the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors comprises outputting, by the key phrase detection system, a binary score that indicates whether or not the particular key phrase was present in the audio waveform during the period of time modeled by the audio frame vectors.