Multi-modal input on an electronic device
A computer-implemented input-method editor process includes receiving a request from a user for an application-independent input method editor having written and spoken input capabilities, identifying that the user is about to provide spoken input to the application-independent input method editor, and receiving a spoken input from the user. The spoken input corresponds to input to an application and is converted to text that represents the spoken input. The text is provided as input to the application.
1. A computer-implemented method when executed on data processing hardware of an electronic device causes the data processing hardware to perform operations comprising:
receiving a speech utterance from a particular user, the speech utterance indicating that the particular user intends to provide a spoken query to an application executing on the electronic device;
receiving, from the particular user, the spoken query to the application;
obtaining a context-specific language model, the context-specific language model trained on training queries associated with a category associated with the application; and
converting, using a speech recognizer and the context-specific language model, the spoken query into corresponding text for processing by the application, the speech recognizer comprising a finite state transducer (FST) encoding a Hidden Markov Model (HMM).
2. The method of claim 1 , wherein the context-specific language model defines multiple language sub-models tailored to the application, each language sub-model defining a particular rule set for use in determining a likely intent of the particular user.
3. The method of claim 1 , wherein the speech utterance and the spoken query are captured by a microphone of the electronic device.
4. The method of claim 1 , wherein the application comprises a music player application.
5. The method of claim 1 , wherein the operations further comprise:
performing intermediate processing on the received spoken query,
wherein the speech recognizer uses the intermediate processing on the received spoken query to convert the spoken query into the corresponding text.
6. The method of claim 1 , wherein the operations further comprise:
executing an application-independent input method editor configured to receive spoken in put for a plurality of applications executable by the electronic device, the plurality of applications comprising the application,
wherein receiving the speech utterance comprises receiving the speech utterance at the application-independent input method editor.
7. The method of claim 6 , wherein the application-independent input method editor has written and spoken input capabilities.
8. The method of claim 1 , wherein the operations further comprise:
determining the category associated with the application,
wherein obtaining the context-specific language model is based on the category associated with the application.
9. The method of claim 1 , wherein the operations further comprise processing, by the application, the corresponding text converted from the spoken query.
10. The method of claim 1 , wherein the electronic device comprises an audio output device.
11. A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware and storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
receiving a speech utterance from a particular user, the speech utterance indicating that the particular user intends to provide a spoken query to an application executing on the electronic device;
receiving, from the particular user, the spoken query to the application;
obtaining a context-specific language model, the context-specific language model trained on training queries associated with a category associated with the application; and
converting, using a speech recognizer and the context-specific language model, the spoken query into corresponding text for processing by the application, the speech recognizer comprising a finite state transducer (FST) encoding a Hidden Markov Model (HMM).
12. The system of claim 11 , wherein the context-specific language model defines multiple language sub-models tailored to the application, each language sub-model defining a particular rule set for use in determining a likely intent of the particular user.
13. The system of claim 11 , wherein the speech utterance and the spoken query are captured by a microphone of the electronic device.
14. The system of claim 11 , wherein the application comprises a music player application.
15. The system of claim 11 , wherein the operations further comprise:
performing intermediate processing on the received spoken query,
wherein the speech recognizer uses the intermediate processing on the received spoken query to convert the spoken query into the corresponding text.
16. The system of claim 11 , wherein the operations further comprise:
executing an application-independent input method editor configured to receive spoken in put for a plurality of applications executable by the electronic device, the plurality of applications comprising the application,
wherein receiving the speech utterance comprises receiving the speech utterance at the application-independent input method editor.
17. The system of claim 16 , wherein the application-independent input method editor has written and spoken input capabilities.
18. The system of claim 11 , wherein the operations further comprise:
determining a category associated with the application,
wherein obtaining the context-specific language model is based on the category associated with the application.
19. The system of claim 11 , wherein the operations further comprise processing, by the application, the corresponding text converted from the spoken query n.
20. The system of claim 11 , wherein the electronic device comprises an audio output device.