User interface for realtime language translation
A language translation application on a user device includes a user interface that provides relevant textual and graphical feedback mechanisms associated with various states of voice input and translated speech.
1. A computer-implemented method, comprising:
displaying a graphical user interface for language translation on a user device that includes (i) one or more automated speech recognizers that recognize speech in a source language only when in a first mode and to recognize speech in a target language only when in a second mode, (ii) a microphone that receives audio, and (iii) a speaker that outputs audio, the graphical user interface comprising a source language visual indicator and a target language visual indicator;
in the first mode in which the one or more automated speech recognizers recognize speech in the source language only, highlighting the source language visual indicator and displaying an input visual indicator on the graphical user interface to provide a visual indication that the user device receives audio input in the source language only; and
in response to an endpointer on the user device automatically determining that the input in the source language only has completed, and without requiring the user to manually switch between the first mode and the second mode, automatically activating the second mode in which the one or more automated speech recognizers recognize speech in the target language only, removing highlighting from the source language visual indicator, highlighting the target language visual indicator, and replacing the input visual indicator with an output visual indicator on the graphical user interface to provide a visual indication that the user device provides audio output in the target language.
2. The method of claim 1 , comprising:
animating, in response to a request to initiate listening for an utterance in the source language, the input visual indicator on the graphical user interface.
3. The method of claim 1 , wherein the visual indication that the user device receives audio input in the source language is provided by highlighting the input visual indicator on the graphical user interface while highlighting the source language visual indicator.
4. The method of claim 1 , wherein the visual indication that the user device provides audio output in the target language is provided by highlighting the output visual indicator on the graphical user interface while highlighting the target language visual indicator.
5. The method of claim 1 , comprising:
animating, in response to completing preparations to provide output in the target language, the output visual indicator on the graphical user interface.
6. The method of claim 1 , comprising:
in a third mode in which the user device translates input received in the source language into the target language, displaying a transcription of the input received in the source language on a first portion of the graphical user interface, and displaying a translation into the target of the transcription of the input received in the source language on a second portion of the graphical user interface.
7. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
displaying a graphical user interface for language translation on a user device that includes (i) one or more automated speech recognizers that recognize speech in a source language only when in a first mode and to recognize speech in a target language only when in a second mode, (ii) a microphone that receives audio, and (iii) a speaker that outputs audio, the graphical user interface comprising a source language visual indicator and a target language visual indicator;
in the first mode in which the one or more automated speech recognizers recognize speech in the source language only, highlighting the source language visual indicator and displaying an input visual indicator on the graphical user interface to provide a visual indication that the user device receives audio input in the source language only; and
in response to an endpointer on the user device automatically determining that the input in the source language only has completed, and without requiring the user to manually switch between the first mode and the second mode, automatically activating the second mode in which the one or more automated speech recognizers recognize speech in the target language only, removing highlighting from the source language visual indicator, highlighting the target language visual indicator, and replacing the input visual indicator with an output visual indicator on the graphical user interface to provide a visual indication that the user device provides audio output in the target language.
8. The system of claim 7 , the operations comprising:
animating, in response to a request to initiate listening for an utterance in the source language, the input visual indicator on the graphical user interface.
9. The system of claim 7 , wherein the visual indication that the user device receives audio input in the source language is provided by highlighting the input visual indicator on the graphical user interface while highlighting the source language visual indicator.
10. The system of claim 7 , wherein the visual indication that the user device provides audio output in the target language is provided by highlighting the output visual indicator on the graphical user interface while highlighting the target language visual indicator.
11. The system of claim 7 , the operations comprising:
animating, in response to completing preparations to provide output in the target language, the output visual indicator on the graphical user interface.
12. The system of claim 7 , the operations comprising:
in a third mode in which the user device translates input received in the source language into the target language, displaying a transcription of the input received in the source language on a first portion of the graphical user interface, and displaying a translation into the target of the transcription of the input received in the source language on a second portion of the graphical user interface.
13. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
displaying a graphical user interface for language translation on a user device that includes (i) one or more automated speech recognizers that recognize speech in a source language only when in a first mode and to recognize speech in a target language only when in a second mode, (ii) a microphone that receives audio, and (iii) a speaker that outputs audio, the graphical user interface comprising a source language visual indicator and a target language visual indicator;
in the first mode in which the one or more automated speech recognizers recognize speech in the source language only, highlighting the source language visual indicator and displaying an input visual indicator on the graphical user interface to provide a visual indication that the user device receives audio input in the source language only; and
in response to an endpointer on the user device automatically determining that the input in the source language only has completed, and without requiring the user to manually switch between the first mode and the second mode, automatically activating the second mode in which the one or more automated speech recognizers recognize speech in the target language only, removing highlighting from the source language visual indicator, highlighting the target language visual indicator, and replacing the input visual indicator with an output visual indicator on the graphical user interface to provide a visual indication that the user device provides audio output in the target language.
14. The computer-readable medium of claim 13 , the operations comprising:
animating, in response to a request to initiate listening for an utterance in the source language, the input visual indicator on the graphical user interface.
15. The computer-readable medium of claim 13 , wherein the visual indication that the user device receives audio input in the source language is provided by highlighting the input visual indicator on the graphical user interface while highlighting the source language visual indicator.
16. The computer-readable medium of claim 13 , wherein the visual indication that the user device provides audio output in the target language is provided by highlighting the output visual indicator on the graphical user interface while highlighting the target language visual indicator.
17. The computer-readable medium of claim 13 , the operations comprising:
animating, in response to completing preparations to provide output in the target language, the output visual indicator on the graphical user interface.
18. The computer-readable medium of claim 13 , the operations comprising in a third mode in which the user device translates input received in the source language into the target language, displaying a transcription of the input received in the source language on a first portion of the graphical user interface, and displaying a translation into the target of the transcription of the input received in the source language on a second portion of the graphical user interface.