IP Library › Granted Patent US 10,496,759
Granted Patent B2
US 10,496,759 · App. 15/973,722 · Granted Dec 3, 2019

User interface for realtime language translation

Inventors: Alexander Jay Cuthbert (Oakland, CA); Sunny Goyal (Mountain View, CA); Matthew Morton Gaba (San Francisco, CA); Joshua J. Estelle (San Francisco, CA); Masakazu Seno (Cupertino, CA)
Assignee: Google LLC
G06F17/289G10L25/48G10L15/26
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,496,759
App. No.
15/973,722
Granted
Dec 3, 2019
Kind
B2
Abstract

A language translation application on a user device includes a user interface that provides relevant textual and graphical feedback mechanisms associated with various states of voice input and translated speech.

Claims (41)

1. A computer-implemented method comprising:

providing, by a mobile device, a user interface for a real-time translation application that receives spoken inputs in a first language outputs and that generates synthesized speech representations of translations of the spoken inputs in a second language, wherein the user interface includes:

a first language region for displaying transcriptions of the spoken inputs in the first language,

a second language region for displaying translations of the spoken inputs in the second language, and

an indicator region for displaying an icon that, when an outline of the icon is animated, indicates that a speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language, wherein the icon comprises a speaker icon whose outline is animated in a spinning pattern only for a period of time before the speaker is prepared to output synthesized speech representations in the second language;

determining, by the mobile device, a state of the real-time translation application; and

in response to determining the state of the real-time translation application, updating, by the mobile device, the user interface to animate the outline of the icon to indicate that the speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language.

2. The method of claim 1 , comprising:

receiving a particular spoken input; and

updating (i) the first language region of the user interface to display a transcription of the particular spoken input in a first language, and (ii) the second language region of the user interface to display a translation of the particular spoken input in a second language.

3. The method of claim 1 , wherein the mobile device includes an automated speech recognizer, and an automated language translator.

4. The method of claim 3 , wherein the automated speech recognizer comprises a continuous speech recognizer.

5. The method of claim 1 , wherein the icon is user-selectable to manually activate or deactivate the speaker or a microphone.

6. The method of claim 1 , wherein the first language region is further for displaying translations of spoken inputs in the second language, in the first language.

7. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

providing, by a mobile device, a user interface for a real-time translation application that receives spoken inputs in a first language outputs and that generates synthesized speech representations of translations of the spoken inputs in a second language, wherein the user interface includes:

a first language region for displaying transcriptions of the spoken inputs in the first language,

a second language region for displaying translations of the spoken inputs in the second language, and

an indicator region for displaying an icon that, when an outline of the icon is animated, indicates that a speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language, wherein the icon comprises a speaker icon whose outline is animated in a spinning pattern only for a period of time before the speaker is prepared to output synthesized speech representations in the second language;

determining, by the mobile device, a state of the real-time translation application; and

in response to determining the state of the real-time translation application, updating, by the mobile device, the user interface to animate the outline of the icon to indicate that the speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language.

8. The system of claim 7 , wherein the operations comprise:

receiving a particular spoken input; and

updating (i) the first language region of the user interface to display a transcription of the particular spoken input in a first language, and (ii) the second language region of the user interface to display a translation of the particular spoken input in a second language.

9. The system of claim 7 , wherein the mobile device includes an automated speech recognizer, and an automated language translator.

10. The system of claim 9 , wherein the automated speech recognizer comprises a continuous speech recognizer.

11. The system of claim 7 , wherein the icon is user-selectable to manually activate or deactivate the speaker or a microphone.

12. The system of claim 7 , wherein the first language region is further for displaying translations of spoken inputs in the second language, in the first language.

13. A computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

providing, by a mobile device, a user interface for a real-time translation application that receives spoken inputs in a first language outputs and that generates synthesized speech representations of translations of the spoken inputs in a second language, wherein the user interface includes:

a first language region for displaying transcriptions of the spoken inputs in the first language,

a second language region for displaying translations of the spoken inputs in the second language, and

an indicator region for displaying an icon that, when an outline of the icon is animated, indicates that a speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language, wherein the icon comprises a speaker icon whose outline is animated in a spinning pattern only for a period of time before the speaker is prepared to output synthesized speech representations in the second language;

determining, by the mobile device, a state of the real-time translation application; and

in response to determining the state of the real-time translation application, updating, by the mobile device, the user interface to animate the outline of the icon to indicate that the speaker of the mobile device is preparing to output synthesized speech representations in the second language but is not yet outputting synthesized speech representations in the second language.

14. The device of claim 13 , wherein the operations comprise:

receiving a particular spoken input; and

updating (i) the first language region of the user interface to display a transcription of the particular spoken input in a first language, and (ii) the second language region of the user interface to display a translation of the particular spoken input in a second language.

15. The device of claim 13 , wherein the mobile device includes an automated speech recognizer, and an automated language translator.

16. The device of claim 15 , wherein the automated speech recognizer comprises a continuous speech recognizer.

17. The device of claim 13 , wherein the icon is user-selectable to manually activate or deactivate the speaker or a microphone.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 9, 2018
From: CUTHBERT, ALEXANDER J.; GOYAL, SUNNY; GABA, MATTHEW; ESTELLE, JOSHUA J.; SENO, MASAKAZU
To: GOOGLE INC.
Reel/Frame 045753/0381 →
ENTITY CONVERSION Recorded May 9, 2018
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 046110/0259 →
Continuity (3)
Continuation 15460360 · Mar 16, 2017
Continuation 14075018 · Nov 8, 2013
Related Publication 20180276203A1 · Sep 27, 2018
Cited By (1)
US 12,462,110