IP Library Granted Patent US 11,334,717
Granted Patent B2
US 11,334,717 · App. 16/683,895 · Granted May 17, 2022

Touch keyboard using a trained model

Inventors: Shumin Zhai (Los Altos, CA); Ciprian Ioan Chelba (Palo Alto, CA)
Assignee: Google LLC
G06F40/284G06F3/0237G06F3/04883G06F3/04886G06F40/242G06F40/263H04L51/02H04L51/063
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,334,717
App. No.
16/683,895
Granted
May 17, 2022
Kind
B2
Abstract

A computing device outputs for display at a presence-sensitive display, a graphical keyboard comprising a plurality of keys, receives an indication of at least one gesture to select a group of keys of the plurality of keys, and determines at least one characteristic associated with the at least one gesture to select the group of keys of the plurality of keys. The computing device modifies a spatial model based at least in part on the at least one characteristic and determines a candidate word based at least in part on data provided by the spatial model and a language model, wherein the spatial model provides data based at least in part on the indication of the at least one gesture and wherein the language model provides data based at least in part on a lexicon. The computing device outputs for display at the presence-sensitive display, the candidate word.

Claims (53)

1. A method comprising:

outputting, by a computing device, for display at a presence-sensitive display, a graphical user interface including a plurality of keys of a graphical keyboard;

executing a model trained using machine-learning to determine text from both sliding and single-touch gestures detected by the presence-sensitive display, the model including a spatial model and a language model, the spatial model including a distribution of touch points that corresponds to at least one respective key of the graphical keyboard;

responsive to receiving first gesture data from the presence-sensitive display, inputting the first gesture data into the model, the first gesture data indicative of one or more first letters selected by a first gesture type that comprises a sliding gesture or a single-touch gesture to select one or more first letter keys of the graphical keyboard, the sliding gesture comprising a continuous gesture over different locations of the graphical user interface, wherein alignment points along the continuous gesture correspond to different letter keys of the graphical keyboard;

responsive to receiving second gesture data from the presence-sensitive display, inputting the second gesture data into the model, the second gesture data indicative of one or more second letters selected by a second gesture type that comprises the sliding gesture or the single-touch gesture to select one or more second letter keys of the graphical keyboard, the second gesture type different than the first gesture type;

determining, based at least in part on both physical cost values from the spatial model and lexical cost values from the language model, one or more words predicted by the model based on the first and second letters, the physical cost values representing first likelihoods that the first gesture data and the second gesture data correspond to the first letters and the second letters, respectively, and modified by the lexical cost values from the language model, the lexical cost values representing second likelihoods that the first letters and the second letters are included in a word in a lexicon of the language model; and

outputting, by the computing device, for display in a text entry area of the graphical user interface, the one or more words predicted by the model.

2. The method of claim 1 , wherein the first gesture type is the sliding gesture and the second gesture type is the single-touch gesture.

3. The method of claim 1 , wherein the model is trained using statistical machine-learning.

4. The method of claim 1 , further comprising:

outputting, by the computing device, for display as a single prediction in a suggestion region of the graphical keyboard, the one or more words predicted by the model.

5. The method of claim 4 , further comprising:

while outputting the one or more words predicted by the model for display in the suggestion region of the graphical keyboard, outputting, for display, additional words predicted by the model as additional predictions in the suggestion region.

6. The method of claim 4 , wherein the suggestion region comprises up to three predictions including the one or more words as a first of the three predictions.

7. The method of claim 1 , wherein the model is trained using machine-learning to determine the one or more words by inferring spatial and linguistic characteristics of the first gesture data relative to spatial and linguistic characteristics of the second gesture data.

8. The method of claim 1 , the method further comprises:

determining a speed at which the first gesture data and the second gesture data are performed; and

modifying, based on the speed of the first gesture data and the second gesture data, one or more physical cost values of the spatial model,

wherein the model is trained using machine-learning to determine the one or more words by inferring the spatial characteristics of the first gesture data and the second gesture data relative to the one or more physical cost values of the spatial model.

9. A computing device comprising:

a presence-sensitive display; and

one or more processors configured to:

output, for display at the presence-sensitive display, a graphical user interface including a plurality of keys of a graphical keyboard;

execute a model trained using machine-learning to determine text from both sliding gestures and single-touch gestures detected by the presence-sensitive display, the model including a spatial model and a language model, the spatial model including a distribution of touch points that corresponds to at least one respective key of the graphical keyboard;

responsive to receiving first gesture data from the presence-sensitive display, input the first gesture data into the model, the first gesture data indicative of one or more first letters selected by a first gesture type that comprises a sliding gesture or a single-touch gesture to select one or more first letter keys of the graphical keyboard, the sliding gesture comprising a continuous gesture over different locations of the graphical user interface, wherein alignment points along the continuous gesture correspond to different letter keys of the graphical keyboard;

responsive to receiving second gesture data from the presence-sensitive display, input the second gesture data into the model, the second gesture data indicative of one or more second letters selected by a second gesture type that comprises the sliding gesture or the single-touch gesture to select one or more second letter keys of the graphical keyboard, the second gesture type different than the first gesture type;

determine, based at least in part on both physical cost values from the spatial model and lexical cost values from the language model, one or more words predicted by the model based on the first and second letters, the physical cost values representing first likelihoods that the first gesture data and the second gesture data correspond to the first letters and the second letters, respectively, and modified by the lexical cost values from the language model, the lexical cost values representing second likelihoods that the first letters and the second letters are included in a word in a lexicon of the language model; and

output, for display as a single prediction in a suggestion region of the graphical keyboard, the one or more words predicted by the model.

10. The computing device of claim 9 , wherein the first gesture type is the sliding gesture and the second gesture type is the single-touch gesture.

11. The computing device of claim 9 , wherein the model is trained using statistical machine-learning.

12. The computing device of claim 9 , wherein the one or more processors are further configured to:

output, for display in a text entry region of the graphical keyboard, the one or more words predicted by the model.

13. The computing device of claim 9 , wherein the one or more processors are further configured to:

while outputting the one or more words predicted by the model for display in the suggestion region of the graphical keyboard, output, for display, additional words predicted by the model as additional predictions in the suggestion region.

14. The computing device of claim 9 , wherein the suggestion region comprises up to three predictions including the one or more words as a first of the three predictions.

15. The computing device of claim 9 , wherein the model is trained using machine-learning to determine the one or more words by inferring spatial and linguistic characteristics of the first gesture data relative to spatial and linguistic characteristics of the second gesture data.

16. The computing device of claim 9 , wherein the one or more processors are further configured to:

determine a speed at which the first gesture data and the second gesture data are performed; and

modify, based on the speed of the first gesture data and the second gesture data, one or more physical cost values of the spatial model,

wherein the model is trained using machine-learning to determine the one or more words by inferring the spatial characteristics of the first gesture data and the second gesture data relative to the one or more physical cost values of the spatial model.

17. A non-transitory, non-signal computer-readable storage medium comprising instructions that, when executed by one or more processors, cause the one or more processors to:

output, for display at a presence-sensitive display, a graphical user interface including a plurality of keys of a graphical keyboard;

execute a model trained using machine-learning to determine text from both sliding gestures and single-touch gestures detected by the presence-sensitive display, the model including a spatial model and a language model, the spatial model including a distribution of touch points that corresponds to at least one respective key of the graphical keyboard;

responsive to receiving first gesture data from the presence-sensitive display, input the first gesture data into the model, the first gesture data indicative of one or more first letters selected by a first gesture type that comprises a sliding gesture or a single-touch gesture to select one or more first letter keys of the graphical keyboard, the sliding gesture comprising a continuous gesture over different locations of the graphical user interface, wherein alignment points along the continuous gesture correspond to different letter keys of the graphical keyboard;

responsive to receiving second gesture data from the presence-sensitive display, input the second gesture data into the model, the second gesture data indicative of one or more second letters selected by a second gesture type that comprises the sliding gesture or the single-touch gesture to select one or more second letter keys of the graphical keyboard, the second gesture type different than the first gesture type;

determine, based at least in part on both physical cost values from the spatial model and lexical cost values from the language model, one or more words predicted by the model based on the first and second letters, the physical cost values representing first likelihoods that the first gesture data and the second gesture data correspond to the first letters and the second letters, respectively, and modified by the lexical cost values from the language model, the lexical cost values representing second likelihoods that the first letters and the second letters are included in a word in a lexicon of the language model; and

output, for display as a single prediction in a suggestion region of the graphical keyboard, the one or more words predicted by the model.

18. The non-transitory, non-signal computer-readable storage medium of claim 17 , wherein the first gesture type is the sliding gesture and the second gesture type is the single-touch gesture.

19. The non-transitory, non-signal computer-readable storage medium of claim 17 , wherein the model is trained using statistical machine-learning.

20. The non-transitory, non-signal computer-readable storage medium of claim 17 comprising further instructions that, when executed by the one or more processors, cause the one or more processors to:

determine a speed at which the first gesture data and the second gesture data were performed; and

modify, based on the speed of the first gesture data and the second gesture data, one or more physical cost values of the spatial model,

wherein the model is trained using machine-learning to determine the one or more words by inferring the spatial characteristics of the first gesture data and the second gesture data relative to the one or more physical cost values of the spatial model.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2019
From: ZHAI, SHUMIN; CHELBA, CIPRIAN IOAN
To: GOOGLE INC.
Reel/Frame 051011/0090 →
CHANGE OF NAME Recorded Nov 14, 2019
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 051032/0924 →
Continuity (5)
Continuation 15795955 · Oct 27, 2017
Continuation 14477490 · Sep 4, 2014
Continuation 13789106 · Mar 7, 2013
Provisional Application 61752790 · Jan 15, 2013
Related Publication 20200081974A1 · Mar 12, 2020