IP Library Granted Patent US 9,830,311
Granted Patent B2
US 9,830,311 · App. 14/477,490 · Granted Nov 28, 2017

Touch keyboard using language and spatial models

Inventors: Shumin Zhai (Los Altos, CA); Ciprian Ioan Chelba (Palo Alto, CA)
Assignee: Google LLC
G06F17/277G06F3/0237G06F3/04883G06F3/04886G06F17/275G06F17/2735
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,830,311
App. No.
14/477,490
Granted
Nov 28, 2017
Kind
B2
Abstract

A computing device outputs for display at a presence-sensitive display, a graphical keyboard comprising a plurality of keys, receives an indication of at least one gesture to select a group of keys of the plurality of keys, and determines at least one characteristic associated with the at least one gesture to select the group of keys of the plurality of keys. The computing device modifies a spatial model based at least in part on the at least one characteristic and determines a candidate word based at least in part on data provided by the spatial model and a language model, wherein the spatial model provides data based at least in part on the indication of the at least one gesture and wherein the language model provides data based at least in part on a lexicon. The computing device outputs for display at the presence-sensitive display, the candidate word.

Claims (47)

1. A method comprising:

outputting, by a computing device and for display, a graphical keyboard comprising a plurality of keys;

receiving, by the computing device, a plurality of indications of input, each respective indication of input from the plurality of indications of input corresponding to a respective location of the graphical keyboard; and

for each respective indication of input from the plurality of indications of input, incrementally:

determining, by the computing device and based at least in part on both physical cost values from a spatial model and lexical cost values from a language model, at least one predicted current word based on a set of characters that correspond to the plurality of indications of input, wherein the spatial model comprises at least one respective distribution of touch points that corresponds to at least one respective key of the graphical keyboard, and wherein the at least one predicted current word is determined based on a physical cost value from the spatial model that is modified by a lexical cost value from the language model, the physical cost value representing a first likelihood that the plurality of indications of input correspond to the set of characters, and the lexical cost value representing a second likelihood that the set of characters are included in any word in a lexicon of the language model;

determining, by the computing device and based at least in part on the at least one predicted current word, at least one predicted next word that follows the at least one predicted current word; and

outputting, by the computing device and for display, the at least one predicted current word and the at least one predicted next word as a soft commit word in a text entry area of a graphical user interface by at least:

outputting, for display in a first visual style, characters of the at least one predicted current word and the at least one predicted next word that are from the set of characters that correspond to the plurality of indications of input; and

outputting, for display in a second visual style that is different than the first visual style, characters of the at least one predicted current word and the at least one predicted next word that are not from the set of characters that correspond to the plurality of indications of input.

2. The method of claim 1 , further comprising:

for each respective indication of input from the plurality of indications of input, responsive to outputting at least the at least one predicted current word or the at least one predicted next word, incrementally modifying, by the computing device and based at least in part on a location of the key relative to a location of the respective indication of input, the spatial model.

3. The method of claim 1 , wherein determining the at least one predicted current word comprises performing, based at least in part on both the spatial model and the language model, error correction and word completion for each respective indication of input to determine the at least one predicted current word.

4. The method of claim 1 , wherein the set of characters is a portion of a complete word.

5. The method of claim 1 , wherein the language model comprises a lexicon that includes context information.

6. The method of claim 1 , wherein the language model comprises a word-level N-gram.

7. A computing device comprising:

at least one processor; and

memory configured to store instructions that, when executed, cause the at least one processor to:

output, for display, a graphical keyboard comprising a plurality of keys;

receive a plurality of indications of input, each respective indication of input from the plurality of indications of input corresponding to a respective location of the graphical keyboard; and

for each respective indication of input from the plurality of indications of input, incrementally:

determine, based at least in part on both physical cost values from a spatial model, and lexical cost values from a language model at least one predicted current word based on a set of characters that correspond to the plurality of indications of input, wherein the spatial model comprises at least one respective distribution of touch points that corresponds to at least one respective key of the graphical keyboard, and wherein the at least one predicted current word is determined based on a physical cost value from the spatial model that is modified by a lexical cost value from the language model, the physical cost value representing a first likelihood that the plurality of indications of input correspond to the set of characters, and the lexical cost value representing a second likelihood that the set of characters are included in any word in a lexicon of the language model;

determine, based at least in part on the at least one predicted current word that is the previous word, at least one predicted next word that follows the at least one predicted current word; and

output, for display, the at least one predicted current word and the at least one predicted next word as a soft commit word in a text entry area of a graphical user interface by at least:

outputting, for display in a first visual style, characters of the at least one predicted current word and the at least one predicted next word that are from the set of characters that correspond to the plurality of indications of input; and

outputting, for display in a second visual style that is different than the first visual style, characters of the at least one predicted current word and the at least one predicted next word that are not from the set of characters that correspond to the plurality of indications of input.

8. The computing device of claim 7 , wherein the instructions, when executed, further cause the at least one processor to determine a plurality of sets of one or more predicted current words, each respective set of one or more predicted current words from the plurality of sets of one or more predicted current words being based on a respective set of characters that correspond to the plurality of indications of input.

9. The computing device of claim 7 , wherein the instructions, when executed, further cause the at least one processor to perform error correction and word completion for each respective indication of input to determine the at least one predicted current word.

10. The computing device of claim 7 , wherein the instructions, when executed, further cause the at least one processor to:

determine, based at least in part on both a lexicon and a word-level N-gram, a frequency value associated with the set of characters, wherein the frequency value represents a frequency of occurrence of the set of characters in the lexicon; and

determine, based at least in part on the frequency value, the at least one predicted current word.

11. The computing device of claim 7 , wherein the lexicon comprises a listing of words and associated context information.

12. The computing device of claim 7 , wherein the set of characters is a portion of a complete word.

13. A non-transitory, non-signal computer-readable medium encoded with instructions that, when executed, cause at least one processor to:

output, for display, a graphical keyboard comprising a plurality of keys;

receive a plurality of indications of input, each respective indication of input from the plurality of indications of input corresponding to a respective location of an input device; and

for each respective indication of input from the plurality of indications of input, incrementally:

determine, based at least in part on both physical cost values from a spatial model, and lexical cost values from a language model, at least one predicted current word based on a set of characters that correspond to the plurality of indications of input, wherein the spatial model comprises at least one respective distribution of touch points that corresponds to at least one respective key of the graphical keyboard, and wherein the at least one predicted current word is determined based on a physical cost value from the spatial model that is modified by a lexical cost value from the language model, the physical cost value representing a first likelihood that the plurality of indications of input correspond to the set of characters, and the lexical cost value representing a second likelihood that the set of characters are included in any word in a lexicon of the language model;

determining, based at least in part on the at least one predicted current word, at least one predicted next word that follows the at least one predicted current word; and

output, for display, the at least one predicted current word and the at least one predicted next word as a soft commit word in a text entry area of a graphical user interface by at least:

outputting, for display in a first visual style, characters of the at least one predicted current word and the at least one predicted next word that are from the set of characters that correspond to the plurality of indications of input; and

outputting, for display in a second visual style that is different than the first visual style, characters of the at least one predicted current word and the at least one predicted next word that are not from the set of characters that correspond to the plurality of indications of input.

14. The non-transitory, non-signal computer-readable medium of claim 13 , further encoded with instructions that, when executed, cause the at least one processor to:

for each respective indication of input from the plurality of indications of input, responsive to outputting at least the at least one predicted current word or the at least one predicted next word, incrementally modify, by the computing device and based at least in part on a location of the key relative to a location of the respective indication of input, the spatial model.

15. The non-transitory, non-signal computer-readable medium of claim 13 , wherein the instructions that cause the at least one processor to determine the at least one predicted current word comprise instructions that, when executed, cause the at least one processor to perform, based at least in part on both the spatial model and the language model, error correction and word completion for each respective indication of input to determine the at least one predicted current word.

16. The non-transitory, non-signal computer-readable medium of claim 13 , wherein the at least one respective distribution of touch points comprises at least one respective two-dimensional Gaussian distribution.

17. The non-transitory, non-signal computer-readable medium of claim 13 , wherein the set of characters is a portion of a complete word.

Assignments (2)
CHANGE OF NAME Recorded Oct 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044129/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 4, 2014
From: ZHAI, SHUMIN; CHELBA, CIPRIAN IOAN
To: GOOGLE INC.
Reel/Frame 033671/0469 →
Continuity (3)
Continuation 13789106 · Mar 7, 2013
Provisional Application 61752790 · Jan 15, 2013
Related Publication 20140372880A1 · Dec 18, 2014