Neural network for keyboard input decoding
In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.
1. A non-transitory computer-readable storage medium comprising instructions that, when executed, configure one or more processors to:
output a graphical keyboard that includes a plurality of keys;
determine features of at least one user input to the graphical keyboard;
include the features in a matrix including features sampled over time;
provide the matrix to a model that is trained to predict a word or word phrase from the matrix; and
output, for display, the word or word phrase predicted by the model.
2. The non-transitory computer-readable storage medium of claim 1 , wherein the features include at least one of a spatial feature, a temporal feature, a lexical feature, or a contextual feature of the at least one user input.
3. The non-transitory computer-readable storage medium of claim 2 , wherein the spatial feature indicates at least one of a location of the at least one user input, a speed of the at least one user input, a direction of the at least one user input, a curvature of the at least one user input, any keys of the plurality of keys traversed by the at least one user input, or a type of the at least one user input.
4. The non-transitory computer-readable storage medium of claim 2 , wherein the temporal feature indicates at least one of an epoch time of the at least one user input or an international standard notation time of the at least one user input.
5. The non-transitory computer-readable storage medium of claim 2 , wherein the contextual feature indicates at least one of a setting or state of information maintained by a computing device, an identity of a user of the computing device, a geolocation of the computing device, an environment or climate of the computing device, audible or visual information obtained by the computing device, sensor information obtained by the computing device, a type of input field for displaying the word or word phrase predicted by the model, an application for receiving the at least one user input to the graphical keyboard.
6. A non-transitory computer-readable, storage medium comprising instructions that, when executed, configure one or more processors to:
output a graphical keyboard that includes a plurality of keys;
determine contextual features of at least one user input to the graphical keyboard;
structure a matrix based on the contextual feature and other contextual features sampled over time;
provide the matrix to a neural network that predicts a word or word phrase from the matrix; and
output, for display, the word or word phrase.
7. The non-transitory computer-readable storage medium of claim 6 , further comprising instructions that, when executed, configure the one or more processors to:
determine additional contextual features from another user input to the graphical keyboard;
structure the matrix to include the additional contextual features in the matrix; and
output, for display, another word or word phrase predicted by the model in response to providing the matrix to the model.
8. The non-transitory computer-readable storage medium of claim 7 , wherein:
each column of the matrix represents a set of spatial features of input values sampled during receipt of the at least one user input to the graphical keyboard;
each row of the matrix represents a unique spatial feature to the set of spatial features; and
each cell in the matrix includes an input value of a particular feature at a particular time.
9. The non-transitory computer-readable storage medium of claim 6 , wherein the matrix is a multi-dimensional array comprising multiple columns and rows.
10. A computing device comprising at least one processor configured to:
output a graphical keyboard that includes a plurality of keys;
determine features of at least one user input to the graphical keyboard;
include the features in a matrix including features sampled over time;
provide the matrix to a model that is trained to predict a word or word phrase from the matrix; and
output, for display, the word or word phrase predicted by the model.
11. The computing device of claim 10 , wherein the features include at least one of a spatial feature, a temporal feature, a lexical feature, or a contextual feature of the at least one user input.
12. The computing device of claim 11 , wherein the spatial feature indicates at least one of a location of the at least one user input, a speed of the at least one user input, a direction of the at least one user input, a curvature of the at least one user input, any keys of the plurality of keys traversed by the at least one user input, or a type of the at least one user input.
13. The computing device of claim 11 , wherein the temporal feature indicates at least one of an epoch time of the at least one user input or an international standard notation time of the at least one user input.
14. The computing device of claim 11 , wherein the contextual feature indicates at least one of a setting or state of information maintained by a computing device, an identity of a user of the computing device, a geolocation of the computing device, an environment or climate of the computing device, audible or visual information obtained by the computing device, sensor information obtained by the computing device, a type of input field for displaying the word or word phrase predicted by the model, an application for receiving the at least one user input to the graphical keyboard.
15. The computing device of claim 10 , wherein the at least one processor is further configured to:
determine contextual features of the at least one user input to the graphical keyboard;
structure the matrix based on the contextual feature and other contextual features sampled over time; and
responsive to structuring the matrix based on the contextual feature and the other contextual features, include the contextual features in the matrix and provide the matrix to the model.
16. The computing device of claim 15 , wherein the at least one processor is further configured to:
determine additional contextual features from another user input to the graphical keyboard;
structure the matrix to include the additional contextual features in the matrix; and
output, for display, another word or word phrase predicted by the model in response to providing the matrix to the model.
17. The computing device of claim 16 , wherein:
each column of the matrix represents a set of spatial features of input values sampled during receipt of the at least one user input to the graphical keyboard;
each row of the matrix represents a unique spatial feature to the set of spatial features; and
each cell in the matrix includes an input value of a particular feature at a particular time.
18. The computing device of claim 15 , wherein the matrix is a multi-dimensional array comprising multiple columns and rows.
19. The computing device of claim 10 , further comprising a display configured to present the graphical keyboard or the word or word phrase when output by the at least one processor.
20. The computing device of claim 10 , wherein the computing device comprises a mobile phone, a tablet computer, a laptop computer, a desktop computer, a gaming device, a media player, an e-book reader, a smart watch, or a television platform.