IP Library Granted Patent US 8,917,238
Granted Patent B2
US 8,917,238 · App. 13/536,778 · Granted Dec 23, 2014

Eye-typing term recognition

Inventors: David Nister (Bellevue, WA); Vaibhav Thukral (Kirkland, WA); Djordje Nijemcevic (Kragujevac, RS); Ruchi Bhargava (Redmond, WA)
Assignee: Microsoft Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,917,238
App. No.
13/536,778
Granted
Dec 23, 2014
Kind
B2
Abstract

Various embodiments related to entering text into a computing device via eye-typing are disclosed. For example, one embodiment provides a method that includes receiving a data set including a plurality of gaze samples, each gaze sample including a gaze location and a corresponding point in time. The method further comprises processing the plurality of gaze samples to determine one or more likely terms represented by the data set.

Claims (42)

1. In a computing device comprising a data-holding subsystem and logic subsystem, a method, comprising:

receiving a data set comprising a plurality of gaze samples, each gaze sample comprising a gaze location and a corresponding point in time;

processing the plurality of gaze samples to determine one or more likely terms represented by the data set by comparing the plurality of gaze samples to a layout of user input keys displayed on a display, and based on overlapping user input key and gaze locations and a length of time spent gazing at each user input key, assigning a per-key score to each user input key for each point in time; and

providing the one or more likely terms to a display device for display.

2. The method of claim 1 , wherein the data set is received from a remote device via a network.

3. The method of claim 1 , wherein receiving the data set further comprises:

receiving a trigger input indicating an initiation of eye-typing input;

receiving the plurality of gaze samples from one or more image sensors; and

receiving an input signaling an end of the eye-typing input, wherein the corresponding point in time is a time from the trigger.

4. The method of claim 1 , further comprising, based on the per-key scores, compiling a string likelihood score for each character string in a character string database.

5. The method of claim 4 , wherein the string likelihood score rewards character strings including two or more characters in a similar order to an order in which corresponding user input keys received a user gaze.

6. The method of claim 1 , further comprising providing to the display device one or more character strings having a high string likelihood score as the one or more likely terms.

7. The method of claim 1 , further comprising converting a set of per-key scores to one or more intervals in time, and assigning an interval score to each interval.

8. The method of claim 7 , further comprising:

compiling a directed graph of one or more characters, each character corresponding to a user input key having an interval score above a threshold over a period of time; and

comparing each character string from a character string database to the directed graph, and sending to the display one or more character strings that match a path from the directed graph to be displayed as the one or more likely terms.

9. The method of claim 1 , further comprising:

receiving a gaze sample indicative of a selection of one of the one or more likely terms; and

requesting from a service information relating to the selected term.

10. In a computing device comprising a display, storage subsystem, and logic subsystem, a method, comprising:

receiving a trigger input indicating initiation of eye-typing;

receiving a plurality of gaze samples each comprising a gaze location and a corresponding time from the trigger;

receiving an input signaling an end of the eye-typing input;

comparing the plurality of gaze samples to a layout of user input keys displayed on the display, and based on overlapping user input key and gaze locations, assigning a per-key score to each user input key for each point in time;

based on the per-key scores, compiling a string likelihood score for each character string in a character string database; and

displaying on the display one or more likely terms corresponding to character strings having a high string likelihood score.

11. The method of claim 10 , wherein the trigger comprises a gaze location at a trigger region displayed on the display.

12. The method of claim 10 , wherein the string likelihood score rewards character strings including two or more characters in a similar order to an order in which corresponding user input keys were gazed at.

13. The method of claim 10 , wherein the string likelihood score penalizes character strings having a length greater than a threshold.

14. The method of claim 10 , wherein the display is a see-through display.

15. In a computing device comprising a display, storage subsystem, and logic subsystem, a method, comprising:

receiving a trigger input indicating initiation of eye-typing;

receiving a plurality of gaze samples each comprising a gaze location and a corresponding time from the trigger;

receiving an input signaling an end of the eye-typing input;

comparing the plurality of gaze samples to a layout of user input keys displayed on the display, and based on overlapping user input key and gaze locations, assigning a per-key score to each user input key for each point in time;

converting a set of per-key scores to one or more intervals in time, and assigning an interval score to each interval;

compiling a directed graph of one or more characters, each character corresponding to a user input key having an interval score above a threshold; and

comparing each character string from a character string database to the directed graph, and displaying on the display one or more likely terms corresponding to character strings that match a path from the directed graph.

16. The method of claim 15 , wherein each per-key score is further based on a length of time each respective key was gazed at.

17. The method of claim 15 , wherein converting the set of per-key scores to one or more intervals in time further comprises assigning an interval in time to a particular key, the interval including a time duration from when a per-key score for that particular key reached a threshold to when that key's per-key score dropped below the threshold.

18. The method of claim 15 , wherein the interval score comprises an average per-key score for that particular key during the interval.

19. The method of claim 15 , wherein displaying on the display one or more likely terms corresponding to character strings that match a path from the directed graph further comprises displaying on the display one or more likely terms corresponding to character strings that are less than a threshold edit distance from a path of the directed graph.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 5, 2014
From: NISTER, DAVID; THUKRAL, VAIBHAV; NIJEMCEVIC, DJORDJE; BHARGAVA, RUCHI
To: MICROSOFT CORPORATION
Reel/Frame 033675/0893 →
Continuity (1)
Related Publication 20140002341A1 · Jan 2, 2014