GESTURE DETECTION FOR NAVIGATION THROUGH A USER INTERFACE
One embodiment of a disclosed system, method, and computer readable storage medium which includes instructions for evaluating the intention of a gesture performed on a contact-sensitive display. A tablet scribe system detects a gesture performed using an input mechanism on a contact-sensitive screen of a display system and identifies a path of points on the contact-sensitive screen that the gesture traversed. The system determines rule values for each of a plurality of gesture rules based on the path of points, which describe a measurement of a property of the gesture that informs whether the gesture was performed intentionally. Based on the rule values, the system determines a representative rule value that describes a likelihood that the gesture was performed intentionally and classifies the gesture as intentional or unintentional based on the representative rule value. If intentional, the system enables (or provides) for display) a rendering of the gesture on the contact-sensitive screen.
1 . A non-transitory computer-readable medium comprising stored computer-readable instructions for evaluating an intention of a gesture performed on a contact-sensitive screen that, when executed by a processor, cause the processor to:
detect, at the contact-sensitive screen, a gesture performed using an input mechanism, the gesture having been encoded in a signal generated by contact between the input mechanism and the contact-sensitive screen;
identify, from the encoded signal of the gesture, a path of points on the contact-sensitive screen that the gesture traversed, wherein the path of points begins at a start point on the contact-sensitive screen and ends at an end point on the contact-sensitive screen;
determine rule values for each of a plurality of gesture rules based on the path of points, wherein each rule value represents a measurement of a property of the gesture that informs whether the gesture was performed intentionally;
determine a representative rule value based on a combination of the rule values determined for the plurality of gesture rules, the representative rule value describing a likelihood that the gesture was performed intentionally based on a combination of conditions for a gesture to be performed intentionally; and
classify, upon a determination that the representative rule value is greater than a threshold representative rule value, the gesture as intentionally performed by the input mechanism; and
enable for display the gesture on the contact-sensitive screen over the path of points on the contact-sensitive screen.
2 . The non-transitory computer-readable medium of claim 1 , comprising further instructions that when executed cause the processor to:
determine a time difference between contact between the input mechanism and the contact-sensitive screen at the start point and contact between the input mechanism and the contact-sensitive screen at a next point on the path of points; and
identify, upon a determination that the time difference is less than a threshold amount of time, remaining points of the path of points traversed by the gesture and the end point.
3 . The non-transitory computer-readable medium of claim 2 , comprising further instructions that when executed cause the processor to:
deactivate, upon a determination that the time difference is greater than a threshold amount of time, contact-sensing sensors of the contact-sensitive screen;
determine a distance between the input mechanism and the contact-sensitive screen; and
reactivate, upon a determination that the determined distance is within a threshold distance, the contact-sensing sensors of the contact-sensitive screen to identify the next point on the path of points.
4 . The non-transitory computer-readable medium of claim 1 , wherein a gesture rule of the plurality of gesture rules describes a threshold linearity of the gesture, the non-transitory computer-readable medium comprising further instructions that when executed cause the processor to:
measure, for each point on the path of points, a straightness of each segment of a gesture, wherein each segment of the gesture is formed between each point and a next point on the path of points;
aggregate the measured straightness for each segment of the gesture to determine a straightness measurement for the gesture; and
determine, upon the straightness measurement for the gesture being greater than the threshold linearity, a rule value for the gesture rule indicating that the gesture was performed intentionally.
5 . The non-transitory computer-readable medium of claim 1 , wherein a gesture rule of the plurality of gesture rules describes a threshold angle between points on the path of points, the non-transitory computer-readable medium comprising further instructions that when executed cause the processor to:
measure, for each point on the path of points, an angle between the point and a next point on the path of points;
determine, upon the measured angle being less than the threshold angle, a rule value for the gesture rule indicating that the gesture was performed intentionally.
6 . The non-transitory computer-readable medium of claim 1 , wherein a gesture rule of the plurality of gesture rules describes a threshold acceleration of intentionally performed gesture, the non-transitory computer-readable medium comprising further instructions that when executed cause the processor to:
determine a distance between a position of the start point on the contact-sensitive screen and a position of the end point on the contact-sensitive screen;
determine an acceleration of the gesture based on the determined distance and a velocity at which the gesture was performed; and
determine, upon a determination that the acceleration of the gesture is less than the threshold acceleration, a rule value for the gesture rule indicating that the gesture was performed intentionally.
7 . The non-transitory computer-readable medium of claim 1 , wherein a gesture rule of the plurality of gestures rules describes a threshold size of points along the gesture, the non-transitory computer-readable medium comprising further instructions that when executed cause the processor to:
measure a size of each point on the path of points based on a count of contact-sensitive sensors activated by contact between the input mechanism and the contact-sensitive screen at the point; and
determine, upon the measured size being less than the threshold size, a rule value for the gesture rule indicating that the gesture was performed intentionally.
8 . A table device comprising:
an input mechanism;
a contact-sensitive surface; and
a non-transitory computer-readable medium comprising stored computer-readable instructions for evaluating an intention of a gesture performed on a contact-sensitive screen that, when executed by a processor, cause the processor to:
detect, at the contact-sensitive screen, a gesture performed using an input mechanism, the gesture having been encoded in a signal generated by contact between the input mechanism and the contact-sensitive screen;
identify, from the encoded signal of the gesture, a path of points on the contact-sensitive screen that the gesture traversed, wherein the path of points begins at a start point on the contact-sensitive screen and ends at an end point on the contact-sensitive screen;
determine rule values for each of a plurality of gesture rules based on the path of points, wherein each rule value represents a measurement of a property of the gesture that informs whether the gesture was performed intentionally;
determine a representative rule value based on a combination of the rule values determined for the plurality of gesture rules, the representative rule value describing a likelihood that the gesture was performed intentionally based on a combination of conditions for a gesture to be performed intentionally; and
classify, based on the representative rule value being greater than a threshold representative rule value, the gesture as intentionally performed by the input mechanism; and
enable for display the gesture on the contact-sensitive screen over the path of points on the contact-sensitive screen.
9 . The tablet device of claim 8 , wherein the non-transitory computer-readable medium comprises further instructions that when executed cause the processor to:
determine a time difference between contact between the input mechanism and the contact-sensitive screen at the start point and contact between the input mechanism and the contact-sensitive screen at a next point on the path of points; and
identify, upon determination that the time difference is less than a threshold amount of time, remaining points of the path of points traversed by the gesture and the end point.
10 . The tablet device of claim 9 , wherein the non-transitory computer-readable medium comprises further instructions that when executed cause the processor to:
deactivate, upon a determination that the time difference is greater than a threshold amount of time, contact-sensing sensors of the contact-sensitive screen;
determine a distance between the input mechanism and the contact-sensitive screen; and
reactivate, upon a determination that the determined distance is within a threshold distance, the contact-sensing sensors of the contact-sensitive screen to identify the next point on the path of points.
11 . The tablet device of claim 9 , wherein a gesture rule of the plurality of gesture rules describes a threshold linearity of the gesture, the non-transitory computer-readable medium comprising further instructions that when executed cause the processor to:
measure, for each point on the path of points, a straightness of each segment of a gesture, wherein each segment of the gesture is formed between each point and a next point on the path of points;
aggregate the measured straightness for each segment of the gesture to determine a straightness measurement for the gesture; and
determine, upon the straightness measurement for the gesture being greater than the threshold linearity, a rule value for the gesture rule indicating that the gesture was performed intentionally.
12 . The tablet device of claim 9 , wherein a gesture rule of the plurality of gesture rules describes a threshold angle between points on the path of points, the non-transitory computer-readable medium comprising further instructions that when executed cause the processor to:
measure, for each point on the path of points, an angle between the point and a next point on the path of points;
determine, upon the measured angle being less than the threshold angle, a rule value for the gesture rule indicating that the gesture was performed intentionally.
13 . The tablet device of claim 9 , wherein a gesture rule of the plurality of gesture rules describes a threshold acceleration of intentionally performed gesture, the non-transitory computer-readable medium comprising further instructions that when executed cause the processor to:
determine a distance between a position of the start point on the contact-sensitive screen and a position of the end point on the contact-sensitive screen;
determine an acceleration of the gesture based on the determined distance and a velocity at which the gesture was performed; and
determine, upon a determination that the acceleration of the gesture is less than the threshold acceleration, a rule value for the gesture rule indicating that the gesture was performed intentionally.
14 . The tablet device of claim 9 , wherein a gesture rule of the plurality of gestures rules describes a threshold size of points on the path of point, the non-transitory computer-readable medium comprising further instructions that when executed cause the processor to:
measure a size of each point on the path of points based on a count of contact-sensitive sensors activated by contact between the input mechanism and the contact-sensitive screen at the point; and
determine, upon the measured size being less than the threshold size, a rule value for the gesture rule indicating that the gesture was performed intentionally.
15 . A system comprising:
a pen input detector configured to detect, at the contact-sensitive screen, a gesture performed using an input mechanism, is the gesture having been encoded in a signal generated by contact between the input mechanism and the contact-sensitive screen;
a gesture processor configured to:
identify, from the encoded signal of the gesture, a path of points on the contact-sensitive screen that the gesture traversed, wherein the path of points begins at a start point on the contact-sensitive screen and ends at an end point on the contact-sensitive screen;
determine rule values for each of a plurality of gesture rules based on the path of points, wherein each rule value represents a measurement of a property of the gesture that informs whether the gesture was performed intentionally;
determine a representative rule value based on a combination of the rule values determined for the plurality of gesture rules, the representative rule value describing a likelihood that the gesture was performed intentionally based on a combination of conditions for a gesture to be performed intentionally; and
a gesture classifier configured to classify the gesture as intentionally performed using the input is upon a determination that the representative rule value being greater than a threshold representative rule value; and
a display system configured to render the gesture on the contact-sensitive screen along the path of points on the contact-sensitive screen.
16 . The system of claim 15 , wherein:
the gesture input detector is further configured to determine a time difference between contact between the input mechanism and the contact-sensitive screen at the start point and contact between the input mechanism and the contact-sensitive screen at a next point on the path of points; and
the gesture processor is further configured to identify remaining points of the path of points traversed by the gesture and the end point upon a determination that the time difference is less than a threshold amount of time.
17 . The system of claim 16 , wherein:
the gesture input detector is further configured to deactivate contact-sensing sensors of the contact-sensitive screen upon a determination that the time difference is greater than a threshold amount of time;
the gesture processor is further configured to:
determine a distance between the input mechanism and the contact-sensitive screen; and
reactivate the contact-sensing sensors of the contact-sensitive screen to identify the next point on the path of points upon a determination that the determined distance is within a threshold distance.
18 . The system of claim 15 , wherein a gesture rule of the plurality of gesture rules describes a threshold linearity of the gesture, the gesture processor further configured to:
measure, for each point on the path of points, a straightness of each segment of a gesture, wherein each segment of the gesture is formed between each point and a next point on the path of points;
aggregate the measured straightness for each segment of the gesture to determine a straightness measurement for the gesture; and
determine, upon the straightness measurement for the gesture being greater than the threshold linearity, a rule value for the gesture rule indicating that the gesture was performed intentionally.
19 . The system of claim 15 , wherein a gesture rule of the plurality of gesture rules describes a threshold angle between points on the path of points, the gesture processor further configured to:
measure, for each point on the path of points, an angle between the point and a next point on the path of points;
determine, upon the measured angle being less than the threshold angle, a rule value for the gesture rule indicating that the gesture was performed intentionally.
20 . The system of claim 15 , wherein a gesture rule of the plurality of gesture rules describes a threshold acceleration of intentionally performed gesture, the gesture processor further configured to:
determine a distance between a position of the start point on the contact-sensitive screen and a position of the end point on the contact-sensitive screen;
determine an acceleration of the gesture based on the determined distance and a velocity at which the gesture was performed; and
determine, upon a determination that the acceleration of the gesture is less than the threshold acceleration, a rule value for the gesture rule indicating that the gesture was performed intentionally.
21 . The system of claim 15 , wherein a gesture rule of the plurality of gestures rules describes a threshold size of points on the path of point, the gesture processor further configured to:
measure a size of each point on the path of points based on a count of contact-sensitive sensors activated by contact between the input mechanism and the contact-sensitive screen at the point; and
determine, upon the measured size being less than the threshold size, a rule value for the gesture rule indicating that the gesture was performed intentionally.