Gesture recognition
View Patent ↗A state machine gesture recognition algorithm for interpreting streams of coordinates received from a touch sensor. The gesture recognition code can be written in a high level language such as C and then compiled and embedded in a microcontroller chip, or CPU chip as desired. The gesture recognition code can be loaded into the same chip that interprets the touch signals from the touch sensor and generates the time series data, e.g. a microcontroller, or other programmable logic device such as a field programmable gate array.
1. A touch sensor device comprising:
a sensor having a sensitive area extending in at least one-dimension and arranged to output sense signals responsive to proximity of an object to the sensitive area;
a processor operable to execute logic stored in one or more tangible media, the logic, when executed by the processor, configured to:
calculate positions of interactions with the sensitive area from an analysis of the sense signals; and
analyze a time series of data to distinguish one or more gesture inputs from the time series of data using gesture-recognition code comprising a plurality of state-machine modules, the plurality of state-machine modules comprising:
a first one-touch state-machine module, the first one-touch state-machine module being operable to recognize at least a first one-touch gesture and generate a first output based on the first one-touch gesture;
a second one-touch state-machine module, the second one-touch state-machine module being operable to recognize at least a second one-touch gesture and generate a second output based on the second one-touch gesture, the first one-touch gesture and the second one-touch gesture collectively comprising at least two simultaneous interactions with the sensitive area; and
a multi-touch state-machine module, the multi-touch state-machine module being operable to recognize, based on at least the first and second outputs, at least one multi-touch gesture, the first one-touch state-machine module, the second one-touch state-machine module, and the multi-touch state-machine module being distinct state-machine modules; and
output the recognized at least one multi-touch gesture.
2. The device of claim 1 , wherein each of the plurality of state-machine modules includes an idle-state module and a plurality of gesture-interpretation-state modules, the idle state module being entered at the start of operation and able to be returned to from at least some of the gesture-interpretation-state modules.
3. The device of claim 2 , wherein each of the plurality of gesture-interpretation-state modules for each of the one-touch state-machine modules includes a touch-state module and the idle-state module passing, responsive to a touch, control to the touch-state module.
4. The device of claim 3 , wherein the plurality of gesture-interpretation-state modules for each of the first and second one-touch state-machine modules includes a press-state module to which control can pass from a touch-state module if an interaction with the sensitive area of a duration longer than a threshold duration is sensed in the touch-state module.
5. The device of claim 2 , wherein the processor is operable to pass control to the multi-touch state-machine module based on the number of simultaneous interactions with the sensitive area defined by the time series data at the time.
6. The device of claim 5 , wherein:
the multi-touch state-machine module is for processing two simultaneous interactions with the sensitive area; and
the device further comprises another multi-touch state-machine module for processing three simultaneous interactions with the sensitive area.
7. The device of claim 2 , wherein the plurality of gesture-interpretation-state modules includes a plurality of state modules operable to recognize motion-related gestures derived from one or more moving interactions with the sensitive area.
8. The device of claim 1 , wherein the gesture-recognition code is configured to recognize gestures having up to N simultaneous interactions with the sensitive area, N being at least 2, and comprising N one-touch state-machine modules operable to recognize only one-touch gestures and N−1 multi-touch state-machine modules each operable to recognize only n-touch gestures, wherein n=2 to N.
9. The device of claim 1 , wherein the logic is accommodated in, and run on, a single integrated circuit.
10. The device of claim 1 , wherein:
the processor comprises at least one first processor and at least one second processor;
the at least one first processor is configured to calculate the positions of the interactions with the sensitive area from the analysis of the sense signals; and
the at least one second processor is configured to:
analyze the time series of data to distinguish the one or more gesture inputs from the time series of data using gesture-recognition code comprising a plurality of state-machine modules; and
output the recognized at least one multi-touch gesture.
11. A method comprising:
calculating, by a processor operable to execute position processing logic, positions of interactions with a touch sensitive area from an analysis of sense signals;
outputting a time series of data indicative of the interaction positions on the sensor, the interaction positions corresponding to touches;
receiving, by a processor operable to execute gesture-processing logic, a time series of data indicative of the interaction positions on the sensor, the interaction positions corresponding to touches;
analyzing, by the processor operable to execute gesture-processing logic, the time series of data to distinguish one or more gesture inputs from the time series of data using gesture-recognition code comprising a plurality of state-machine modules, the plurality of state-machine modules comprising:
a first one-touch state-machine module, the first one-touch state-machine module being operable to recognize at least a first one-touch gesture and generate a first output based on the first one-touch gesture;
a second one-touch state-machine module, the second one-touch state-machine module being operable to recognize at least a second one-touch gesture and generate a second output based on the second one-touch gesture, the first one-touch gesture and the second one-touch gesture collectively comprising at least two simultaneous interactions with the sensitive area; and
a multi-touch state-machine module, the multi-touch state-machine module being operable to recognize, based on at least the first and second outputs, at least one multi-touch gesture, the first one-touch state-machine module, the second one-touch state-machine module, and the multi-touch state-machine module being distinct state-machine modules; and
outputting, by the processor operable to execute gesture-processing logic, the recognized at least one multi-touch gesture.
12. The method of claim 11 , wherein each of the plurality of state-machine modules includes an idle-state module and a plurality of gesture-interpretation-state modules, the idle-state module being entered at the start of operation and being able to be returned to from at least some of the gesture-interpretation-state modules.
13. The method of claim 12 , wherein the plurality of gesture-interpretation-state modules includes a plurality of state modules operable to recognize motion-related gestures derived from one or more moving interactions with the sensitive area.
14. The method of claim 12 , wherein each of the plurality of gesture-interpretation-state modules for each of the one-touch state-machine modules includes a touch-state module and the idle-state module passing, responsive to an interaction with the sensitive area, control to the touch-state module.
15. The method of claim 12 , wherein the gesture-processing logic is operable to pass control to the multi-touch state-machine module based on the number of simultaneous interactions with the sensitive area defined by the time series of data at the time.
16. The method of claim 15 , wherein:
the multi-touch state-machine module is for processing two simultaneous interactions with the sensitive area; and
the plurality of state-machine modules further comprises another multi-touch state-machine module for processing three simultaneous interactions with the sensitive area.
17. The method of claim 14 , wherein the plurality of gesture-interpretation-state modules for each of the first and second one-touch state-machine modules includes a press-state module to which control can pass from a touch-state module if an interaction with the sensitive area of a duration longer than a threshold duration is sensed in the touch-state module.
18. The method of claim 17 , wherein the press-state is a super-state comprising multiple sub-states to distinguish between different durations of press.
19. The method of claim 11 , wherein the gesture-recognition code is configured to recognize gestures having up to N simultaneous interactions with the sensitive area, N being at least 2, and comprising N one-touch state-machine modules operable to recognize only one-touch gestures and N−1 multi-touch state-machine modules each operable to recognize only n-touch gestures, wherein n=2 to N.
20. An apparatus comprising:
a memory element;
a processor operable to execute logic stored in one or more tangible media, the logic, when executed by the processor, operable to:
receive a time series of data indicative of the interaction positions on a sensor, the interaction positions corresponding to touches;
analyze the time series of data to distinguish one or more gesture inputs from the time series of data using gesture-processing logic, the gesture-processing logic being coded with gesture-recognition code comprising a plurality of state-machine modules, each of the plurality of state-machine modules including a plurality of gesture-interpretation modules, the plurality of state-machine modules further comprising:
a first one-touch state-machine module, the first one-touch state-machine module being operable to recognize at least a first one-touch gesture and generate a first output based on the first one-touch gesture;
a second one-touch state-machine module, the second one-touch state-machine module being operable to recognize at least a second one-touch gesture and generate a second output based on the second one-touch gesture, the first one-touch gesture and the second one-touch gesture collectively comprising at least two simultaneous interactions with the sensitive area; and
a multi-touch state-machine module, the multi-touch state-machine module being operable to recognize, based on at least the first and second outputs, at least one multi-touch gesture, the first one-touch state-machine module, the second one-touch state-machine module, and the multi-touch state-machine module being distinct state-machine modules; and
output the recognized at least one multi-touch gesture.