Interaction using touch and non-touch gestures
View Patent ↗A computer interface may use touch- and non-touch-based gesture detection systems to detect touch and non-touch gestures on a computing device. The systems may each capture an image, and interpret the image as corresponding to a predetermined gesture. The systems may also generate similarity values to indicate the strength of a match between a captured image and corresponding gesture, and the system may combine gesture identifications from both touch- and non-touch-based gesture identification systems to ultimately determine the gesture. A threshold comparison algorithm may be used to apply different thresholds for different gesture detection systems and gesture types.
1. A method comprising:
receiving image data corresponding to a user's gesture;
processing, by a computing device, the image data to generate a touch gesture similarity value that defines similarity of the user's gesture to one or more touch gestures and a non-touch gesture similarity value that defines similarity of the user's gesture to one or more non-touch gestures;
processing the touch gesture similarity value and the non-touch gesture similarity value to identify the user's gesture, resulting in an identified gesture; and
generating an output in response to the identified gesture.
2. The method of claim 1 , wherein processing the touch gesture similarity value and the non-touch gesture similarity value includes:
comparing the touch gesture similarity value with a first threshold value; and
comparing the non-touch similarity value with a second threshold value.
3. The method of claim 2 , further comprising setting the first and second threshold values to different values based on gesture type.
4. The method of claim 1 , wherein a first portion of the image data is received from a touch-based gesture detection system, and a second portion of the image data is received from a non-touch based gesture detection system.
5. The method of claim 4 , wherein processing the image data includes using the first portion of the image data from the touch-based gesture detection system to detect a non-touch gesture.
6. The method of claim 1 , further comprising determining a differential similarity value, and comparing the differential similarity value with a difference between the touch gesture similarity value and the non-touch gesture similarity value.
7. The method of claim 1 , further comprising:
using contextual information to select between a touch and non-touch gesture when processing the touch gesture similarity value and the non-touch gesture similarity value.
8. The method of claim 7 , wherein the contextual information includes information identifying a previously-identified gesture.
9. The method of claim 7 , wherein the contextual information includes information identifying a command set of an application.
10. One or more computer-readable media, storing computer-executable instructions that, when executed, cause a computer to at least:
receive image data corresponding to a user's gesture;
process the image data to generate a touch gesture similarity value that defines similarity of the user's gesture to one or more touch gestures and a non-touch gesture similarity value that defines similarity of the user's gesture to one or more non-touch gestures;
process the touch gesture similarity value and the non-touch gesture similarity value to identify the user's gesture, resulting in an identified gesture; and
generate an output in response of the identified gesture.
11. The one or more computer-readable media of claim 10 , wherein processing the touch gesture similarity value and the non-touch gesture similarity value includes:
comparing the touch gesture similarity value with a first threshold value; and
comparing the non-touch gesture similarity value with a second threshold value.
12. The one or more computer-readable media of claim 11 , wherein the first and second threshold values differ based on different gesture types.
13. The one or more computer-readable media of claim 10 , wherein a first portion of the image data is received from a touch-based gesture detection system, and a second portion of the image data is received from a non-touch based gesture detection system.
14. The one or more computer-readable media of claim 13 , further storing computer-executable instructions that, when executed, cause the computer to: use the first portion of the image data from the touch-based gesture detection system to detect a non-touch gesture.
15. The one or more computer-readable media of claim 10 , further storing computer-executable instructions that, when executed, cause the computer to compare a differential similarity value with a difference between the touch gesture similarity value and the non-touch gesture similarity value.
16. The one or more computer-readable media of claim 10 , further storing computer-executable instructions that, when executed, cause the computer to:
use contextual information to select between a touch gesture and a non-touch gesture.
17. The one or more computer-readable media of claim 16 , wherein the contextual information includes information identifying a previously-identified gesture.
18. The one or more computer-readable media of claim 16 , wherein the contextual information includes information identifying a command set of an application.
19. An apparatus, comprising:
one or more processors; and
memory storing computer-executable instructions that, with the processor, cause the apparatus to at least:
receive image data corresponding to a user's gesture;
process the image data to generate a touch gesture similarity value that defines similarity of the user's gesture to one or more touch gestures and a non-touch gesture similarity value that defines similarity of the user's gesture to one or more non-touch gestures;
process the touch gesture similarity value and the non-touch gesture similarity value to identify the user's gesture, resulting in an identified gesture; and
generate an output in response to the identified gesture.
20. The apparatus of claim 19 , wherein processing the touch gesture similarity value and the non-touch gesture similarity value includes:
comparing the touch gesture similarity value with a first threshold value; and
comparing the non-touch similarity value with a second threshold value.
21. The apparatus of claim 20 , wherein the first and second threshold values differ based on different gesture types.
22. The apparatus of claim 19 , wherein a first portion of the image data is received from a touch-based gesture detection system, and a second portion of the image data is received from a non-touch based gesture detection system.
23. The apparatus of claim 19 , wherein the memory further stores computer-executable instructions that, with the processor, cause the apparatus to use the first portion of the image data from a touch-based gesture detection system to detect a non-touch gesture.
24. The apparatus of claim 19 , wherein the memory further stores computer-executable instructions that, with the processor, cause the apparatus to compare a differential similarity value with a difference between the touch gesture and non-touch gesture similarity values.
25. The apparatus of claim 19 , wherein the memory further stores computer-executable instructions that, with the processor, cause the apparatus to use contextual information to select between a touch gesture and a non-touch gesture.
26. The apparatus of claim 25 , wherein the contextual information includes information identifying a previously-identified gesture.
27. The apparatus of claim 25 , wherein the contextual information includes information identifying a command set of an application.
28. The apparatus of claim 19 , further comprising an infrared camera and a plurality of infrared light sources positioned below a screen, wherein the infrared camera is configured to supply at least a portion of the image data.
29. The apparatus of claim 19 , further comprising a visible light camera positioned above a screen, wherein the visible light camera is configured to supply at least a portion of the image data.