Gesture-based device control using pressure-sensitive sensors
Embodiments of the present invention are directed toward controlling electronic devices based on hand gestures detected by detecting the topography of a portion of a user's body. For example, pressure data indicative of a user's bone and tissue position corresponding to a certain movement, position, and/or pose of a user's hand may be detected. An electromyographic (EMG) sensor coupled to the user's skin can also be used to determine gestures made by the user. These sensors can be coupled to a camera that can be used to capture images, based on recognized gestures, of a device. The device can then be identified and controlled.
1. A system for controlling a device based on hand gestures, the system comprising:
a camera;
a pressure-sensitive sensor configured to detect a topography of a portion of a body of a user and provide pressure data to a processing module;
the processing module configured to:
identify a first gesture associated with the topography of the portion of the body of the user based on a first pressure data;
cause the camera to capture an image of the device subsequent to identifying the first gesture;
identify the device using the image of the device captured by the camera; and
identify a second gesture associated with the topography of the portion of the body of the user based on a second pressure data and
a transmitter configured to transmit information to the device based on the identified second gesture.
2. The system for controlling the device based on hand gestures of claim 1 , wherein:
the portion of the body of the user is a back of a hand of the user; and
the pressure-sensitive sensor is configured to detect the topography of the back of the hand as affected by metacarpal position.
3. The system for controlling the device based on hand gestures of claim 1 , wherein:
the portion of the body of the user is a wrist of the user; and
the pressure-sensitive sensor is configured to detect the topography of at least a portion of the wrist.
4. The system for controlling the device based on hand gestures of claim 1 , further comprising:
a motion sensor configured to detect motion of the user and transmit motion data to the processing module.
5. The system for controlling the device based on hand gestures of claim 4 , wherein:
the processing module is further configured to identify the second gesture using the motion data.
6. The system for controlling the device based on hand gestures of claim 4 , wherein the motion sensor comprises at least one of an accelerometer or a gyroscope.
7. The system for controlling the device based on hand gestures of claim 1 , wherein the portion of the body of the user comprises a forearm of the user.
8. The system for controlling the device based on hand gestures of claim 1 , further comprising an electromyographic sensor.
9. The system for controlling the device based on hand gestures of claim 1 , further comprising a receiver configured to receive, from the device, information indicative of commands for controlling the device using hand gestures.
10. The system for controlling the device based on hand gestures of claim 1 , further comprising a second pressure-sensitive sensor configured to detect a topography of a second portion of the body of the user and provide a third pressure data to the processing module, wherein the processing module is further configured to identify the second gesture using the third pressure data.
11. The system for controlling the device based on hand gestures of claim 1 , wherein the identified second gesture comprises a closed fist.
12. The system for controlling the device based on hand gestures of claim 1 , wherein the identified second gesture comprises a finger touching a thumb.
13. The system for controlling the device based on hand gestures of claim 1 , wherein the identified second gesture comprises a wrist being flexed such that the hand moves up, down, left, or right.
14. A method for controlling a device based on gestures, the method comprising:
collecting a first pressure data that indicates a topography of a portion of a body of a user;
identifying a first gesture associated with the topography of the portion of the body of the user based on the first pressure data
causing a camera to capture an image of the device subsequent to identifying the first gesture;
identifying the device using the image;
collecting a second pressure data that indicates the topography of the portion of the body of the user;
identifying a second gesture associated with the topography of the portion of the body of the user based on the second pressure data; and
transmitting information to the device based on the identified second gesture.
15. The method for controlling the device based on gestures of claim 14 , wherein:
the portion of the body of the user is a back of a hand of the user; and
the collecting comprises collecting pressure data using a pressure-sensitive sensor configured to collect the pressure data that indicates the topography of the back of the hand as affected by metacarpal position.
16. The method for controlling the device based on gestures of claim 14 , wherein:
the portion of the body of the user is a wrist of the user; and
the collecting comprises collecting pressure data using a pressure-sensitive sensor configured to collect the pressure data that indicates the topography of at least a portion of the wrist.
17. The method for controlling the device based on gestures of claim 14 , further comprising detecting motion of the user.
18. The method for controlling the device based on gestures of claim 17 , wherein identifying the second gesture is further based on the detected motion.
19. The method for controlling the device based on gestures of claim 17 , further comprising detecting the motion of the user with at least one of an accelerometer or a gyroscope.
20. The method for controlling the device based on gestures of claim 14 , wherein the portion of the body of the user comprises a forearm of the user.
21. The method for controlling the device based on gestures of claim 14 , wherein identifying the second gesture is further based on information from an electromyographic sensor.
22. The method for controlling the device based on gestures of claim 14 , further comprising collecting a third pressure data from a second portion of the body, wherein identifying the second gesture is further based on the third pressure data.
23. The method for controlling the device based on gestures of claim 14 , wherein the identified second gesture comprises a closed fist.
24. The method for controlling the device based on gestures of claim 14 , wherein the identified second gesture comprises a finger touching a thumb.
25. The method for controlling the device based on gestures of claim 14 , wherein the identified second gesture comprises a wrist being flexed such that the hand moves up, down, left, or right.
26. A non-transitory computer-readable medium having instructions embedded thereon for controlling a device based on gestures, the instructions including computer-executable code for:
collecting a first pressure data that indicates a topography of a portion of a body of a user;
identifying a first gesture associated with the topography of the portion of the body of the user based on the first pressure data;
causing a camera to capture an image of the device subsequent to identifying the first gesture;
identifying the device using the image;
collecting a second pressure data that indicates the topography of the portion of the body of the user;
identifying a second gesture associated with the topography of the portion of the body of the user based on the second pressure data; and
transmitting information to the device based on the identified second gesture.
27. A system for controlling a device based on gestures, the system comprising:
means for collecting a first pressure data that indicates a topography of a portion of a body of a user;
means for identifying a first gesture associated with the topography of the portion of the body of the user based on the first pressure data;
means for capturing an image of the device subsequent to identifying the first gesture;
means for identifying the device using the image;
means for collecting a second pressure data that indicates the topography of the portion of the body of the user;
means for identifying a second gesture associated with the topography of the portion of the body of the user based on the second pressure data; and
means for transmitting information to the device based on the identified second gesture.
28. The system for controlling a device based on gestures of claim 27 , wherein the means for identifying the second gesture includes means for identifying the second gesture associated with the topography of a back of a hand of the user.
29. The system for controlling a device based on gestures of claim 27 , wherein the means for identifying the second gesture includes means for identifying the second gesture associated with a wrist of the user.
30. The system for controlling a device based on gestures of claim 27 , further comprising:
means for detecting motion of the user; and
means for further basing the identifying the second gesture on the detected motion.
31. The system for controlling a device based on gestures of claim 27 , wherein the means for identifying the second gesture includes means for identifying the second gesture associated with the topography of a forearm of the user.
32. The system for controlling a device based on gestures of claim 27 , including means for further basing the identifying the second gesture on information from an electromyographic sensor.
33. The system for controlling a device based on gestures of claim 27 , further comprising means for collecting a third pressure data from a second portion of the body, wherein identifying the second gesture is further based on the third pressure data.
34. The system for controlling a device based on gestures of claim 27 , wherein the means for identifying the second gesture comprise means for identifying a closed fist.
35. The system for controlling a device based on gestures of claim 27 , wherein the means for identifying the second gesture comprise means for identifying a finger touching a thumb.
36. The system for controlling a device based on gestures of claim 27 , wherein the means for identifying the second gesture comprise means for identifying the flexing a wrist such that the hand moves up, down, left, or right.