Systems for interpreting thumb movements of in-air hand gestures for controlling user interfaces based on spatial orientations of a user's hand, and method of use thereof
Methods of interpreting in-air hand gestures based on orientations of a wrist-wearable device are provided. The method includes receiving, from sensors of a wrist-wearable device, data associated with performance of an in-air hand gesture during a first period of time. The method includes, when determining the data indicates that the wrist-wearable device had a first orientation when the in-air hand gesture was performed during the first period of time, causing performance of a first operation. The method includes receiving, from sensors of the wrist-wearable device, new data associated with performance of the in-air hand gesture during a second period of time, the second period of time being after the first period of time. The method includes, when determining that the new data indicates that the wrist-wearable device had a second orientation when the in-air hand gesture was performed during the second period of time, causing performance of a second operation.
1. A non-transitory computer-readable storage medium including instructions that, when executed by a wearable device, cause the wearable device to perform or cause performance of operations for:
receiving, from one or more sensors of a wrist-wearable device, data associated with performance of an in-air hand gesture during a first period of time by a wearer of the wrist-wearable device;
in accordance with determining that the data indicates that the wrist-wearable device had a first orientation when the in-air hand gesture was performed during the first period of time, causing performance of a first operation at the wrist-wearable device;
receiving, from the one or more sensors of the wrist-wearable device, new data associated with performance of the in-air hand gesture during a second period of time by the wearer of the wrist-wearable device, the second period of time being after the first period of time; and
in accordance with determining that the new data indicates that the wrist-wearable device had a second orientation when the in-air hand gesture was performed during the second period of time, causing performance of a second operation at the wrist-wearable device,
wherein:
the in-air hand gesture that was performed while the wrist-wearable device had the first orientation and the in-air hand gesture that was performed while the wrist-wearable device had the second orientation correspond to a same in-air hand gesture, and
the second operation is distinct from the first operation, and
the second orientation of the wrist-wearable device is distinct from the first orientation of the wrist-wearable device.
2. The non-transitory, computer-readable storage medium of claim 1 , further comprising instructions for:
receiving, from the one or more sensors of the wrist-wearable device, additional data associated with performance of the in-air hand gesture during a third period of time by the wearer of the wrist-wearable device, the third period of time being after the first period of time and the second period of time; and
in accordance with determining that the additional data indicates that the wrist-wearable device had a third orientation when the in-air hand gesture was performed during the third period of time, causing performance of a third operation at the wrist-wearable device, wherein:
the third operation is distinct from the first operation and second operation, and
the third orientation of the wrist-wearable device is distinct from the first orientation of the wrist-wearable device and the second orientation of the wrist-wearable device.
3. The non-transitory, computer-readable storage medium of claim 2 , wherein the first operation is an operation associated with use of a camera of the wrist-wearable device, and the second operation does not relate to use of the camera.
4. The non-transitory, computer-readable storage medium of claim 1 , wherein the in-air hand gesture is performed while the wrist-wearable device is displaying a clock-face user interface.
5. The non-transitory, computer-readable storage medium of claim 2 , wherein:
the first operation is an operation associated with use of a camera of the wrist-wearable device,
the second operation is an operation associated with providing a voice command, and
the third operation is an operation associated with use of a media player.
6. The non-transitory, computer-readable storage medium of claim 5 , wherein:
the wrist-wearable device is determined to be in the first orientation when sensor data from the wrist-wearable device indicates that the wrist-wearable device is orientated to take a picture of the wearer's face,
the wrist-wearable device is determined to be the second orientation when sensor data from the wrist-wearable device indicates that the wrist-wearable device is oriented below a chin level of a user, and
the wrist-wearable device is determined to be the third orientation when sensor data from the wrist-wearable device indicates that the wrist-wearable device is in an orientation that places the wrist-wearable device next to an ear of the user.
7. The non-transitory, computer-readable storage medium of claim 2 , further comprising instructions for:
receiving, from the one or more sensors of the wrist-wearable device, further additional data associated with performance of another in-air hand gesture, different from the in-air hand gesture, during a fourth period of time by the wearer of the wrist-wearable device; and
in accordance with determining that the further additional data indicates that the wrist-wearable device had the third orientation when the other in-air hand gesture was performed during the fourth period of time, causing performance of a fifth operation at the wrist-wearable device, different from the third operation.
8. The non-transitory, computer-readable storage medium of claim 1 , further comprising instructions for:
receiving, from the one or more sensors of the wrist-wearable device, additional data associated with performance of another in-air hand gesture, different from the in-air hand gesture, during a third period of time by the wearer of the wrist-wearable device; and
in accordance with determining that the additional data indicates that the wrist-wearable device had the first orientation when the other in-air hand gesture was performed during the third period of time, causing performance of a third operation at the wrist-wearable device, different from the first operation.
9. The non-transitory, computer-readable storage medium of claim 8 , further comprising instructions for:
receiving, from the one or more sensors of the wrist-wearable device, more data associated with performance of the other in-air hand gesture during a fourth period of time by the wearer of the wrist-wearable device; and
in accordance with determining that the more data indicates that the wrist-wearable device had the second orientation when the other in-air hand gesture was performed during the fourth period of time, causing performance of a fourth operation at the wrist-wearable device, different from the second operation.
10. The non-transitory, computer-readable storage medium of claim 1 , wherein the in-air hand gesture is at least partially detected by data sensed by one or more of an inertial measurement unit (IMU) and a neuromuscular-signal sensor.
11. The non-transitory, computer-readable storage medium of claim 1 , wherein the first operation is performed using a first application, and the second operation is performed using a second application, or
the first operation is performed after an operating system of the wrist-wearable device provides data to the first application concerning performance of the in-air hand gesture, and the second operation is performed after an operating system of the wrist-wearable device provides data to the second application concerning performance of the other in-air hand gesture.
12. The non-transitory, computer-readable storage medium of claim 1 , wherein the first operation and the second operation are controlled by an operating system of the wrist-wearable device.
13. The non-transitory, computer-readable storage medium of claim 1 , wherein the in-air hand gesture is performed while the wrist-wearable device is in a display-off state.
14. A wrist-wearable device, comprising:
a display;
one or more processors; and
memory comprising instructions that, when executed by the wrist-wearable device, cause performance of operations for:
receiving, from one or more sensors of a wrist-wearable device, data associated with performance of an in-air hand gesture during a first period of time by a wearer of the wrist-wearable device;
in accordance with determining that the data indicates that the wrist-wearable device had a first orientation when the in-air hand gesture was performed during the first period of time, causing performance of a first operation at the wrist-wearable device;
receiving, from the one or more sensors of the wrist-wearable device, new data associated with performance of the in-air hand gesture during a second period of time by the wearer of the wrist-wearable device, the second period of time being after the first period of time; and
in accordance with determining that the new data indicates that the wrist-wearable device had a second orientation when the in-air hand gesture was performed during the second period of time, causing performance of a second operation at the wrist-wearable device,
wherein:
the in-air hand gesture that was performed while the wrist-wearable device had the first orientation and the in-air hand gesture that was performed while the wrist-wearable device had the second orientation correspond to a same in-air hand gesture, and
the second operation is distinct from the first operation, and
the second orientation of the wrist-wearable device is distinct from the first orientation of the wrist-wearable device.
15. The wrist-wearable device of claim 14 , wherein the memory further comprises instructions for:
receiving, from the one or more sensors of the wrist-wearable device, additional data associated with performance of the in-air hand gesture during a third period of time by the wearer of the wrist-wearable device, the third period of time being after the first period of time and the second period of time; and
in accordance with determining that the additional data indicates that the wrist-wearable device had a third orientation when the in-air hand gesture was performed during the third period of time, causing performance of a third operation at the wrist-wearable device, wherein:
the third operation is distinct from the first operation and second operation, and
the third orientation of the wrist-wearable device is distinct from the first orientation of the wrist-wearable device and the second orientation of the wrist-wearable device.
16. A method of interpreting in-air hand gestures based on orientation of a wrist-wearable device, the method comprising:
receiving, from one or more sensors of a wrist-wearable device, data associated with performance of an in-air hand gesture during a first period of time by a wearer of the wrist-wearable device;
in accordance with determining that the data indicates that the wrist-wearable device had a first orientation when the in-air hand gesture was performed during the first period of time, causing performance of a first operation at the wrist-wearable device;
receiving, from the one or more sensors of the wrist-wearable device, new data associated with performance of the in-air hand gesture during a second period of time by the wearer of the wrist-wearable device, the second period of time being after the first period of time; and
in accordance with determining that the new data indicates that the wrist-wearable device had a second orientation when the in-air hand gesture was performed during the second period of time, causing performance of a second operation at the wrist-wearable device,
wherein:
the in-air hand gesture that was performed while the wrist-wearable device had the first orientation and the in-air hand gesture that was performed while the wrist-wearable device had the second orientation correspond to a same in-air hand gesture, and
the second operation is distinct from the first operation, and
the second orientation of the wrist-wearable device is distinct from the first orientation of the wrist-wearable device.
17. The method of claim 16 , further including:
receiving, from the one or more sensors of the wrist-wearable device, additional data associated with performance of the in-air hand gesture during a third period of time by the wearer of the wrist-wearable device, the third period of time being after the first period of time and the second period of time; and
in accordance with determining that the additional data indicates that the wrist-wearable device had a third orientation when the in-air hand gesture was performed during the third period of time, causing performance of a third operation at the wrist-wearable device, wherein:
the third operation is distinct from the first operation and second operation, and
the third orientation of the wrist-wearable device is distinct from the first orientation of the wrist-wearable device and the second orientation of the wrist-wearable device.
18. The method of claim 16 , further including:
receiving, from the one or more sensors of the wrist-wearable device, additional data associated with performance of another in-air hand gesture, different from the in-air hand gesture, during a third period of time by the wearer of the wrist-wearable device; and
in accordance with determining that the additional data indicates that the wrist-wearable device had the first orientation when the other in-air hand gesture was performed during the third period of time, causing performance of a third operation at the wrist-wearable device, different from the first operation.
19. The wrist-wearable device of claim 14 , further comprising instructions for:
receiving, from the one or more sensors of the wrist-wearable device, additional data associated with performance of another in-air hand gesture, different from the in-air hand gesture, during a third period of time by the wearer of the wrist-wearable device; and
in accordance with determining that the additional data indicates that the wrist-wearable device had the first orientation when the other in-air hand gesture was performed during the third period of time, causing performance of a third operation at the wrist-wearable device, different from the first operation.
20. The wrist-wearable device of claim 14 , wherein the first operation is an operation associated with use of a camera of the wrist-wearable device, and the second operation does not relate to use of the camera.