Low-power hand-tracking system for wearable device
A method for a low-power hand-tracking system is described. In one aspect, a method includes polling a proximity sensor of a wearable device to detect a proximity event, the wearable device includes a low-power processor and a high-power processor, in response to detecting the proximity event, operating a low-power hand-tracking application on the low-power processor based on proximity data from the proximity sensor, and ending an operation of the low-power hand-tracking application in response to at least one of: detecting and recognizing a gesture based on the proximity data, detecting without recognizing the gesture based on the proximity data, or detecting a lack of activity from the proximity sensor within a timeout period based on the proximity data.
1 . A method comprising:
operating a low-power gesture recognition application executed on a low-power processor of a wearable device;
determining that the low-power gesture recognition application fails to recognize a gesture; and
in response to determining that that the low-power gesture recognition application fails to recognize the gesture, activating a high-power gesture recognition application executed on a high-power processor of the wearable device.
2 . The method of claim 1 , wherein operating the low-power gesture recognition application further comprises:
executing a gesture detection and recognition routine using the low-power processor.
3 . The method of claim 1 , further comprising:
detecting a proximity event based on proximity sensor data from a proximity sensor of the wearable device,
wherein operating the low-power gesture recognition application is in response to detecting the proximity event.
4 . The method of claim 1 , further comprising:
generating a level of confidence of the gesture based on a gesture dictionary, using a gesture recognition algorithm operating on the low-power processor; and
identifying the gesture in response to the level of confidence exceeding a preset threshold.
5 . The method of claim 1 , further comprising:
identifying an operation corresponding to the gesture based on the high-power gesture recognition application; and
requesting the high-power processor to perform the operation.
6 . The method of claim 1 , further comprising:
identifying an operation of a mixed reality application corresponding to the gesture; and
requesting the high-power processor to perform the operation of the mixed reality application.
7 . The method of claim 1 , further comprising:
returning the wearable device to an idle state that polls a proximity sensor of the wearable device at a regular interval after the high-power gesture recognition application recognizes the gesture.
8 . The method of claim 1 , wherein the low-power processor comprises a microcontroller, wherein the high-power processor comprises at least one system-on-chip (SoC).
9 . The method of claim 1 , wherein the low-power processor is configured to only operate a proximity sensor of the wearable device and the low-power gesture recognition application, wherein the high-power processor is configured to operate all sensors of the wearable device.
10 . The method of claim 1 , wherein the low-power gesture recognition application uses a neural network to detect and recognize a hand gesture of a user of the wearable device based on proximity data from a proximity sensor of the wearable device, the neural network being configured to recognize a first set of hand gestures, and
wherein the high-power processor comprises a high-power hand-tracking application configured to recognize a second set of hand gestures using camera data from a higher resolution camera of the wearable device, the second set of hand gestures being larger than the first set of hand gestures.
11 . A wearable device comprising:
a low-power processor;
a high-power processor; and
a memory storing instructions that, when executed by one of the low-power processor or the high-power processor, configure the wearable device to perform operations comprising:
operating a low-power gesture recognition application executed on the low-power processor;
determining that the low-power gesture recognition application fails to recognize a gesture; and
in response to determining that that the low-power gesture recognition application fails to recognize the gesture, activating a high-power gesture recognition application executed on the high-power processor.
12 . The wearable device of claim 11 , wherein operating the low-power gesture recognition application further comprises:
executing a gesture detection and recognition routine using the low-power processor.
13 . The wearable device of claim 11 , wherein the operations further comprise:
detecting a proximity event based on proximity sensor data from a proximity sensor of the wearable device,
wherein operating the low-power gesture recognition application is in response to detecting the proximity event.
14 . The wearable device of claim 11 , wherein the operations further comprise:
generating a level of confidence of the gesture based on a gesture dictionary, using a gesture recognition algorithm operating on the low-power processor; and
identifying the gesture in response to the level of confidence exceeding a preset threshold.
15 . The wearable device of claim 11 , wherein the operations further comprise:
identifying an operation corresponding to the gesture based on the high-power gesture recognition application; and
requesting the high-power processor to perform the operation.
16 . The wearable device of claim 11 , wherein the operations further comprise:
identifying an operation of a mixed reality application corresponding to the gesture; and
requesting the high-power processor to perform the operation of the mixed reality application.
17 . The wearable device of claim 11 , wherein the operations further comprise:
returning the wearable device to an idle state that polls a proximity sensor of the wearable device at a regular interval after the high-power gesture recognition application recognizes the gesture.
18 . The wearable device of claim 11 , wherein the low-power processor comprises a microcontroller, wherein the high-power processor comprises at least one system-on-chip (SoC).
19 . The wearable device of claim 11 , wherein the low-power processor is configured to only operate a proximity sensor of the wearable device and the low-power gesture recognition application, wherein the high-power processor is configured to operate all sensors of the wearable device.
20 . A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
operating a low-power gesture recognition application executed on a low-power processor of a wearable device;
determining that the low-power gesture recognition application fails to recognize a gesture; and
in response to determining that that the low-power gesture recognition application fails to recognize the gesture, activating a high-power gesture recognition application executed on a high-power processor of the wearable device.