Wrist-worn device-based inputs for an operating system
A user interface device includes at least a biopotential sensor and a location sensor. The user interface device is constructed to be worn by a person. A responsive device, such as a computer, receives data based on the outputs from the biopotential sensor and the location sensor, and performs actions based on the received data. Primitive gestures detected in the received data are associated with inputs of an operating system of the responsive device. For a wrist-worn device, a primitive gesture can be based on a pose of the hand or location of a wrist or both. The association of primitive gestures with inputs of the operating system can include associating primitive gestures with mouse inputs, inputs to request launching an application or switching applications, or inputs to perform other operations.
1. A system comprising:
a wearable device configured to be worn at a wrist on a body of a person, the wearable device comprising:
a biopotential sensor, the biopotential sensor being configured to sense biopotentials at the wrist related to activity of muscles of a hand of the person, the biopotential sensor providing as an output a biopotential signal indicative of the sensed biopotentials; and
a location sensor, the location sensor being configured to sense a location of the wrist of the person, the location sensor providing as an output a location signal indicative of the sensed location;
the wearable device being configured to transmit data based on the output from the biopotential sensor and the output of the location sensor to a second device, wherein:
the second device comprises a processor, computer storage, one or more applications, and an operating system;
the second device comprises a locked state, an active state;
transmitting first data, from the wearable device, indicating that the person performed a wake word gesture causes a first instruction to be passed to the operating system of the second device, the first instruction to the operating system causing the second device to transition from the locked state to the active state; and
transmitting second data, from the wearable device, indicating that the person performed an application launch gesture causes a second instruction to be passed to the operating system of the second device, the second instruction to the operating system causing an application of the one or more applications to be launched on the second device; wherein the system further comprises:
a motion silencing state, the system being nonresponsive to the output of the biopotential sensor when in the motion silencing state; and
a biopotential sensing state, the system being responsive to the output of the biopotential sensor when in the biopotential sensing state;
wherein the system is configured to transition from the motion silencing state to the biopotential sensing state when the location sensor detects an absence of movement of the wrist relative to the body of the person.
2. The system of claim 1 , wherein the wake word gesture comprises moving the body part in a first direction and then moving the body part in a second direction opposite the first.
3. The system of claim 1 , wherein the wake word gesture comprises shaking the wrist back and forth.
4. The system of claim 1 , wherein the wake word gesture comprises rotating a hand back and forth.
5. The system of claim 1 , wherein the wearable device comprises:
a sleep state in which the wearable device is configured to transmit data based solely on sensors that are configured to detect the wake word gesture;
an awakened state in which the wearable device is configured to transmit data based on sensors that are not necessary to detect the wake word gesture; and
the wearable device is configured to transition from the sleep state to the awakened state when the person performs the wake word gesture.
6. The system of claim 1 , wherein the system further comprises a lift-and-hold state, the system being configured to enter the lift-and-hold state when the output from the biopotential sensor indicates an intent by the person to maintain an index finger in an extended position.
7. The system of claim 6 , wherein the system is further configured such that when the system is in the lift-and-hold state, a motion of the person's wrist, as detected based on the output from the location sensor, causes an instruction to be passed to the operating system of the second device to perform a click-and-drag operation.
8. The system of claim 6 , wherein the system is further configured such that when the system is in the lift-and-hold state, upon detecting, based on the output from the biopotential sensor, that the person intends to release the index finger from the extended position, the system is configured to transition back to the motion silencing state.
9. The system of claim 1 , wherein the system is further configured such that, when the launched application is active on the second device, the system is configured to pass a third instruction based on the output from the biopotential sensor and the location sensor of the wearable device, the third instruction causing the launched application to perform an action.
10. The system of claim 9 , wherein the system is further configured such that, when the launched application is active on the second device, the system is configured to pass a fourth instruction based on the output from the biopotential sensor and the location sensor of the wearable device to the operating system of the second device, the fourth instruction causing a second application of the one or more applications to be launched.
11. A method for biopotential-based control, the method being performed using system comprising a wearable device that is worn at a wrist on a body of a person and comprises a biopotential sensor and a location sensor, the method comprising:
sensing biopotentials at the wrist, using the biopotential sensor, the sensed biopotentials being related to activity of muscles of a hand of the person, the biopotential sensor providing as an output a biopotential signal indicative of the sensed biopotentials;
sensing a location of the wrist of the person, using the location sensor, the location sensor providing as an output a location signal indicative of the location of the wrist of the person;
transmitting first data based on the output from the biopotential sensor and the output of the location sensor to a second device, the second device comprising a processor, computer storage, one or more applications, an operating system, a locked state, and an active state;
determining, based on the first data, that the person performed a wake word gesture;
in response to determining that the person performed the wake word gesture, passing a first instruction to the operating system of the second device, the first instruction to the operating system causing the second device to transition from the locked state to the active state;
transmitting second data based on the output from the biopotential sensor and the output of the location sensor to the second device;
determining, based on the second data, that the person performed an application launch gesture;
in response to determining that the person performed the application launch gesture, passing a second instruction to the operating system of the second device, the second instruction to the operating system causing an application of the one or more applications to be launched on the second device;
wherein the system further comprises a motion silencing state, the system being nonresponsive to the output of the biopotential sensor when in the motion silencing state; and
a biopotential sensing state, the system being responsive to the output of the biopotential sensor when in the biopotential sensing state; the method further comprising:
transitioning from the motion silencing state to the biopotential sensing state when the location sensor detects an absence of movement of the wrist.
12. The system of claim 11 , wherein the wake word gesture comprises moving the body part in a first direction and then moving the body part in a second direction opposite the first.
13. The system of claim 11 , wherein the wake word gesture comprises shaking the wrist back and forth.
14. The system of claim 11 , wherein the wake word gesture comprises rotating a hand back and forth.
15. The system of claim 11 , wherein the wearable device comprises:
a sleep state in which the wearable device is configured to transmit data based solely on sensors that are configured to detect the wake word gesture;
an awakened state in which the wearable device is configured to transmit data based on sensors that are not necessary to detect the wake word gesture; and
the wearable device is configured to transition from the sleep state to the awakened state when the person performs the wake word gesture.
16. The system of claim 11 , wherein the system further comprises a lift-and-hold state, the system being configured to enter the lift-and-hold state when the output from the biopotential sensor indicates an intent by the person to maintain an index finger in an extended position.
17. The system of claim 11 , wherein the system is further configured such that, when the launched application is active on the second device, the system is configured to pass a third instruction based on the output from the biopotential sensor and the location sensor of the wearable device, the third instruction causing the launched application to perform an action.
18. The system of claim 17 , wherein the system is further configured such that, when the launched application is active on the second device, the system is configured to pass a fourth instruction based on the output from the biopotential sensor and the location sensor of the wearable device to the operating system of the second device, the fourth instruction causing a second application of the one or more applications to be launched.