METHODS AND APPARATUS FOR PREDICTING MUSCULO-SKELETAL POSITION INFORMATION USING WEARABLE AUTONOMOUS SENSORS
Methods and apparatus for providing a dynamically-updated computerized musculo-skeletal representation comprising a plurality of rigid body segments connected by joints. The method comprises recording, using a plurality of autonomous sensors arranged on one or more wearable devices, a plurality of autonomous signals from a user, wherein the plurality of autonomous sensors include a plurality of neuromuscular sensors configured to record neuromuscular signals. The method further comprises providing as input to a trained statistical model, the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals. The method further comprises determining, based on an output of the trained statistical model, musculo-skeletal position information describing a spatial relationship between two or more connected segments of the plurality of rigid body segments of the computerized musculo-skeletal representation, and updating the computerized musculo-skeletal representation based, at least in part, on the musculo-skeletal position information.
1 . A system, comprising:
a wearable device, comprising a plurality of sensors arranged at different locations on the wearable device, each sensor configured to measure a plurality of electrical signals from a wrist or arm of a user; and
at least one computer processor programmed to:
provide as input to a machine learning model, the plurality of electrical signals from the wrist or arm of the user;
determine, based on an output of the machine learning model, musculoskeletal position information that estimates a position of a hand of the user.
2 . The system according to claim 1 , wherein the plurality of sensors includes one or more autonomous sensors arranged on the wearable device.
3 . The system according to claim 1 , wherein the system determines a computerized musculoskeletal representation of a hand of the user.
4 . The system according to claim 3 , wherein the musculoskeletal position information describes a spatial relationship between two or more connected segments of a plurality of rigid body segments of the computerized representation of the hand of the user.
5 . The system according to claim 1 , wherein the at least one computer processor is located within the wearable device.
6 . The system according to claim 1 , wherein each sensor comprises an electrode located Between a surface of the wearable device and the wrist or arm of the user.
7 . The system according to claim 1 , wherein the electrical signals are detected using electrodes configured to detect electric potentials on a surface of the body of the user.
8 . The system according to claim 1 , further comprising an element that is configured to determine a musculoskeletal representation comprising a plurality of rigid body segments connected by joints that represents the hand, wrist, and arm position of the user.
9 . The system according to claim 1 , further comprising a display controller configured to send a plurality of signals to a computer-based device, wherein the plurality of signals cause the computer-based device to display a graphical representation of the output that describes the hand position .
10 . The system of claim 1 , wherein the machine learning model is configured to derive an indication of force from the received values of the electrical signals.
11 . The system of claim 1 , wherein the machine learning model is configured to use information derived from the electrical signals to compute the output, the information comprising one or more of a frequency-domain representation of the electrical signals and a time-domain representation of the electrical signals.
12 . The system of claim 1 , wherein the computer processor is further configured to extract features from information derived from the electrical signals, the features comprising angles between joints defining the hand position of the user.
13 . The system of claim 1 , wherein the wearable device further comprises an inertial measurement unit configured to generate inertial signals corresponding to movement of the wearable device and the user's arm, wherein the computer processor is further configured to determine, using information derived from the inertial signals with the machine learning model, the output that describes the hand position.
14 . The system of claim 1 , wherein the at least one processor is further programmed to send based, at least in part, on the musculoskeletal position information, one or more control signals to a controller configured to instruct a device to perform an action based on the one or more control signals.
15 . The system of claim 1 , wherein the computed output further describes forces exerted by at least a portion of the hand of the user on one or more entities or objects.
16 . A method comprising acts of:
providing for a wearable device, the wearable device comprising a plurality of sensors arranged at different locations on the wearable device, each sensor configured to measure a plurality of electrical signals from a wrist or arm of a user;
providing, as input to a machine learning model, the plurality of electrical signals from the wrist or arm of the user; and
determining, based on an output of the machine learning model, musculoskeletal position information that estimates a position of a hand of the user.
17 . The method according to claim 16 , further comprising an act of providing for, on the wearable device, one or more autonomous sensors.
18 . The method according to claim 16 , further comprising an act of determining a computerized musculoskeletal representation of a hand of the user.
19 . The method according to claim 18 , wherein the musculoskeletal position information describes a spatial relationship between two or more connected segments of a plurality of rigid body segments of the computerized representation of the hand of the user.
20 . The method according to claim 16 , further comprising executing the machine learning model within the wearable device.
21 . The method according to claim 16 , wherein each sensor comprises an electrode, and wherein the method comprises locating the electrode between a surface of the wearable device and the wrist or arm of the user.
22 . The method according to claim 16 , further comprising an act of detecting the electrical signals using electrodes configured to detect electric potentials on a surface of the body of the user.
23 . The method according to claim 16 , further comprising an act of determining a musculoskeletal representation comprising a plurality of rigid body segments connected by joints that represents the hand, wrist, and arm position of the user.
24 . The method according to claim 16 , further comprising an act of sending a plurality of signals to a computer-based device, wherein the plurality of signals are used for controlling the computer-based device to display a graphical representation of the output that describes the hand position .
25 . The method according to claim 16 , further comprising an act of deriving, by the machine learning model, an indication of force from the received values of the electrical signals.
26 . The method according to claim 16 , further comprising using, by the machine learning model, information derived from the electrical signals to compute the output, the information comprising one or more of a frequency-domain representation of the electrical signals and a time-domain representation of the electrical signals.
27 . The method according to claim 16 , further comprising an act of extracting features from information derived from the electrical signals, the features comprising angles between joints defining the hand position of the user.
28 . The method according to claim 16 , wherein the wearable device further comprises an inertial measurement unit configured to generate inertial signals corresponding to movement of the wearable device and the user's arm, and wherein the method further comprises an act of determining, using information derived from the inertial signals with the machine learning model, the output that describes the hand position.
29 . The method according to claim 16 , further comprising an act of sending based, at least in part, on the musculoskeletal position information, one or more control signals to a controller configured to instruct a device to perform an action based on the one or more control signals.
30 . The method according to claim 16 , further comprising an act of providing an output describing forces exerted by at least a portion of the hand of the user on one or more entities or objects.