Real-time spike detection and identification
Methods and apparatus for substantially real-time detection of spike events in neuromuscular data. The method comprises receiving a plurality of neuromuscular signals from a plurality of neuromuscular sensors arranged on one or more wearable devices worn by a user, detecting, based on the plurality of neuromuscular signals or information derived from the plurality of neuromuscular signals, at least one spike event corresponding to firing of an action potential in at least one motor unit, determining, based on the plurality of neuromuscular signals or the information derived from the plurality of neuromuscular signals, a biological source of the detected at least one spike event, and generating at least one output based, at least in part, on the detected at least one spike event and/or the determined biological source of the detected at least one spike event.
1. A computerized system, comprising:
a plurality of neuromuscular sensors configured to record a plurality of neuromuscular signals that arise from the central nervous system of a user, wherein the plurality of neuromuscular sensors are arranged on one or more wearable devices; and
at least one computer processor programmed to:
detect, based on the plurality of neuromuscular signals or information derived from the plurality of neuromuscular signals, at least one spike event corresponding to firing of an action potential in at least one motor unit;
determine, based on the plurality of neuromuscular signals or the information derived from the plurality of neuromuscular signals, a biological source of the detected at least one spike event; and
generate, in substantially real-time, at least one control signal for operation of at least one device,
wherein the at least one control signal is based, at least in part, on the detected at least one spike event; and
the at least one computer processor is further programmed to detect the at least one spike event using an iterative process comprising:
subtracting a detected spike event waveform from at least one of the plurality of neuromuscular signals to generate a residual neuromuscular signal;
attempting to detect additional spike events in the residual neuromuscular signal; and
stopping the iterative detection process when no additional spike events are detected in the residual neuromuscular signal.
2. The computerized system of claim 1 , wherein the at least one computer processor is further programmed to:
apply one or more criteria for selecting a biological source from a plurality of biological sources associated with respective detected spike events; and
select at least one spike event associated with the biological source.
3. The computerized system of claim 2 , wherein the at least one computer processor is further programmed to perform the iterative process by processing the detected spike events until none of the biological sources are present within the residual signal that meets a minimum threshold for selection as the biological source for spike events.
4. The computerized system of claim 1 , wherein the at least one computer processor is further programmed to group the detected spike events, including the detected at least one spike event, into a muscle-specific group based on co-activations and sort spike events within the muscle-specific group to approximate a recruitment curve.
5. The computerized system of claim 1 , wherein the at least one computer processor is further programmed to apply at least one filter to a time-lagged representation of the plurality of neuromuscular signals, and
wherein the detecting the at least one spike event and the determining the biological source of the detected at least one spike event is performed based on the filtered time-lagged representation of the plurality of neuromuscular signals.
6. The computerized system of claim 5 , wherein the applying the at least one filter to the time-lagged representation of the plurality of neuromuscular sensors comprises using a beamforming process to apply a plurality of beamforming filters to the time-lagged representation of the plurality of neuromuscular signals, wherein the plurality of beamforming filters are filters generated based on spatiotemporal patterns of one or more spike events.
7. The computerized system of claim 6 , wherein the beamforming process comprises using at least one of a minimum variance distortionless response technique and a linear constrained minimum variance technique.
8. The computerized system of claim 6 , wherein the at least one computer processor is further programmed to determine the spatiotemporal patterns of the one or more spike events corresponding to the plurality of beamforming filters.
9. The computerized system of claim 8 , wherein the determining the spatiotemporal patterns of the one or more spike events corresponding to the plurality of beamforming filters comprises:
detecting a plurality of spike events in the recorded neuromuscular signals;
clustering the detected plurality of spike events; and
determining the spatiotemporal patterns based on the clusters of spike events.
10. The computerized system of claim 8 , wherein the detecting a plurality of spike events comprises:
detecting within the plurality of neuromuscular signals, periods of low activity; and
detecting within the period of low activity, putative spike events.
11. The computerized system of claim 10 , wherein the detecting the plurality of spike events further comprises analyzing the detected putative spike events to discard spike events having one or more particular characteristics, and wherein the one or more particular characteristics include a duration longer than a particular threshold duration.
12. The computerized system of claim 5 , wherein the at least one computer processor is further programmed to threshold the filtered time-lagged representation of the plurality of neuromuscular signals to detect the at least one spike event.
13. The computerized system of claim 1 , wherein the at least one computer processor is further programmed to detect the at least one spike event and/or determine the biological source of the detected at least one spike event using one or more neural networks, comprising at least one of a convolutional neural network and a recurrent neural network.
14. The computerized system of claim 1 , wherein the at least one computer processor is further programmed to detect the at least one spike event and determine the biological source of the detected at least one spike event using a multi-step iterative technique to decompose a time-lagged representation of the plurality of neuromuscular signals into signal components corresponding to at least one biological source, and detecting the at least one spike event from the at least one biological source.
15. The computerized system of claim 1 , wherein the generating the at least one control signal comprises generating compressed data including an indication of the at least one spike event, wherein the indication of the at least one spike event is provided as the at least one control signal to the at least one device, wherein the at least one device comprises a computer-based system, and wherein the indication of the at least one spike event is provided as at least one of a group comprising:
a discrete control signal;
a continuous control signal; and
a composite control signal.
16. The computerized system of claim 1 , wherein the generating the at least one control signal comprises generating an indication of the at least one spike event, and wherein the indication of the at least one spike event includes an indication of the biological source of the at least one spike event and a time of occurrence of the at least one spike event.
17. The computerized system of claim 1 , wherein the at least one computer processor is programmed to provide feedback to the user responsive to the at least one control signal based, at least in part, on the detected at least one spike event and/or the determined biological source of the detected at least one spike event.
18. The computerized system of claim 1 , further comprising an inertial sensor configured to determine movement artifacts or shifts in spatial location of muscle fibers of the at least one motor unit relative to one or more of the plurality of neuromuscular sensors.
19. The computerized system of claim 1 , wherein the detecting the at least one spike event comprises detecting a spatiotemporal pattern of the at least one spike event, and
wherein generating the at least one control signal comprises the generating the at least one control signal based, at least in part, on at least one characteristic of the detected spatiotemporal pattern of the at least one spike event while the plurality of neuromuscular signals are recorded.
20. The computerized system of claim 19 , wherein the at least one characteristic comprises a rate of the at least one spike event and/or a spatial distribution of the detected spatiotemporal pattern of the at least one spike event.
21. The computerized system of claim 1 , wherein the determined biological source comprises at least one of a group comprising:
a motor unit;
a plurality of motor units;
a muscle; and
a plurality of muscles.
22. The computerized system of claim 1 , wherein the determining the biological source of the at least one spike event comprises determining that the at least one spike event is associated with a motor unit or group of motor units,
wherein the at least one computer processor is further programmed to determine a muscle to which the motor unit or group of motor units belongs, and
the generating the at least one control signal comprises generating the at least one control signal based on the determined muscle to which the motor unit or group of motor units belongs.
23. The computerized system of claim 22 , wherein the determined muscle is associated with a motor unit recruitment sequence describing a sequence of activation of motor units for the determined muscle, and
wherein the at least one computer processor is further programmed to determine where the motor unit or group of motor units fall within the motor unit recruitment sequence of the determined muscle.
24. The computerized system of claim 1 , further comprising:
at least one auxiliary sensor configured to record a plurality of auxiliary signals, and
wherein the at least one computer processor is further programmed to generate the at least one control signal based, at least in part, on the plurality of auxiliary signals.
25. The computerized system of claim 24 , wherein the at least one computer processor is programmed to provide feedback to the user as part of a user training process, and wherein the feedback includes at least one of a group comprising auditory, visual, haptic, and multi-sensory feedback.
26. The computerized system of claim 24 , wherein the at least one auxiliary sensor comprises at least one inertial measurement unit (IMU) sensor configured to record a plurality of IMU signals, and
wherein the at least one computer processor is further programmed to generate the at least one control signal based, at least in part, on the plurality of IMU signals and/or information derived from the plurality of IMU signals.
27. The computerized system of claim 24 , wherein the at least one auxiliary sensor comprises at least one camera configured to record one or more images, and
wherein the at least one computer processor is further programmed to generate the at least one control signal based, at least in part, on the one or more images and/or information derived from the one or more images.
28. The computerized system of claim 1 , wherein the at least one computer processor comprises at least one first computer processor included as a portion of an element separate from and in communication with the plurality of neuromuscular sensors arranged on the one or more wearable devices and at least one second computer processor integrated with the one or more wearable devices on which the plurality of neuromuscular sensors are arranged, and wherein the plurality of neuromuscular sensors are configured to transmit at least some of the plurality of neuromuscular signals to the at least one first computer processor,
wherein the at least one first computer processor is programmed to:
train, based on the at least some of the plurality of neuromuscular signals transmitted from the plurality of neuromuscular sensors, at least one spike detection model and/or at least one spike identification model; and
transmit the trained at least one spike detection model and/or the at least one spike identification model to the at least one second computer processor, and
wherein the at least one second computer processor is programmed to detect the at least one spike event and determine the biological source of the detected at least one spike event using the at least one spike detection model and/or the at least one spike identification model transmitted from the at least one first computer processor.
29. The computerized system of claim 28 , wherein the at least one spike detection model and/or the least one spike identification model are trained to estimate at least one of a group comprising:
whether the user is activating a particular motor unit;
whether the user is activating a particular motor unit with a particular timing; and
whether the user is activating a particular combination of motor units.
30. The computerized system of claim 1 , wherein the detecting the at least one spike event corresponding to the firing of the action potential in the at least one motor unit comprises detecting at least one spike event corresponding to firing of an action potential in a plurality of motor units.