Meeting brain-computer interface user performance expectations using a deep neural network decoding framework

1 . A brain-computer interface (BCI) comprising:

a multichannel stimulator operatively connected to deliver stimulation pulses to a functional electrical stimulation (FES) device to control delivery of FES to an anatomical region; and

a decoder operatively connected to receive at least one neural signal from at least one electrode operatively connected with a motor cortex and to control the multichannel stimulator based on the received at least one neural signal from the at least one electrode operatively connected with the motor cortex to cause the anatomical region to perform a predicted movement;

wherein the decoder comprises a computer programmed to process the received at least one neural signal from the at least one electrode operatively connected with the motor cortex using a deep neural network including an output neural network layer having units corresponding to a set of possible intended movements plus rest, wherein the deep neural network comprises a long short-term memory (LSTM) layer outputting to a convolutional layer in turn outputting to at least one fully connected neural network layer, and the convolutional layer is a one-dimensional convolutional layer that performs convolution in the time domain only, and wherein the unit with highest probability at a given time is the predicted movement or predicted rest for that given time.

2 . The BCI of claim 1 wherein the LSTM layer includes hidden units that use a hard-sigmoid activation function and an output of the LSTM layer uses a hyperbolic tangent activation.

3 . The BCI of claim 1 , wherein the at least one fully connected neural network layer includes a fully connected neural network layer that uses a rectified linear unit activation function.

4 . The BCI of claim 3 wherein the fully connected neural network layer outputs to the output neural network layer.

5 . The BCI of claim 4 wherein the output neural network layer uses a softmax activation function scaling the outputs to correspond to probabilities.

6 . The BCI of claim 1 , wherein:

wherein an activation function for an output of the convolutional layer is a leaky-rectified linear unit; or

wherein the at least one fully connected neural network layer includes a fully connected neural network layer that uses a rectified linear unit activation function.

7 . The BCI of claim 6 , wherein the fully connected neural network layer outputs to the output neural network layer.

8 . The BCI of claim 7 wherein the output neural network layer uses a softmax activation function scaling the outputs to correspond to probabilities.

9 . The BCI of claim 1 wherein the unit of the output neural network layer of the decoder corresponding to rest implements a no-information decoder.

10 . The BCI of claim 9 wherein the computer is programmed to process the received at least one neural signal also not using a boxcar filter.

11 . An assistance device for assisting a patient having a spinal cord injury to manipulate an anatomical region that is paralyzed due to the spinal cord injury, the assistance device comprising:

a BCI as set forth in claim 1 ; and

a functional electrical stimulation (FES) device disposed on the anatomical region and including electrodes configured to deliver FES to the anatomical region;

wherein the multichannel stimulator of the BCI is operatively connected to deliver stimulation pulses to the FES device to control delivery of FES to the anatomical region.

12 . An assistance method for assisting a patient having a spinal cord injury to manipulate an anatomical region of the patient that is paralyzed due to the spinal cord injury, the assistance method comprising:

applying a decoder to at least one neural signal received from at least one electrode operatively connected with a motor cortex of the patient to generate a predicted movement of the anatomical region, wherein the decoder comprises a computer programmed to process the at least one neural signal from the at least one electrode operatively connected with the motor cortex of the patient using a deep neural network to generate the predicted movement of the anatomical region by selecting the predicted movement from a set of possible intended movements plus rest, wherein the deep neural network comprises a long short-term memory (LSTM) layer outputting to a one-dimensional convolutional layer that performs convolution in the time domain only, in turn outputting to at least one fully connected neural network layer; and

performing functional electrical stimulation (FES) to cause the predicted movement of the anatomical region using a multichannel stimulator that is operatively connected to deliver stimulation pulses to a functional electrical stimulation (FES) device operatively connected to the anatomical region.

13 . An assistance system comprising:

a brain-computer interface (BCI) comprising a computer operatively connected to receive at least one neural signal from a motor cortex of a patient and programmed to apply a decoder to the at least one neural signal from the motor cortex of the patient to generate a predicted movement, wherein the decoder comprises a deep neural network including an output neural network layer having units corresponding to a set of possible intended movements, wherein the unit with highest probability at a given time is the predicted movement for that given time; and

an assistive device, the BCI operatively connected to control the assistive device to perform the predictive movement;

wherein:

the deep neural network comprises a long short-term memory (LSTM) layer outputting to a convolutional layer in turn outputting to at least one fully connected neural network layer; and

the convolutional layer is a one-dimensional convolutional layer that performs convolution in the time domain only.

14 . The assistance system of claim 13 wherein the assistive device comprises a functional electrical stimulation (FES) device, a robotic arm, a computer cursor, or a communication device.

15 . The BCI of claim 1 wherein the one-dimensional convolutional layer performs convolution in the time domain only; or

wherein an activation function for an output of the one-dimensional convolutional layer is a leaky-rectified linear unit.

16 . The assistance device of claim 11 further comprising:

at least one electrode operatively connected with a motor cortex of the patient;

wherein the decoder is operatively connected to receive the at least one neural signal from the at least one electrode operatively connected with the motor cortex of the patient.

17 . The BCI of claim 1 wherein rest comprises the anatomical region remaining in a neutral resting position.