Brain computer interface running a trained associative model applying multiway regression to simulate electrocorticography signal features from sensed EEG signals, and corresponding method
Brain computer interface BCI comprising an input adapted to be connected to at least one electroencephalography EEG sensor to receive EEG signals, the BCI further comprising a processor running an associative model trained to simulate electrocorticography ECoG signal features from EEG signals received via the input, the BCI comprising an output to transmit the simulated ECoG signal features.
1. Brain computer interface (“BCI”) comprising an input connected to at least one electroencephalography (“EEG”) sensor and receiving EEG signals generated by the at least one EEG sensor, the BCI further comprising a processor running an associative model applying a multiway regression approach trained to simulate electrocorticography (“ECOG”) signal features from the received EEG signals, the BCI comprising an output to transmit the simulated ECOG signal features.
2. The BCI according to claim 1 , wherein the processor running the associative model is configured to operate in two stages, wherein
in a first stage of the two stages, features in a first frequency band of a simulated ECOG signal are simulated based on the received EEG signals, wherein the first frequency band corresponds to a frequency band of the received EEG signals, wherein the simulated ECoG signal is simulated based on the received EEG signals;
in a second stage of the two stages, features in a frequency band of the simulated ECOG signal with the highest frequency are simulated based on the received EEG signals.
3. The BCI according to claim 2 , wherein in the second stage a frequency band of the simulated ECOG signal with the highest frequency is simulated based on the received EEG signals indirectly based on the first stage features in the first frequency band of the simulated ECOG signal.
4. The BCI according to claim 2 , wherein the frequency band of the simulated ECoG signal with the highest frequency comprises a low gamma sub-band and a high gamma sub-band and wherein processor is further configured, in the second stage, to separately perform feature simulation in a low gamma sub-band and in a high gamma sub-band.
5. The BCI according to claim 1 , wherein the processor running the associative model comprises a training state and an operational state, wherein at least in the training state, the BCI comprises a further input connected to at least one ECoG sensor and receiving ECOG signals generated by the at least one ECoG sensor such that via the input and the further input, the received EEG signals and the received ECOG signals can be simultaneously received, and wherein the processor is adapted to train the associative model by feeding the associative model with the simultaneously received EEG and ECOG signals.
6. The BCI according to claim 5 , wherein the training state comprises three training stages, a first training stage of the three training stages relating to perceived speech, a second training stage of the three training stages relating to performed speech, a third training stage of the three training stages relating to inner speech, the received EEG signals being fed into the associative model at least for each of these stages separately.
7. The BCI according to claim 1 , wherein the associative model comprises a block term tensor regression BTTR scheme.
8. The BCI according to claim 1 , further comprising an ECOG signal feature decoder connected to said output to decode the simulated ECOG signal features.
9. The BCI according to claim 8 , operationally connected to a vocoder located downstream of the decoder to transform the decoded ECOG signal features into speech.
10. A method for processing brain activity, the method comprising the steps:
receiving electroencephalography (“EEG”) signals via an input of a brain computer interface (“BCI”), the EEG signals being generated by at least one EEG sensor connected to the input;
running an associative model applying a multiway regression approach trained to simulate electrocorticography (“ECOG”) signal features from the received EEG signals via a processor of the BCI; and
transmitting the simulated ECOG signal features via an output of the BCI.
11. The method according to claim 10 , wherein the step of running the associative model comprises a training state and an operational state, wherein at least in the training state, the BCI comprises a further input connected to at least one ECoG sensor and receiving ECOG signals generated by the at least one ECoG sensor such that via the input and the further input, the received EEG signals and the received ECOG signals can be simultaneously received, and wherein the method comprises training the associative model by feeding the model with the simultaneously received EEG and ECOG signals.
12. The method according to claim 11 , wherein the step of training the associative model comprises:
training the associative model with first signals relating to perceived speech;
training the associative model with second signals relating to performed speech; and
training the associative model with third signals relating to inner speech; and
wherein the first, second and third signals are fed into the model separately.