Brain-machine interface based intention determination device and method using virtual environment
Provided is a brain-machine interface based intention determination device using a virtual environment, including: an input unit that receives a control mode; a training output unit that outputs training information; a collection unit that collects a brain signal; a first preprocessing unit that extracts time-frequency information; a second preprocessing unit that generates physical information; a learning unit that learns control information according to a pattern of physical information; a determination unit that determines control information; and a command output unit that outputs a control command matching the control information.
1. A device configured to perform an intention determination process, the process comprising:
receiving an indication of a thought mode selected by a user input;
selecting content according to the indication of the thought mode;
displaying the content while collecting a first brain signal sensed from the user while the user is viewing the displayed content;
extracting a first filtered signal from the first brain signal using a frequency band filter selected based on the indication of the thought mode, wherein the thought mode is selected from predefined thought modes, and the frequency band filter is selected from a plurality of frequency band filters each associated with the predefined thought modes;
generating physical information representing a physical magnitude of the extracted first filtered signal by calculating a first power spectral density from the extracted first filtered signal;
determining control information according to the physical information based on a classification model generated by learning control information representing a user's intent according to a pattern in the physical information provided in advance;
outputting a control command when the control command matches the control information; and
when the control command does not match the control information, regenerating the physical information by calculating a second power spectral density based on a second brain signal having a time and frequency different from the first brain signal,
wherein the regenerated physical information is input to the classification model to generate matched control information.
2. The device according to claim 1 , wherein the thought mode includes:
an imagination mode classified according to a type of imagination;
a space mode representing a space provided in advance as a virtual environment; and
a motion mode representing a motion provided to be imagined by the user according to the imagination mode and the space mode.
3. The device according to claim 1 , wherein the thought mode is selected from a motor imagery mode, a visual imagery mode, and a speech imagery mode.
4. The device according to claim 3 , wherein the thought mode is the motor imagery mode, and
wherein the frequency band filter is configured to filter a frequency band from 8 Hz to 13 Hz.
5. The device according to claim 3 , wherein the thought mode is the visual imagery mode, and
wherein the frequency band filter is configured to filter a frequency band from 8 Hz to 12 Hz.
6. The device according to claim 3 , wherein the thought mode is the speech imagery mode, and
wherein the frequency band filter is configured to filter a frequency band from Hz to 15 Hz.
7. The device of claim 1 , wherein the predefined thought modes include a motor imagery mode and a speech imagery mode.
8. An intention determination method of determining a user's intention according to a brain-machine interface based intention determination device using a virtual environment, the intention determination method comprising:
receiving a control mode input from a user;
outputting training information set according to the control mode;
collecting a brain signal generated from an imagination of the user for the training information;
extracting frequency information according to a preset frequency band from the brain signal, and extracting time-frequency information of a preset time interval from the frequency information;
generating physical information representing a physical magnitude of the extracted time-frequency information by calculating a power spectral density from the extracted time-frequency information;
determining control information according to the physical information based on a classification model generated by learning control information indicating a user's intention according to a pattern in the physical information provided in advance;
outputting a control command when the control command matches the control information; and
when the control command does not match the control information, regenerating the physical information by calculating another power spectral density based on another brain signal having a time and frequency different from the brain signal,
wherein the regenerated physical information is input to the classification model to generate matched control information.
9. The intention determination method according to claim 8 , wherein the control mode further includes:
an imagination mode classified according to a type of imagination;
a space mode representing a space provided in advance as a virtual environment; and
a motion mode representing a motion provided to be imagined by the user according to the imagination mode and the space mode.
10. The intention determination method according to claim 8 , wherein, in the receiving of the control mode, at least one imagination mode among a motor imagery mode, a visual imagery mode, and a speech imagery mode is input.
11. The intention determination method according to claim 10 , wherein, in the extracting of the time-frequency information, when the imagination mode is the motor imagery mode, the frequency information is extracted from a frequency band of 8 Hz to 13 Hz.
12. The intention determination method according to claim 10 , wherein, in the extracting of the time-frequency information, when the imagination mode is the visual imagery mode, the frequency information is extracted from a frequency band of 8 Hz to 12 Hz.
13. The intention determination method according to claim 10 , wherein, in the extracting of the time-frequency information, when the imagination mode is the speech imagery mode, the frequency information is extracted from a frequency band of 10 Hz to 15 Hz.
14. The intention determination method according to claim 8 , wherein the intention determination device is configured to extract the time-frequency information of a time range of 5,000 ms from the extracted frequency information of 30,000 ms.
15. The intention determination method according to claim 8 , wherein the pattern of physical information is determined by a frequency band and a time range of any of the time-frequency information from which the physical information is generated, and a temporal position of the time-frequency information in the frequency information.
16. The intention determination method according to claim 8 , wherein the classification model is obtained by learning a pattern of physical information for a brain signal occurring by any of a visual scene and an action that makes a sound.