Image classification by brain computer interface
A method of classifying an image is disclosed. The method comprises: applying a computer vision procedure to the image to detect therein candidate image regions suspected as being occupied by a target; presenting to an observer each candidate image region as a visual stimulus, while collecting neurophysiological signals from a brain of the observer; processing the neurophysiological signals to identify a neurophysiological event indicative of a detection of the target by the observer; and determining an existence of the target in the image is based, at least in part, on the identification of the neurophysiological event.
1. A method of classifying an image, comprising:
applying a computer vision procedure to the image to detect a target therein, and assigning a computer detection score to the image using said computer vision procedure;
presenting the image to an observer as a visual stimulus, while collecting neurophysiological signals from a brain of said observer;
processing said neurophysiological signals to identify a neurophysiological event indicative of a detection of the target by said observer, and assigning to the image a neurophysiological detection score, based on said identification;
determining an existence of said target in the image based on both said computer detection score and said neurophysiological detection score;
determining a mismatch between said computer detection score and said neurophysiological detection score; and
when said mismatch is detected, re-presenting said image to said observer.
2. The method according to claim 1 , wherein said neurophysiological signals comprise EEG signals, and wherein the method comprises calculating a ratio of slow wave EEG to fast wave EEG and determining a neurophysiological state of said observer based on said ratio.
3. The method according to claim 1 , wherein said computer vision procedure employs clustering.
4. The method according to claim 1 , wherein said computer vision procedure employs neural networks.
5. The method according to claim 1 , wherein said collecting said neurophysiological signals is at a sampling rate of at least 150 Hz.
6. The method according to claim 1 , further comprising applying a low pass filter to said collected neurophysiological signals.
7. The method according to claim 1 , wherein said processing said neurophysiological signals comprises applying a Spatially Weighted Fisher Linear Discriminant (SWFLD) classifier to said neurophysiological signals.
8. The method according to claim 1 , wherein said processing said neurophysiological signals comprises applying a convolutional neural network (CNN) classifier to said neurophysiological signals.
9. The method according to claim 8 , wherein said CNN comprises a first convolution layer applying spatial filtering for each of a plurality of time points characterizing said neurophysiological signals, a second convolution layer applying temporal filtering to outputs provided by said first convolution layer, and a third convolution layer applying temporal filtering to outputs provided by said second convolution layer.
10. The method according to claim 1 , wherein said processing said neurophysiological signals comprises:
applying a Spatially Weighted Fisher Linear Discriminant (SWFLD) classifier;
calculating a SWFLD classification score based on said SWFLD classifier;
applying a convolutional neural network (CNN) classifier to said neurophysiological signals;
calculating a CNN classification score based on said CNN classifier; and
combining said SWFLD score and said CNN score.
11. The method according to claim 1 , further comprising presenting to said observer a feedback regarding said identification of said neurophysiological event.
12. The method according to claim 11 , wherein said feedback is binary.
13. The method according to claim 11 , wherein said feedback is non-binary.
14. The method according to claim 1 , being used in a virtual reality system or an augmented reality system.
15. The method according to claim 14 , further comprising varying a field-of-view provided by said virtual reality or augmented reality system, in response to said determination of said existence.
16. The method according to claim 1 , being used in an augmented reality system.
17. The method according to claim 1 , further comprising processing said neurophysiological signals to identify eye blinks, and re-presenting said image to said observer in response to a positive identification of said eye blink during previous presentation of said image.
18. The method according to claim 1 , further comprising repeating said presentation and said identification of said neurophysiological event, comparing said identification to a previous identification of said image, and determining a neurophysiological state of said observer based on said comparison.
19. The method according to claim 1 , further comprising presenting to said observer a database image containing said target, processing said neurophysiological signals to identify a neurophysiological event indicative of a detection of the target in said database image by said observer, and determining a neurophysiological state of said observer based on said identification.
20. The method according to claim 1 , further comprising processing said neurophysiological signals to identify muscle tonus, and re-presenting said image to said observer in response to a positive identification of said muscle tonus during previous presentation of said image.
21. The method according to claim 1 , further comprising processing said neurophysiological signals to identify eye blinks, evaluating a temporal pattern of said eye blinks and determining a neurophysiological state of said observer based on said temporal pattern.