Visual brain-computer interface
A method and system for tracking visual attention are disclosed. By generating at least one visual stimulus with a characteristic modulation, the characteristic modulation being applied to high spatial frequency, HSF, components of the visual stimulus and displaying the or each visual stimulus via a graphical user interface, GUI, of a display, a neural response may be induced in the user's brain. By receiving neural signals of a user from a neural signal capture device, such as an EEG device, a point of focus of the user (when the user views the visual stimulus) may be determined based on the neural signals, since the neural signals include information associated with the characteristic modulation of a visual stimulus to which the user's visual attention is directed.
1. A method of tracking visual attention, the method comprising: obtaining a base image; generating a visual stimulus by performing operations comprising: processing a portion of the base image to extract high spatial frequency (HSF) components of the base image, the visual stimulus having a characteristic modulation, the characteristic modulation being applied to the HSF components of the base image, the HSF components having a spatial frequency higher than a predetermined high spatial frequency threshold; processing the portion of the base image to generate low spatial frequency (LSF) components of the base image, the LSF components having a spatial frequency lower than a predetermined low spatial frequency threshold; applying at least one of a spatial frequency filter or a spatial frequency transform to the portion of the base image; and storing the LSF components and the HSF components as LSF map data and HSF map data respectively; displaying the visual stimulus via a graphical user interface (GUI) of a display; receiving neural signals of a user from a neural signal capture device; and on the basis of detecting information associated with the characteristic modulation of the visual stimulus in the neural signals, determining a point of focus of the user in the viewed image, wherein generating the stimulus further comprises: on the basis of determining the HSF map data does not fulfil a predetermined spatial frequency threshold, performing operations comprising: generating enhanced HSF map data, thereby introducing additional HSF stimulation in the visual stimulus; and applying a temporal modulation to the enhanced HSF map data.
2. The method of claim 1 , wherein generating enhanced HSF map data further comprises:
on the basis of determining the luminosity of the portion of the base image falls within a predetermined range, generating an HSF noise map for use as the enhanced HSF map data.
3. The method of claim 1 , wherein generating enhanced HSF map data further comprises:
on a basis of determining the luminosity of the portion of the base image does not fall within a predetermined range, performing operations comprising:
generating a gray overlay map; and
generating an HSF noise map for use as the enhanced HSF map data, and
wherein displaying the visual stimulus further comprises:
merging the enhanced HSF map data with the gray overlay map to provide a blended image; and
displaying the blended image via the GUI.
4. The method of claim 3 , wherein the HSF noise map comprises a hard noise map.
5. The method of claim 1 , wherein displaying the visual stimulus further comprises:
merging the visual stimulus with the base image to provide a blended image; and
displaying the blended image via the GUI.
6. A computing apparatus comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the computing apparatus to perform operations comprising: obtaining a base image; generating a visual stimulus by performing operations comprising: processing a portion of the base image to extract high spatial frequency (HSF) components of the base image, the visual stimulus having a characteristic modulation, the characteristic modulation being applied to the HSF components of the base image, the HSF components having a spatial frequency higher than a predetermined high spatial frequency threshold; processing the portion of the base image to generate low spatial frequency (LSF) components of the base image, the LSF components having a spatial frequency lower than a predetermined low spatial frequency threshold; applying at least one of a spatial frequency filter or a spatial frequency transform to the portion of the base image; and storing the LSF components and the HSF components as LSF map data and HSF map data respectively; displaying the visual stimulus via a graphical user interface (GUI) of a display; receiving neural signals of a user from a neural signal capture device; and on the basis of detecting information associated with the characteristic modulation of the visual stimulus in the neural signals, determining a point of focus of the user in the viewed image, wherein the instructions that, when executed by the one or more processors, cause the computing apparatus to perform operations comprising generating the stimulus further cause the computing apparatus to perform operations comprising: on the basis of determining the HSF map data does not fulfil a predetermined spatial frequency threshold, performing operations comprising: generating enhanced HSF map data, thereby introducing additional HSF stimulation in the visual stimulus; and applying a temporal modulation to the enhanced HSF map data.
7. The computing apparatus of claim 6 wherein the instructions that, when executed by the one or more processors, cause the computing apparatus to perform operations comprising enhanced HSF map data further cause the computer to perform operations comprising: on the basis of determining the luminosity of the portion of the base image falls within a predetermined range, generating an HSF noise map for use as the enhanced HSF map data.
8. The computing apparatus of claim 6 , wherein the instructions that, when executed by the one or more processors, cause the computing apparatus to perform operations comprising generating enhanced HSF map further cause the computing apparatus to perform operations comprising:
on a basis of determining the luminosity of the portion of the base image does not fall within a predetermined range, performing operations comprising:
generating a gray overlay map; and
generating an HSF noise map for use as the enhanced HSF map data, and wherein displaying the visual stimulus further comprises:
merging the enhanced HSF map data with the gray overlay map to provide a blended image; and
displaying the blended image via the GUI.
9. The computing apparatus of claim 8 , wherein the HSF noise map comprises a hard noise map.
10. The computing apparatus of claim 6 , wherein the instructions that, when executed by the one or more processors, cause the computing apparatus to perform operations comprisng displaying the or each visual stimulus further cause the computing apparatus to perform operations comprising:
merging the visual stimulus with the base image to provide a blended image; and
displaying the blended image via the GUI.
11. A non-transitory computer-readable storage medium, the non-statutory computer-readable storage medium including instructions that when executed by one or more processors of a computer, cause the computer to perform operations comprising: obtaining a base image; generating a visual stimulus by performing operations comprising: processing a portion of the base image to extract high spatial frequency (HSF) components of the base image, the visual stimulus having a characteristic modulation, the characteristic modulation being applied to the HSF components of the base image, the HSF components having a spatial frequency higher than a predetermined high spatial frequency threshold; processing the portion of the base image to generate low spatial frequency (LSF) components of the base image, the LSF components having a spatial frequency lower than a predetermined low spatial frequency threshold; applying at least one of a spatial frequency filter or a spatial frequency transform to the portion of the base image; and storing the LSF components and the HSF components as LSF map data and HSF map data respectively; displaying the visual stimulus via a graphical user interface (GUI) of a display; receiving neural signals of a user from a neural signal capture device; and on the basis of detecting information associated with the characteristic modulation of the visual stimulus in the neural signals, determining a point of focus of the user in the viewed image, wherein the instructions that, when executed by the one or more processors, cause the computer to perform operations comprising generating the stimulus further cause the computer to perform operations comprising: on the basis of determining the HSF map data does not fulfil a predetermined spatial frequency threshold, performing operations comprising: generating enhanced HSF map data, thereby introducing additional HSF stimulation in the visual stimulus; and applying a temporal modulation to the enhanced HSF map data.
12. The non-statutory computer-readable storage medium of claim 11 , wherein the instructions that, when executed by the one or more processors, cause the computer to perform operations comprising generating enhanced HSF map data further cause the computer to perform operations comprising:
on the basis of determining the luminosity of the portion of the base image falls within a predetermined range, generating an HSF noise map for use as the enhanced HSF map data.
13. The non-statutory computer-readable storage medium of claim 11 , wherein the computer instructions that, when executed by the one or more processors cause the computer to perform operations comprising generating enhanced HSF map data further cause the computer to perform operations comprising:
on a basis of determining the luminosity of the portion of the base image does not fall within a predetermined range, performing operations comprising:
generating a gray overlay map; and
generating an HSF noise map for use as the enhanced HSF map data, and
wherein displaying the visual stimulus further comprises:
merging the enhanced HSF map data with the gray overlay map to provide a blended image; and
displaying the blended image via the GUI.
14. The non-statutory computer-readable storage medium of claim 13 , wherein the HSF noise map comprises a hard noise map.
15. The non-statutory computer-readable storage medium of claim 11 , wherein the instructions that, when executed by the one or more processors, cause the computer to perform operations comprising displaying the visual stimulus further cause the computer to perform operations comprising:
merging the visual stimulus with the base image to provide a blended image; and
displaying the blended image via the GUI.