System for mapping extracted Neural activity into Neuroceptual graphs
Described is a system for neural decoding of neural activity. Using at least one neural feature extraction method, neural data that is correlated with a set of behavioral data is transformed into sparse neural representations. Semantic features are extracted from a set of semantic data. Using a combination of distinct classification modes, the set of semantic data is mapped to the sparse neural representations, and new input neural data can be interpreted.
1. A system for neural decoding, the system comprising;
one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when executed, the one or more processors perform operations of:
using at least one neural feature extraction method, transforming the neural data into sparse neural representations, the neural data being correlated with a set of behavioral data;
extracting a set of semantic features from a set of semantic data;
using a combination of distinct classification modes, mapping the set of semantic data to the sparse neural representations,
wherein in a first classification mode, learning a transformation between the set of semantic data and the sparse neural representations,
where in learning the transformation between the set of semantic data and the sparse neural representations, the one or more processors perform operations of:
using a linear support vector machine (SVM) to generate a SVM matrix of rows and columns, wherein each row in the SVM matrix represents unknown sparse neural representations and each column represents a probability of that sparse neural representation being present in the set of semantic data;
transforming the SVM matrix into a semantic-based matrix, wherein for each word in the set of semantic data, its ontological attributes are determined and indexed into columns to generate the semantic-based matrix;
averaging the columns of the semantic-based matrix to produce a value representing how likely the word is in the unknown sparse neural representations; and
generating a ranking matrix comprising values representing a plurality of words in the set of semantic data; and
decoding new input neural data.
2. The system as set forth in claim 1 , wherein a plurality of neural feature extraction methods are used serially.
3. The system as set forth in claim 1 , wherein the one or more processors further perform operations of:
training the distinct classification modes with the set of behavioral data, the set of semantic data, and the sparse neural representations;
and
in a second classification mode, performing neuroceptual lattice classification.
4. The system as set forth in claim 3 , wherein the one or more processors further perform an operation of generating a ranked list of words from new input neural data using the trained classification modes.
5. The system as set forth in claim 1 , wherein in selecting a subset of neural data, the one or more processors further perform at least one of:
sorting voxels from a set of neural data by stability, and selecting a subset of voxels determined to be most stable; and
identifying a subset of voxels from the set of neural data that is correlated with a feature subset in the set of behavioral data using sparse canonical correlation analysis (SCCA).
6. The system as set forth in claim 1 , wherein the new input neural data is neural activity of a driver of a vehicle, wherein neural decoding of the driver's neural activity is used to control at least one component of the vehicle.
7. The system as set forth in claim 1 , wherein the neural data includes functional magnetic resonance imaging (fMRI) data.
8. A computer implemented method for neural decoding, the method comprising an act of:
causing one or more processors to execute instructions encoded on a non-transitory computer-readable medium, such that upon execution, the one or more processors perform operations of:
using at least one neural feature extraction method, transforming the neural data into sparse neural representations, the neural data being correlated with a set of behavioral data;
extracting a set of semantic features from a set of semantic data;
using a combination of distinct classification modes, mapping the set of semantic data to the sparse neural representations,
wherein in a first classification mode, learning a transformation between the set of semantic data and the sparse neural representations,
where in learning the transformation between the set of semantic data and the sparse neural representations, the one or more processors perform operations of:
using a linear support vector machine (SVM) to generate a SVM matrix of rows and columns, wherein each row in the SVM matrix represents unknown sparse neural representations and each column represents a probability of that sparse neural representation being present in the set of semantic data;
transforming the SVM matrix into a semantic-based matrix, wherein for each word in the set of semantic data, its ontological attributes are determined and indexed into columns to generate the semantic-based matrix;
averaging the columns of the semantic-based matrix to produce a value representing how likely the word is in the unknown sparse neural representations; and
generating a ranking matrix comprising values representing a plurality of words in the set of semantic data; and
decoding new input neural data.
9. The method as set forth in claim 8 , wherein a plurality of neural feature extraction methods are used serially.
10. The method as set forth in claim 8 , wherein the one or more processors further perform operations of:
training the distinct classification modes with the set of behavioral data, the set of semantic data, and the sparse neural representations;
and
in a second classification mode, performing neuroceptual lattice classification.
11. The method as set forth in claim 10 , wherein the one or more processors further perform an operation of generating a ranked list of words from new input neural data using the trained classification modes.
12. The method as set forth in claim 8 , wherein in selecting a subset of neural data, the one or more processors further perform at least one of:
sorting voxels from a set of neural data by stability, and selecting a subset of voxels determined to be most stable; and
identifying a subset of voxels from the set of neural data that is correlated with a feature subset in the set of behavioral data using sparse canonical correlation analysis (SCCA).
13. The method as set forth in claim 8 , wherein the new input neural data is neural activity of a driver of a vehicle, wherein neural decoding of the driver's neural activity is used to control at least one component of the vehicle.
14. The method as set forth in claim 8 , wherein the neural data includes functional magnetic resonance imaging (fMRI) data.
15. A computer program product for neural decoding, the computer program product comprising:
a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions by one or more processors, the one or more processors perform operations of:
using at least one neural feature extraction method, transforming the neural data into sparse neural representations, the neural data being correlated with a set of behavioral data;
extracting a set of semantic features from a set of semantic data;
using a combination of distinct classification modes, mapping the set of semantic data to the sparse neural representations,
wherein in a first classification mode, learning a transformation between the set of semantic data and the sparse neural representations,
where in learning the transformation between the set of semantic data and the sparse neural representations, the one or more processors perform operations of:
using a linear support vector machine (SVM) to generate a SVM matrix of rows and columns, wherein each row in the SVM matrix represents unknown sparse neural representations and each column represents a probability of that sparse neural representation being present in the set of semantic data;
transforming the SVM matrix into a semantic-based matrix, wherein for each word in the set of semantic data, its ontological attributes are determined and indexed into columns to generate the semantic-based matrix;
averaging the columns of the semantic-based matrix to produce a value representing how likely the word is in the unknown sparse neural representations; and
generating a ranking matrix comprising values representing a plurality of words in the set of semantic data; and
decoding new input neural data.
16. The computer program product as set forth in claim 15 , wherein a plurality of neural feature extraction methods are used serially.
17. The computer program product as set forth in claim 15 , further comprising instructions for causing the one or more processors to further perform operations of:
training the distinct classification modes with the set of behavioral data, the set of semantic data, and the sparse neural representations;
and
in a second classification mode, performing neuroceptual lattice classification.
18. The computer program product as set forth in claim 17 , further comprising instructions for causing the one or more processors to further perform an operation of generating a ranked list of words from new input neural data using the trained classification modes.
19. The computer program product as set forth in claim 15 , further comprising instructions for causing the one or more processors in selecting a subset of neural data to further perform at least one of:
sorting voxels from a set of neural data by stability, and selecting a subset of voxels determined to be most stable; and
identifying a subset of voxels from the set of neural data that is correlated with a feature
subset in the set of behavioral data using sparse canonical correlation analysis (SCCA).