Methods for mapping data into lower dimensions
Methods and systems for creating ensembles of hypersurfaces in high-dimensional feature spaces, and to machines and systems relating thereto. More specifically, exemplary aspects of the invention relate to methods and systems for generating supervised hypersurfaces based on user domain expertise, machine learning techniques, or other supervised learning techniques. These supervised hypersurfaces may optionally be combined with unsupervised hypersurfaces derived from unsupervised learning techniques. Lower-dimensional subspaces may be determined by the methods and systems for creating ensembles of hypersurfaces in high-dimensional feature spaces. Data may then be projected onto the lower-dimensional subspaces for use, e.g., in further data discovery, visualization for display, or database access. Also provided are tools, systems, devices, and software implementing the methods, and computers embodying the methods and/or running the software, where the methods, software, and computers utilize various aspects of the present invention relating to analyzing data.
1. A method for accessing and modeling a high-dimensional feature space for presentation on a two-dimensional computer display, comprising:
obtaining, from at least one data repository, a biological or biomedical data set including a high-dimensional feature space of at least four dimensions, with the biological or biomedical data set including data relating to at least one of gene expression, protein expression and clinical study data;
generating, by a processor, a supervised hypersurface using supervised learning techniques on the biological or biomedical data set based on known categories in the biological or biomedical data set;
generating, by the processor, an unsupervised hypersurface using unsupervised learning techniques on the biological or biomedical data set;
combining, by the processor, the generated supervised hypersurface and unsupervised hypersurface to create a lower-dimensional subspace, wherein respective vectors normal to each of the supervised hypersurface and the unsupervised hypersurface determine directions of the subspace;
generating, by the processor, an integrated data model by projecting data from the high-dimensional feature space onto the created lower-dimensional subspace by combining the supervised hypersurface and unsupervised hypersurface; and
configuring and displaying the integrated data model as a user interface on the two-dimensional computer display as a pseudo-three dimensional representation,
wherein the integrated data model displays at least a portion of the biological or biomedical data within the created lower-dimensional subspace that is displayed on the user interface and defined by three ortho-normalized axes with at least one of the three axes generated from one of the respective vectors normal either the generated supervised hypersurface or the generated unsupervised hypersurface.
2. The method of claim 1 , wherein the subspace is defined on an orthonormal basis such that the vector normal to the supervised hypersurface and the vector normal to the unsupervised hypersurface are made orthogonal to each other.
3. The method of claim 1 , wherein the lower-dimensional subspace comprises a new model of the data from the high-dimensional feature space.
4. The method of claim 1 , wherein the projected data is used to conduct further analysis of the projected data.
5. The method of claim 4 , wherein the further analysis is selected from the group consisting of data discovery, data display, and database exploration.
6. The method of claim 1 , wherein the projected data is used to generate a visual data display on the two-dimensional computer display.
7. The method of claim 1 , wherein the unsupervised hypersurface identifies groupings within the data set.
8. The method of claim 1 , wherein the data projected onto the lower dimensional subspace is used to represent a combination of an svm or learning machine and an unsupervised learning method in a single visual subspace model.
9. The method of claim 1 , wherein the supervised learning techniques include a machine learning solution.
10. The method of claim 1 , further comprising:
selecting a vector of a hypothetical or actual data pattern as a direction for the hypothetical or actual data pattern; and
combining the supervised hypersurface and unsupervised hypersurface with the hypothetical or actual data pattern, wherein each of the supervised hypersurface, the unsupervised hypersurface, and the selected vector determines a direction of the subspace.
11. The method of claim 1 , wherein the configuring and display of the integrated data model comprises generating a display interface that is configured for a user to manipulate the pseudo-three dimensional representation by turning, rotating, and altering a point of view.
12. The method of claim 1 , further comprising:
receiving a selection of a representation of a record of the data set in the pseudo-three dimensional representation; and
retrieving the record from the data set.
13. A system for accessing and modeling a high-dimensional feature space for presentation on a two-dimensional computer display, the system comprising:
a memory; and
a processor configured to implement instructions stored on the memory so as to:
generate a supervised hypersurface using supervised learning techniques on a biological or biomedical data set in at least one data repository based on known categories in the biological or biomedical data set, wherein the biological or biomedical data set includes data relating to at least one of gene expression, protein expression and clinical study data and comprises a high-dimensional feature space of at least four dimensions;
generate an unsupervised hypersurface using unsupervised learning techniques on the biological or biomedical data set;
combine the generated supervised hypersurface and unsupervised hypersurface to create a lower-dimensional subspace, wherein respective vectors normal to each of the supervised hypersurface and the unsupervised hypersurface determine directions of the subspace;
generate an integrated data model by projecting data from the high-dimensional feature space onto the create lower-dimensional subspace; and
configure and display the integrated data model as a user interface on the two-dimensional computer display as a pseudo-three dimensional representation,
wherein the integrated data model displays at least a portion of the biological or biomedical data within the created lower-dimensional subspace that is displayed on the user interface and defined by three ortho-normalized axes with at least one of the three axes generated from one of the respective vectors normal either the generated supervised hypersurface or the generated unsupervised hypersurface.
14. The system of claim 13 , wherein the lower-dimensional subspace is defined on an orthonormal basis such that the vector normal to the supervised hypersurface and the vector normal to the unsupervised hypersurface are made orthogonal to each other.
15. The system of claim 13 , wherein the lower-dimensional subspace comprises a new model of the data from the high-dimensional feature space.
16. The system of claim 13 , wherein the projected data is used to conduct further analysis of the projected data.
17. The system of claim 16 , wherein the further analysis is selected from the group consisting of data discovery, data display, and database exploration.
18. The system of claim 13 , wherein the projected data is used to generate a visual data display on the two-dimensional computer display.
19. The system of claim 13 , wherein the unsupervised hypersurface identifies groupings within the data set.
20. The system of claim 13 , wherein the data projected onto the lower dimensional subspace is used to represent a combination of an svm or learning machine and an unsupervised learning method in a single visual subspace model.
21. The system of claim 13 , wherein the supervised learning techniques include a machine learning solution.
22. The system of claim 13 , wherein the processor is configured to implement instructions stored on the memory so as to:
select a vector of a hypothetical or actual data pattern as a direction for the hypothetical or actual data pattern; and
combine the supervised hypersurface and unsupervised hypersurface with the hypothetical or actual data pattern, wherein each of the supervised hypersurface, the unsupervised hypersurface, and the selected vector determines a direction of the lower-dimensional subspace.
23. The system of claim 13 , wherein the configuring and display of the integrated data model comprises generating a display interface that is configured for a user to manipulate the pseudo-three dimensional representation by turning, rotating, and altering a point of view.
24. The system of claim 13 , wherein the processor is configured to implement instructions stored on the memory so as to:
receive a selection of a representation of a record of the data set in the pseudo-three dimensional representation; and
retrieve the record from the data set.