SYSTEM TO ORGANIZE SEARCH AND DISPLAY UNSTRUCTURED DATA
A system to organize, search and display unstructured data comprising a token retrieval module, a document indexing engine, a subspace search module and a user interface module has been devised. The system retrieves a plurality of tokens and associates them with coordinates in subspace. It also retrieves documents and creates a multidimensional matrix of documents and tokens where each cell contains the number of times the token occurs in each document. That matrix is employed in a search using user specified search terms. The search results are displayed such that the search tokens occupy specific spatial coordinates and documents spatial coordinates are dictated by the relative preponderance of each search term in each document.
1 . A system to organize, search and display unstructured data comprising:
a token indexer stored in a memory of and operating on a processor of a computing device and configured to:
retrieve a plurality of tokens from a token data store, each token comprising a discrete portion of natural language information stored in an electronic format;
associate at least a portion of the retrieved tokens with multidimensional coordinates in a subspace data model; and
store at least a portion of the subspace coordinate associated tokens;
a document indexer stored in a memory of and operating on a processor of a computing device and configured to:
retrieve a plurality of documents from a document source;
create a raw multidimensional matrix of documents and subspace coordinate associated tokens, wherein each cell in the multidimensional matrix contains the number of times a particular token occurs in the corresponding document;
create a normalized matrix of documents and subspace coordinate associated tokens based at least in part on the raw multidimensional matrix; and
create a rank subspace matrix based at least in part on the normalized matrix;
a search management module stored in a memory of and operating on a processor of computing device and configured to:
receive a plurality of subspace coordinate associated tokens as search tokens based on a query provided by a user interface module;
create a rank basis matrix from the rank subspace matrix of the normalized matrix;
perform a search of documents using the search tokens and using rank basis matrix;
calculate document display coordinates from search results; and
refine document display coordinates based upon extreme feature calculations;
a user interface module stored in a memory of and operating on a processor of a computing device and configured to:
allow a user to select document source for an analysis;
allow the user to select token source for analysis;
accept selected subspace coordinate associated search tokens from the user; and
display search tokens and documents at view space coordinates calculated from document display coordinates retrieved from the search management module.
2 . A method for organization, search and display unstructured data comprising the steps of:
retrieving a plurality of tokens and assigning subspace coordinates to them using a token indexer;
retrieving a plurality of documents and indexing them into a multidimensional matrix with the retrieved tokens using a document indexer;
normalizing and centering the rows and columns of the multidimensional matrix using the document indexer;
searching the normalized multidimensional matrix with user specified tokens and creating display coordinate for each token and each searched document using a search management module;
displaying search result dictated tokens and documents at coordinates that reflect the relative preponderance of each search token in each analyzed document using a user controlled display module.