System to organize search and display unstructured data
A system to organize, search and display unstructured data comprising a token retrieval module, a document indexing engine, a subspace search module and a user interface module has been devised. The system retrieves a plurality of tokens and associates them with coordinates in subspace. It also retrieves documents and creates a multidimensional matrix of documents and tokens where each cell contains the number of times the token occurs in each document. That matrix is employed in a search using user specified search terms. The search results are displayed such that the search tokens occupy specific spatial coordinates and documents spatial coordinates are dictated by the relative preponderance of each search term in each document.
1. A system, comprising:
one or more processors; and
memory storing instructions that, as a result of execution by the one or more processors, cause the system to at least:
provide a user interface that:
presents a plurality of icons on a graph to display a search result, the plurality of icons individually representing records of the search result and having coordinates calculated based at least in part on a first set of weights of tokens used to calculate the search result;
presents a first icon representative of a token, the token being from the tokens;
allows a user to manipulate the icon representative of the token by changing coordinates of the token to change a weight of the token in calculation of coordinates of icons representing records;
recalculates the coordinates of the plurality of icons based at least in part on the manipulation of the icon and a second set of weights of the tokens indicated by the manipulation; and
updates at least a position of a second icon representing a record based at least in part on the recalculated coordinates to display a different view of the search result.
2. The system of claim 1 , wherein the first set of weights is determined based at least in part on a first position of the icon prior to the manipulation of the second icon, wherein the second set of weights is determined based at least in part on a second position of the icon resulting from the manipulation of the second icon, the second position being different from the first position and the first set of weights being different from the second set of weights.
3. The system of claim 1 , wherein the record comprises unstructured data.
4. The system of claim 1 , wherein the record comprises structured data.
5. The system of claim 1 , wherein the record comprises network flow data.
6. The system of claim 1 , wherein the token comprises a value in the record.
7. The system of claim 1 , wherein:
a location of the second icon is determined based at least in part on a projection of the token onto a subspace of a multidimensional space; and
locations for the plurality of icons in the user interface are determined such that higher occurrence of the token results in closer proximity to the location of the second icon.
8. The system of claim 1 , wherein the system allows the user to manipulate the icon representative of the token by deleting the token to result in an coordinates of records to be calculated independently of the token.
9. The system of claim 1 , wherein the system allows the user to manipulate the icon representative of the token by adding a token to result in coordinates of results to be calculated based at least in part on the added token.
10. The system of claim 1 , wherein the user interface recalculates the coordinates by at least:
creating a partial basis matrix for search from the tokens based at least in part on the second set of weights;
performing a search of records using the partial basis matrix for search and another partial basis matrix of a reduced rank subspace of a space of tokens, thereby resulting in coordinates of search tokens within the subspace.
11. A computer-implemented method, comprising:
providing a user interface that:
displays search results by displaying a first plurality of icons, the plurality of icons comprising a first icon representing a record of the search result and a second icon representing a token used to calculate the search result, the first icon and second icon having respective coordinates calculated based at least in part on a first set of weights of tokens; and
allows manipulation of the second icon by changing coordinates of the second icon to change a weight of the second token and thereby indicate a second set of weights of tokens;
as a result of the manipulation of the second icon, determining an updated search result representative of a plurality of records, the plurality of records comprising the record, wherein the updated search result is based at least in part on the second set of weights; and
displaying a second plurality of icons representative of the updated search result, the second plurality of icons having respective coordinates calculated based, at least in part, on the second set of weights of tokens.
12. The computer-implemented method of claim 11 , wherein the user interface displays the first plurality of icons based at least in part on a respective combination of tokens.
13. The computer-implemented method of claim 12 , wherein the combination is a weighted combination based at least in part on a location of an icon.
14. The computer-implemented method of claim 11 , wherein the record comprises a plurality of field values and the token is a field value of the plurality of field values.
15. The computer-implemented method of claim 11 , wherein the record comprises unstructured data and the token is a value determined from the unstructured data.
16. The computer-implemented method of claim 11 , wherein the record comprises structured data and the token is a value determined from the structured data.
17. The computer-implemented method of claim 11 , further comprising determining locations for the second plurality of icons based at least in part on the manipulation.
18. The computer-implemented method of claim 17 , wherein determining the locations comprises calculating a plurality of projections onto a subspace of a multidimensional space.
19. The computer-implemented method of claim 18 , wherein coordinate values in the multidimensional space are tokens and a basis vector for the subspace is determined based at least in part on a plurality of token values.
20. The computer-implemented method of claim 11 , wherein manipulation of the token comprises deletion of the token, addition of another token to the token, or a change of coordinates of the token.