Tokenized cache

1. A method comprising:

parsing text of items in a database to create a plurality of tokens, wherein each token of the plurality of tokens includes a plurality of characters having a length and a frequency value, wherein the frequency value is indicative of a frequency with which that token appears in the database;

storing the plurality of tokens in a memory cache;

receiving a textual search query having contents;

parsing the textual search query into tokens;

selecting a search token from the textual search query based on the frequency value of one of the plurality of tokens in the memory cache;

searching a subset of the database for the contents of the textual search query, wherein the subset of the database includes items having the search token; and

providing the items as a result.

2. The method of claim 1 , wherein the length is two characters.

3. The method of claim 1 , wherein the length is three characters.

4. The method of claim 1 , wherein selecting the search token comprises selecting the token having the lowest frequency value in the memory cache.

5. The method of claim 1 , wherein the selecting the search token comprises selecting the token of the textual search query having the frequency value in the memory cache that is below a threshold.

6. The method of claim 1 , wherein the selecting the search token comprises:

calculating a probability that a first token of the search query appears adjacent to a second token of the search query; and

selecting the first token having a combined frequency value in the memory cache and calculated probability that is below a threshold.

7. The method of claim 1 , further comprising:

storing in the memory cache a list of words that commonly appear in the database;

associating the search token with a word of the list of words;

parsing the textual search query to determine if the word of the textual search query is stored in the list of words; and

searching a subset of the database for contents of the textual search query, wherein the subset of the database is a set of items that include the search token associated with the word of the textual search query.

8. The method of claim 1 , further comprising:

storing the textual search query as a first textual search query in the memory cache;

storing a result of a search using the first textual search query in the memory cache;

receiving a second textual search query;

determining that the second textual search query matches the first textual search query; and

returning the result of the first textual search query.

9. A system for searching a database of data items stored on a non-transitory computer readable medium, wherein the data items each comprise a set of text characters divisible into a plurality of tokens, each token of the plurality of tokens being a set of characters having a length within the data item, the system comprising:

a memory cache having a matrix storing each of the plurality of tokens of the database and a frequency value, wherein the frequency value is indicative of a frequency with which each token appears in the database,

wherein a subset of the data items are searched in the database based on the frequency value for each token stored in the memory cache.

10. The system of claim 9 , wherein the length is three characters.

11. The system of claim 9 , wherein the length is two characters.

12. The system of claim 9 , wherein the subset of the data items is a set of data items having a token of a textual search query with a lowest frequency value in the memory cache among other tokens of the textual search query.

13. The system of claim 9 , wherein the subset of the data items is a set of data items having a token of a textual search query with a frequency value below a threshold.

14. The system of claim 9 , wherein the subset of the data items is a set of data items having a token of a textual search query, wherein the token is selected by:

calculating a probability that a first token of the textual search query appears adjacent to a second token of the textual search query; and

selecting the first token of the textual search query having a combined frequency value in the memory cache and calculated probability that is below a threshold.

15. The system of claim 9 , wherein the memory cache further includes a list of words that frequently appear in the database and a search token for each word of the list.

16. The system of claim 9 , wherein the memory cache further includes a list of prior textual search queries and a search result associated with each prior textual search query of the list.

17. A method comprising:

dividing a database into a plurality of search spaces, wherein each search space of the plurality of search spaces comprises items in the database containing a similar token;

selecting the search space based on a frequency with which a token of a search item provided by a search query appears in the items of the database; and

searching the search space for the search item provided by the search query.

18. The method of claim 17 , wherein the selecting the search space further comprises selecting the search space corresponding to the token having a lowest-in the search item provided by the search query.

19. The method of claim 17 , wherein the selecting the search space further comprises selecting the search space corresponding to a first token in the item provided by the search query having a frequency lower than a threshold.

20. The method of claim 17 , wherein the selecting the search space further comprises:

calculating a probability that a first token of the search item provided by the search query appears adjacent to a second token of the search item provided by the search query; and

selecting the search space corresponding to the first token of the search item provided by the search query that has a combined frequency and calculated probability below a threshold.