Searching a subset of a database having search tokens
Methods of and systems for searching a catalog include parsing the items of the catalog into tokens, determining the frequency with which each token appears in the catalog, and storing the frequencies in a cache. Queries to the catalog are likewise parsed into tokens, and the tokens of the query string are compared to frequency values in the cache to identify a smaller search space within the catalog.
1. A method comprising:
parsing a textual search query into a plurality of search tokens;
selecting a search token from the textual search query based on a frequency value of one or more of a plurality of tokens,
wherein the plurality of tokens is created by parsing text of items,
wherein each token of the plurality of tokens includes a plurality of characters having a frequency value, and
wherein the frequency value is indicative of a frequency with which each token appears in the database;
searching a subset of a database for at least a portion of contents of the textual search query, wherein the subset of the database includes items having the search token; and
providing a subset of the items that satisfy the textual search query.
2. The method of claim 1 , wherein the plurality of characters has a length of at least one of two characters or three characters.
3. The method of claim 1 , further comprising storing the plurality of tokens in a memory cache.
4. The method of claim 1 , wherein the selecting the search token comprises selecting the token having the lowest frequency value in a memory cache.
5. The method of claim 1 , wherein the selecting the search token comprises selecting the token of the textual search query having the frequency value in a memory cache that is below a threshold.
6. The method of claim 1 , wherein the selecting the search token comprises:
calculating a probability that a first token of the search query appears adjacent to a second token of the search query; and
selecting the first token having a combined frequency value in a memory cache and calculated probability that is below a threshold.
7. The method of claim 1 , further comprising:
storing in a memory cache a list of tokens that commonly appear in the database;
associating the search token with a word of the list of words;
parsing the textual search query to determine if the word of the textual search query is stored in the list of words; and
searching a subset of the database for contents of the textual search query, wherein the subset of the database is a set of items that include the search token associated with the word of the textual search query.
8. The method of claim 1 , further comprising:
storing the textual search query as a first textual search query in a memory cache;
storing a result of a search using the first textual search query in the memory cache;
receiving a second textual search query;
determining that the second textual search query matches the first textual search query; and
returning the result of the first textual search query.
9. A system configured for searching a database of data items stored on a non-transitory computer readable medium, the system comprising:
a database of data items;
wherein each of the data items comprise a set of text characters divisible into a plurality of tokens;
a matrix comprising a data structure having an array of the plurality of tokens in rows and columns,
wherein the matrix is treated as a single entity and manipulated according to rules,
wherein the matrix is configured to store each of the plurality of tokens in the database, wherein the matrix is configured to store a frequency value for each of the plurality of tokens,
wherein the frequency value for each of the plurality of tokens is indicative of a frequency with which each of the plurality of tokens appears in the database,
wherein a subset of the data items in the database is configured to be searched in the database based on the frequency value for each of the plurality of tokens;
wherein the subset of the data items in the database has a search token,
wherein the subset of the data items in the database is configured to be searched in the database for at least a portion of contents satisfying a textual search query, and
wherein the subset of the data items satisfies the textual search query.
10. The system of claim 9 , wherein each token of the plurality of tokens being a set of characters having a length within the data item, wherein the length is at least one of two characters or three characters.
11. The system of claim 9 , wherein the matrix is stored in a memory cache.
12. The system of claim 9 , wherein the subset of the data items is a set of data items having a token of the textual search query with a lowest frequency value among other tokens of the textual search query.
13. The system of claim 9 , wherein the subset of the data items is a set of data items having a token of the textual search query with a frequency value below a threshold.
14. The system of claim 9 , wherein the subset of the data items is a set of data items having a token of the textual search query, wherein the token is selected by:
calculating a probability that a first token of the textual search query appears adjacent to a second token of the textual search query; and
selecting the first token of the textual search query having a combined frequency value and calculated probability that is below a threshold.
15. The system of claim 9 , further comprising a list of words that frequently appear in the database and the search token for each word of the list.
16. The system of claim 9 , further comprising a list of prior textual search queries and a search result associated with each prior textual search query of the list.
17. A method comprising:
dividing a database of data items stored on a non-transitory computer readable medium into a plurality of search spaces, wherein each search space of the plurality of search spaces comprises items containing a token having at least a portion of similar characters;
selecting the search space based on a frequency with which a token of a search item appears in the items of the search space, wherein the selecting comprises:
calculating a probability that a first token of the search item appears adjacent to a second token of the search item provided;
selecting the search space corresponding to the first token of the search item that has a combined frequency and calculated probability below a threshold; and
searching the search space for the search item that satisfies a textual search query.
18. The method of claim 17 , wherein the selecting the search space further comprises selecting the search space corresponding to the token having a lowest frequency in the search item.
19. The method of claim 17 , wherein the selecting the search space further comprises selecting the search space corresponding to a first token in the item having a frequency lower than a threshold.