System and method for performing longest common prefix strings searches
A method and system a method for compressing and searching a plurality of strings. The method includes inputting a plurality of strings into a compression engine. The method also includes converting each of the plurality of strings into a new, prefix-preserving compressed string, using the compression engine. For every string P that is a strict prefix of a string S, P's resulting compressed string is a strict prefix of S's resulting compressed string.
1. A method of searching for a particular string among one or more prefix-preserving compressed strings, the method comprising:
receiving a plurality of strings at a data collection site, wherein a subset of the plurality of strings have a strict prefix;
compressing, using a compression engine at the data collection site, the plurality of strings into a plurality of prefix-preserving compressed strings, wherein the plurality of prefix-preserving compressed strings comprises a compressed subset of strings corresponding to the subset of the plurality of strings such that the compressed subset of strings have a matching compressed strict prefix;
storing the prefix-preserving compressed strings; and
compressing a particular string to obtain a compressed particular string and applying a search algorithm to find the compressed particular string among the stored prefix-preserving compressed strings.
2. The method according to claim 1 , wherein one or more strings of the plurality of strings represents at least one of: a uniform resource locator, and an IP address.
3. The method according to claim 1 , further comprising:
storing one or more of the prefix-preserving compressed strings in at least one of: an ordered list data structure, a database, a tree data structure, a graph data structure, a trie data structure, and a client library.
4. The method according to claim 1 , wherein the compression engine comprises at least one of: a dictionary based compression engine, an entropy based compression engine, a run-length compression engine, an LZ77 compression engine, an LZ78 compression engine, an LZW compression engine, a Huffman compression engine, a Golomb compression engine, a universal code compression engine, an Elias gamma compression engine, a dynamic markov compression engine, and an arithmetic compression engine.
5. The method according to claim 1 , wherein the search algorithm comprises at least one of: a linear search, a binary search, an interpolation search, a select query, a full text search, a search or find database operation, a breadth-first search algorithm, a depth-first search algorithm, an A* search algorithm, a breadth-first trie search algorithm, a depth-first trie search algorithm, and an A* trie search algorithm.
6. The method according to claim 1 , further comprising:
determining a closest matching string in the one or more prefix-preserving compressed strings, the closest matching string having a longest common prefix with the compressed particular string.
7. A system for compressing one or more strings, the system comprising:
a network;
a data collection site on the network, the data collection site comprising a compression engine, the compression engine adapted to:
receive a plurality of strings, wherein a subset of the plurality of strings have a strict prefix; and
convert the plurality of strings into prefix-preserving compressed strings, wherein the plurality of prefix-preserving compressed strings comprises a compressed subset of strings corresponding to the subset of the plurality of strings such that the compressed subset of strings have a matching compressed strict prefix; and
a security appliance, the security appliance configured to store the one or more prefix-preserving compressed strings and apply a search algorithm to find a particular compressed string among the one or more prefix-preserving compressed strings.
8. The system according to claim 7 , wherein one or more strings of the plurality of strings represents at least one of: a uniform resource locator, and an IP address.
9. The system according to claim 7 , wherein the compression engine comprises at least one of: a dictionary based compression engine, an entropy based compression engine, a run-length compression engine, an LZ77 compression engine, an LZ78 compression engine, an LZW compression engine, a Huffman compression engine, a Golomb compression engine, a universal code compression engine, an Elias gamma compression engine, a dynamic markov compression engine, and an arithmetic compression engine.
10. The system according to claim 7 , wherein the search algorithm comprises at least one of: a linear search, a binary search, an interpolation search, a select query, a full text search, a search or find database operation, a breadth-first search algorithm, a depth-first search algorithm, an A* search algorithm, a breadth-first trie search algorithm, a depth-first trie search algorithm, and an A* trie search algorithm.
11. A method for arithmetic coding uniform resource locator strings, the method comprising:
receiving a plurality of uniform resource locators at an arithmetic coding engine, each of the plurality of uniform resource locators having a string value;
analyzing the string value of each of the plurality of uniform resource locators;
generating an arithmetic coded hash for each of the plurality of uniform resource locators according to the string value of each of the plurality of uniform resource locators to create a plurality of hashed uniform resource locators; and
grouping each of the plurality of hashed uniform resource locators according to a hashed string value of each of the plurality of uniform resource locators to create a subset of the plurality of hashed uniform resource locators;
wherein a first hashed uniform resource locator of the plurality of hashed uniform resource locators has a string value that is within a predetermined string range and has a numeric hash value that is in character proximity to a numeric hash value of a second hashed uniform resource locator of the plurality of hashed uniform resource locators having a string value within the predetermined string range.
12. The method according to claim 11 , wherein each of the plurality of uniform resource locators represents a Web site.
13. The method according to claim 11 , wherein the uniform resource locator has a hash value of 32 characters.
14. The method according to claim 11 , further comprising:
storing the plurality of hashed uniform resource locators in a database.
15. The method according to claim 11 , further comprising:
storing the plurality of hashed uniform resource locators in a client library.
16. The method according to claim 11 , further comprising:
transmitting the plurality of hashed uniform resource locators over a network to a local area network to be stored in a database of the local area network.
17. The method according to claim 16 , further comprising:
receiving a request for an Internet service at a security appliance of the local area network from a client-side device, the client-side device controlling access to the Internet;
analyzing a hashed uniform resource locator for the Internet service, the hashed uniform resource locator having a string value within a predetermined string range and a numeric hash value that is in character proximity to a numeric hash value of another hashed uniform resource locator of the plurality of hashed uniform resource locators that has a string value within the predetermined string range; and
determining access to the Internet service based on the numeric hash value of the hashed uniform resource locator being within a range established for the local area network.
18. The method according to claim 17 , wherein the client-side device is a personal computer.
19. The method according to claim 17 , wherein the client-side device is a PDA.
20. The method according to claim 17 , wherein the Internet service is a Web site.
21. The method according to claim 16 , further comprising:
receiving a uniform resource locator request for an Internet service at a security appliance of the local area network from a client-side device, the client-side device controlling access to an Internet service;
analyzing an arithmetic coded hash value for the uniform resource locator for the Internet service to determine if the arithmetic coded hash value is within an interval of a prefix; and
conditioning access to the Internet service based on whether the arithmetic coded hash value for the uniform resource locator is within the interval of a prefix.
22. The method according to claim 11 , wherein generating the arithmetic coded hash for each of the plurality of uniform resource locators is performed at a data collection site.
23. A system for compressing one or more strings, the system comprising:
a network comprising at least one client side device;
a data collection site on the network, the data collection site comprising a compression engine, the compression engine comprising:
means for receiving a plurality of strings, wherein a subset of the plurality of strings have a strict prefix;
means for compressing the plurality of strings into one or more prefix-preserving compressed strings, wherein the plurality of prefix-preserving compressed strings comprises a compressed subset of strings corresponding to the subset of the plurality of strings such that the compressed subset of strings have a matching compressed strict prefix;
means for storing the one or more prefix-preserving compressed strings; and
means for applying a search algorithm to find one or more compressed strings from the one or more prefix-preserving compressed strings.