Distributed, compressed Bloom filter Web cache server
Compressed Bloom filters that act as a message as well as a data structure provide smaller false positive rates, reduced bits broadcast and/or reduced computational overhead in distributed Web proxy servers and other distributed networks.
1. A distributed, compressed bloom filter Web server providing reduced probabilities of false positives, comprising:
a plurality of cache servers each having a cache memory and a cache processor coupled to the memory that is operative (1) to represent Web objects stored in its cache memory as a Bloom filter data array having a preselected number of hash functions and a preselected array size which have been chosen to minimize the rate of false positives for a preselected transmission size when said preselected transmission size differs from said preselected array size, (2) to compress the Bloom filter data array to said transmission size, and (3) to periodically disseminate the compressed Bloom filter data array to neighboring servers when there is a change in its stored Web objects.
2. The distributed, compressed Bloom filter Web server providing reduced probabilities of false positives of claim 1 , wherein the Bloom filter data array size is made as large as possible for a given cache memory size.
3. The distributed, compressed Bloom filter Web server providing reduced probabilities of false positives of claim 1 , wherein arithmetic coding is employed to compress the Bloom filter data array to said transmission size.
4. The distributed, compressed Bloom filter Web server providing reduced probabilities of false positives of claim 1 , wherein said cache processor is further operative to store in its cache memory at least one decompressed Bloom filter data array representative of Web objects stored in a different Web server.
5. A method of reducing false positives in a network having distributed Web servers each storing information in cache memory as a Bloom filter data array representative of the information in its cache memory and broadcasting that data array to other Web servers periodically, comprising:
generating a Bloom filter data array representing a plurality of Web objects by choosing a number of hash functions and an array size for a Bloom filter, wherein said chosen array size is greater than a fixed transmission size and said number of hash functions is chosen to minimizes a rate of false positives in said Bloom filter data array after said Bloom filter data array has been compressed to said fixed transmission size, transmitted, and decompressed.
6. A distributed computer network, comprising:
a plurality of periodically intercommunicating distributed network nodes;
each node including a cache memory and a processor coupled to the cache memory operative to (1) represent in its memory contents as a Bloom filter data structure having a preselected number of hash functions and a preselected array size which have been chosen for a target compression size to optimize at least one of the rate of false positives of the Bloom filter representing the memory contents and the computational requirements of the preselected number of hash functions when said target transmission size is less than said preselected array size, to (2) compress the Bloom filter data structure to the target compression size using a predetermined compression algorithm, and to (3) broadcast the compressed Bloom filter data structure to at least one other node whenever the contents of its cache memory has changed.
7. The distributed computer network of claim 6 , wherein said nodes are Web proxy servers.
8. The distributed computer network of claim 6 , wherein said nodes are mobile or stationary agents in a network of mobile nodes, and the Web objects correspond to agent locations.
9. The distributed computer network of claim 6 , wherein said predetermined compression algorithm is arithmetic coding.
10. A method employing compressed Bloom filters for storing and transmitting data in a distributed network of nodes each having a processor coupled to a memory, comprising:
representing the data contents of a memory of a node as a compressed Bloom filter data structure stored in a memory of a node having a preselected number of hash functions and a preselected array size which have been chosen to optimize at least one of the rate of false positives of the Bloom filter representing the data contents and the computational requirements of the preselected number of hash functions for a transmission compression size when said transmission compression size is less than said preselected array size;
compressing the Bloom filter data structure to the transmission compression size; and
periodically transmitting the compressed Bloom filter data structure to at least one other node.
11. A method of storing data in memory for transmission, comprising:
representing the data in said memory as a compressed Bloom filter data structure having a preselected number of hash functions and a preselected array size which have been chosen for a target transmission compression size to optimize at least one of the rate of false positives of the Bloom filter data structure representing the data and the computational requirements of the preselected number of hash functions when said target transmission compression size is less than said preselected array size.
12. A distributed computer network, comprising:
a plurality of periodically intercommunicating distributed network nodes;
each node including a cache memory and processor coupled to the cache memory operative to (1) represent its memory contents as a Bloom filter data structure having a preselected number of hash functions and a preselected array size which have been chosen for a target rate of false positives to optimize at least one of a target compression size of the Bloom filter data structure and computational requirements of the preselected number of hash functions when said target compression size is less than said preselected array size, (2) compress the Bloom filter data structure to the target compression size using a predetermined compression algorithm, and (3) broadcast the compressed B loom filter data structure to at least one other node whenever the contents of its cache memory have changed.
13. The distributed computer network of claim 12 , wherein said nodes are web proxy servers.
14. The distributed computer network of claim 12 , wherein said nodes are mobile or stationary agents in a network of mobile nodes, and the Web objects correspond to agent locations.
15. The distributed computer network of claim 12 , wherein said predetermined compression algorithm is arithmetic coding.
16. A method employing Bloom filters for storing and transmitting data in a distributed network of nodes each having a processor coupled to a memory, comprising:
representing the data contents of the memory of a node as a Bloom filter data structure stored in memory of a node, said Bloom filter data structure having a preselected number of hash functions and a preselected array size which have been chosen to optimize a transmission compression size of the Bloom filter data structure for a given rate of false positives when said transmission compression size is less than said preselected array size;
compressing the Bloom filter data structure to the transmission compression size; and
periodically transmitting the compressed Bloom filter data structure to at least one other node.
17. A method of storing data in memory far transmission, comprising:
representing the data as a Bloom filter data structure in said memory, said Bloom filter data structure having a preselected number of hash functions and a preselected array size which have been chosen for a target rate of false positives to optimize a transmission compression size of the Bloom filter data structure when said transmission compression size is less than said preselected array size.