Using a bloom filter in a web analytics application
View Patent ↗Embodiments of the present invention are directed to the use of a Bloom Filter in a web analytics application. Large sets of data, obtained from on and offline sources, may be analyzed using a Bloom Filter to find desired patterns within the data. The use of a Bloom Filter in the analysis allows for a more efficient use of resources to perform the analysis.
1. A non-transitory computer readable medium embodying a set of instructions that when executed by a processor cause the processor to perform the steps comprising:
identifying a first set of data elements that satisfy a first criteria;
entering identification information for each element of the first set of data elements into a bloom filter;
applying the bloom filter to a set of user activities; and
storing each element from the set of user activities that is included in the bloom filter in a results set;
wherein the first set of data elements comprises a user segment, a user segment including a set of user identifiers that identify specific users;
wherein the step of entering identification information for each element of the first set of data elements into a bloom filter further comprises:
processing each user identifier of the set of user identifiers with at least one hash function, said hash function mapping the user identifier to a bit osition within the bloom filter; and
setting the bit position identified by the at least one hash function to a value indicating true;
wherein the at least one hash function further comprises:
generating a 32 bit hash value of the user identifier;
calculating the modulus of the 32 bit hash value with a size of the bloom filter, a result of the calculation indicating the bit position within the bloom filter that should be set to true; and
setting the bit position of the bloom filter corresponding to the result to true.
2. The tangible computer readable medium of claim 1 wherein the at least one hash function comprises six hash functions.
3. The tangible computer readable medium of claim 2 wherein the second set of data elements comprises activities performed by the second set of users.
4. The tangible computer readable medium of claim 3 further comprising:
identifying a second set of data elements that satisfy a second criteria; and entering
identification information for each element of the second set of data elements into the bloom filter.
5. The tangible computer readable medium of claim 4 wherein applying the bloom filter to the set of user activities further comprises:
processing each user identifier associated with each user activity of the set of user activities with the six hash functions; and
comparing the bit positions determined by the six hash functions with the corresponding bit positions in the bloom filter, wherein a match of all six bit positions indicates the user identifier associated with the user activity is included in the bloom filter.
6. The tangible computer readable medium of claim 4 wherein the bloom filter is 64 megabytes long.
7. A system comprising:
a processor; and
a memory coupled to the processor, the memory comprising a set of instructions that when executed by the processor cause the processor to perform the steps of: identifying a first set of data elements that satisfy a first criteria; entering identification information for each element of the first set of data elements into a bloom filter; applying the bloom filter to a set of user activities; and storing each element from the set of user activities that is included in the bloom filter in a results set;
wherein the memory further comprises instructions for processing each user identifier of the set of user identifiers with at least one hash function, said hash function mapping the user identifier to a bit position within the bloom filter; and setting the bit position identified by the at least one hash function to a value indicating true;
wherein the at least one hash function comprises instructions for generating a 32 bit hash value of the user identifier; calculating the modulus of the 32 bit hash value with a size of the bloom filter, a result of the calculation indicating the bit position within the bloom filter that should be set to true; and setting the bit position of the bloom filter corresponding to the result to true.
8. The system of claim 7 wherein the at least one hash function comprises six hash functions.
9. The system of claim 8 wherein applying the bloom filter to the set of user activities further comprises instructions for processing each user identifier associated with each user activity of the set of user activities with the six hash functions; and comparing the bit positions determined by the six hash functions with the corresponding bit positions in the bloom filter, wherein a match of all six bit positions indicates the user identifier associated with the user activity is included in the bloom filter.
10. A method for processing user activity data comprising:
identifying a first set of data elements that satisfy a first criteria;
entering identification information for each element of the first set of data elements into a bloom filter;
applying the bloom filter to a set of user activities;
storing each element from the set of user activities that is included in the bloom filter in a results set;
processing, by a processor, each user identifier of the set of user identifiers with at least one hash function, said hash function mapping the user identifier to a bit position within the bloom filter; and
setting the bit position identified by the at least one hash function to a value indicating true;
wherein the at least one hash function further comprises:
generating a 32 bit hash value of the user identifier;
calculating the modulus of the 32 bit hash value with a size of the bloom filter, a result of the calculation indicating the bit position within the bloom filter that should be set to true; and
setting the bit position of the bloom filter corresponding to the result to true.
11. The method of claim 10 wherein the at least one hash function comprises six hash functions.
12. The method of claim 11 further comprising:
processing each user identifier associated with each user activity of the set of user activities with the six hash functions; and
comparing the bit positions determined by the six hash functions with the corresponding bit positions in the bloom filter, wherein a match of all six bit positions indicates the user identifier associated with the user activity is included in the bloom filter.