Method and system for processing measurement data for website statistics
View Patent ↗Methods and systems for storing web-site visitor data in a specific bucket among a plurality of buckets based on a unique visitor identifier. The methods and systems may dynamically change the number of buckets based on certain conditions, such as when one of the buckets reaches a predetermined size. Several processors may be used in parallel to process the visitor data from two or more of the buckets simultaneously, for example to generate or obtain report data. The systems and methods may aggregate the report data into a result table.
1. A system for processing measurement data for website statistics, comprising:
a plurality of processors, at least some of the plurality of processors are implemented at least partially by hardware; and
a memory system, operably connected to the plurality of processors, containing instructions that cause the plurality of processors to perform operations comprising:
receiving measurement data from a website, wherein the measurement data comprises a plurality of visitor data indicating at least activities associated with corresponding visitors of the website;
selecting a first bucket of a plurality of buckets based on a hash value calculated from a unique user identifier of a first unique visitor, wherein the first bucket is associated with a first plurality of hash values, wherein a bucket contains at least a log file associated with at least an activity of a particular visitor;
storing all visitor data for the first unique visitor in the first bucket of the plurality of buckets;
selecting a second bucket of the plurality of buckets based on a hash value calculated from a unique user identifier of a second unique visitor, wherein the second bucket is associated with a second plurality of hash values different from the first plurality of hash values;
storing all visitor data for the second unique visitor in the second bucket of the plurality of buckets;
dynamically changing an amount of the plurality of buckets when at least one of the first bucket and the second bucket reaches a predetermined size,
wherein the amount is based on a size of the website;
receiving a report request, from a client device, that comprises a query for report data on the first unique visitor and the second unique visitor;
in response to receiving the report request, processing the plurality of visitor data stored in the plurality of buckets using the plurality of processors to obtain the report data, wherein:
the first bucket and all the visitor data for the first unique visitor are processed by a first processor of the plurality of processors;
the second bucket and all the visitor data for the second unique visitor are processed by a second processor of the plurality of processors; and
the first processor and the second processor are processing the first and second buckets simultaneously; and
providing the report data to the client device.
2. The system according to claim 1 , wherein a bucket comprises one or more labels which are grouped and stored together in the bucket.
3. The system according to claim 1 , the operations further comprising aggregating the report data into a result table.
4. A method for processing measurement data for website statistics, comprising:
receiving measurement data from a website, wherein the measurement data comprises a plurality of visitor data indicating at least activities associated with corresponding visitors of the website;
selecting a first bucket of a plurality of buckets based on a hash value calculated from a unique user identifier of a first unique visitor, wherein the first bucket is associated with a first plurality of hash values, wherein a bucket contains at least a log file associated with at least an activity of a particular visitor;
storing all visitor data for the first unique visitor in the first bucket of the plurality of buckets;
selecting a second bucket of the plurality of buckets based on a hash value calculated from a unique user identifier of a second unique visitor, wherein the second bucket is associated with a second plurality of hash values different from the first plurality of hash values;
storing all visitor data for the second unique visitor in the second bucket of the plurality of buckets;
dynamically changing an amount of the plurality of buckets when at least one of the first bucket and the second bucket reaches a predetermined size, wherein the amount is based on a size of the website;
receiving a report request, from a client device, that comprises a query for report data on the first unique visitor and the second unique visitor;
in response to receiving the report request, processing the plurality of visitor data stored in the plurality of buckets using the plurality of processors to obtain the report data, wherein:
the first bucket and all the visitor data for the first unique visitor are processed by a first processor of the plurality of processors;
the second bucket and all the visitor data for the second unique visitor are processed by a second processor of the plurality of processors; and
the first processor and the second processor are processing the first and second buckets simultaneously; and
providing the report data to the client device.
5. The method according to claim 4 , further comprising grouping one or more labels to obtain a group of labels and storing the group of labels in a bucket.
6. The method according to claim 4 , further comprising aggregating the report data into a result table.
7. A computer program element having instructions stored on non-transitory computer readable media for processing measurement data for website statistics which, when executed by a plurality of processors, is adapted to implement a method comprising:
receiving measurement data from a website, wherein the measurement data comprises a plurality of visitor data indicating at least activities associated with corresponding visitors of the website;
selecting a first bucket of a plurality of buckets based on a hash value calculated from a unique user identifier of a first unique visitor, wherein the first bucket is associated with a first plurality of hash values, wherein a bucket contains at least a log file associated with at least an activity of a particular visitor;
storing all visitor data for the first unique visitor in the first bucket of the plurality of buckets;
selecting a second bucket of the plurality of buckets based on a hash value calculated from a unique user identifier of a second unique visitor, wherein the second bucket is associated with a second plurality of hash values different from the first plurality of hash values;
storing all visitor data for the second unique visitor in the second bucket of the plurality of buckets;
dynamically changing an amount of the plurality of buckets when at least one of the first bucket and the second bucket reaches a predetermined size, wherein the amount is based on a size of the website;
receiving a report request, from a client device, that comprises a query for report data on the first unique visitor and the second unique visitor;
in response to receiving the report request, processing the plurality of visitor data stored in the plurality of buckets using the plurality of processors to obtain the report data, wherein:
the first bucket and all the visitor data for the first unique visitor are processed by a first processor of the plurality of processors;
the second bucket and all the visitor data for the second unique visitor are processed by a second processor of the plurality of processors; and
the first processor and the second processor are processing the first and second buckets simultaneously; and
providing the report data to the client device.
8. The computer program element according to claim 7 , further adapted for grouping one or more labels to obtain a group of labels and storing the group of labels in a bucket.
9. The computer program element according to claim 7 , further adapted for the method further comprising aggregating the report data into a result table.