System and method for web mining and clustering
View Patent ↗A method and system for web mining and clustering is described. The method includes receiving and dividing input data into a plurality of primitive datasets. Additionally, one or more combinations of the plurality of primitive datasets may be created. Further, a model for each primitive dataset in the plurality of primitive datasets and each of the one or more combinations of the plurality of primitive datasets may be generated. Subsequently, a cost associated with a model corresponding to each primitive dataset in the plurality of primitive datasets, and each of the one or more combinations of the plurality of primitive datasets may be computed. Further, a sum of the costs associated with the models corresponding to each primitive dataset in the plurality of primitive datasets may be compared with the cost associated with each model corresponding to each of the one or more combinations of the plurality of primitive datasets. Finally, the plurality of primitive datasets may be partitioned into one or more clusters based on the comparison of the costs such that each primitive dataset is a part of a cluster in the one or more clusters or a stand-alone primitive dataset.
1. A method of web mining, comprising:
via one or more computer processing devices:
receiving input data;
dividing the input data into a plurality of primitive datasets;
creating one or more combinations of the plurality of primitive datasets;
generating a model for each primitive dataset in the plurality of primitive datasets and each of the one or more combinations of the plurality of primitive datasets;
computing a cost associated with a model corresponding to each primitive dataset in the plurality of primitive datasets, and each of the one or more combinations of the plurality of primitive datasets;
comparing a sum of the costs associated with the models corresponding to each primitive dataset in the plurality of primitive datasets with the cost associated with each model corresponding to each of the one or more combinations of the plurality of primitive datasets; and
partitioning the plurality of primitive datasets into one or more clusters based on the comparison of the costs such that each primitive dataset is a part of a cluster in the one or more clusters or a stand-alone primitive dataset.
2. The method of claim 1 , wherein partitioning the plurality of primitive datasets into one or more clusters comprises grouping the plurality of primitive datasets into the one or more clusters upon determining that the sum of the costs associated with the models corresponding to each primitive dataset in the plurality of primitive datasets is greater than the cost associated with each model corresponding to each of the one or more combinations of the plurality of primitive datasets.
3. The method of claim 1 , wherein partitioning the plurality of primitive datasets into one or more clusters comprises forming the stand-alone primitive dataset when the sum of the costs associated with the models corresponding to each primitive dataset in the plurality of primitive datasets is less than the cost associated with each model corresponding to each of the one or more combinations of the plurality of primitive datasets.
4. The method of claim 1 , wherein the input data comprises an input data stream, log data, click-stream data, web server data, web transaction data, or combinations thereof.
5. The method of claim 1 , wherein generating the model for each primitive dataset in the plurality of primitive datasets and each of the one or more combinations of the plurality of primitive datasets comprises applying a compression algorithm.
6. The method of claim 5 , wherein the compression algorithm is a grammar-based algorithm based on minimum description length principles.
7. The method of claim 1 , wherein computing a cost associated with a model corresponding to each primitive dataset in the plurality of primitive datasets and each of the one or more combinations of the plurality of primitive datasets comprises determining a number of bits required to represent each primitive dataset in the plurality of primitive datasets and the corresponding models and each of the one or more combinations of the plurality of primitive datasets and the corresponding models.
8. The method of claim 1 , further comprising:
determining a pattern corresponding to each of the one or more clusters or the stand-alone primitive dataset; and
displaying an advertisement, recommending content, personalizing a website, or combinations thereof, based on the determined pattern.
9. The method of claim 8 , wherein determining the pattern corresponding to each of the one or more clusters or each stand-alone primitive dataset comprises determining a typical characteristic, an atypical characteristic, or both the typical characteristic and the atypical characteristic corresponding to each of the one or more clusters or each stand-alone primitive dataset.
10. The method of claim 9 , further comprising displaying the advertisement, recommending content, personalizing the website, or combinations thereof, based on the determined typical characteristic, the determined atypical characteristic, or a combination thereof, corresponding to each of the one or more clusters or each stand-alone primitive dataset.
11. A web mining and clustering system, comprising:
one or more computer processing devices, comprising:
a pre-processor that receives input data and generates a plurality of primitive datasets and one or more combinations of the plurality of primitive datasets using the input data;
a grammar generator that generates a model corresponding to each primitive dataset in the plurality of primitive datasets, and the one or more combinations of the plurality of primitive datasets, wherein the grammar generator is operationally coupled to the pre-processor;
a grammar applicator that computes a cost associated with the model corresponding to each primitive dataset in the plurality of primitive datasets, and the one or more combinations of the plurality of primitive datasets; and
a classifier that partitions the plurality of primitive datasets into one or more clusters based on a comparison of a sum of the costs associated with the models corresponding to each primitive dataset in the plurality of primitive datasets with the cost associated with each model corresponding to each of the one or more combinations of the plurality of primitive datasets such that each primitive dataset is a part of a cluster in the one or more clusters or a stand-alone primitive dataset.
12. The web mining and clustering system of claim 11 , further comprising an input database that stores the plurality of primitive datasets and the one or more combinations of the plurality of primitive datasets.
13. The web mining and clustering system of claim 11 , wherein the input data comprises an input data stream, log data, click-stream data, web server data, web transaction data, or combinations thereof.
14. The web mining and clustering system of claim 11 , wherein the grammar generator generates the models corresponding to each primitive dataset in the plurality of primitive datasets and the one or more combinations of the plurality of primitive datasets by applying a compression algorithm.
15. The web mining and clustering system of claim 14 , wherein the compression algorithm is a grammar-based algorithm based on minimum description length principles.
16. The system of claim 11 , wherein the grammar applicator computes a cost associated with a model corresponding to each primitive dataset in the plurality of primitive datasets and each of the one or more combinations of the plurality of primitive datasets based on a number of bits required to represent each primitive dataset in the plurality of primitive datasets and the corresponding models and each of the one or more combinations of the plurality of primitive datasets and the corresponding models.
17. The web mining and clustering system of claim 11 , further comprising a post-processor that:
determines a pattern corresponding to each of the one or more clusters; and
displays an advertisement, recommends content, personalizes a website, or combinations thereof, based on the determined pattern.
18. The web mining and clustering system of claim 17 , wherein the post-processor determines a typical characteristic, an atypical characteristic, or both the typical characteristic and the atypical characteristic corresponding to each of the one or more clusters or each stand-alone primitive dataset.
19. The method of claim 18 , wherein the post-processor displays the advertisement, recommends content, personalizes the website, or combinations thereof, based on the determined typical characteristic, the determined atypical characteristic, or a combination thereof, corresponding to each of the one or more clusters or each stand-alone primitive dataset.
20. A non-transient computer readable media embodying instructions for web mining and clustering, which when executed by a processor cause the computer to execute the steps of:
receiving input data;
dividing the input data into a plurality of primitive datasets;
creating one or more combinations of the plurality of primitive datasets;
generating a model for each primitive dataset in the plurality of primitive datasets and each of the one or more combinations of the plurality of primitive datasets;
computing a cost associated with a model corresponding to each primitive dataset in the plurality of primitive datasets, and each of the one or more combinations of the plurality of primitive datasets;
comparing a sum of the costs associated with the models corresponding to each primitive dataset in the plurality of primitive datasets with the cost associated with each model corresponding to each of the one or more combinations of the plurality of primitive datasets; and
partitioning the plurality of primitive datasets into one or more clusters based on the comparison of the costs such that each primitive dataset is a part of a cluster in the one or more clusters or a stand-alone primitive dataset.