Heuristic clustering
Methods and apparatus are disclosed regarding an e-commerce system that places customers into a plurality of clusters and tailors services provided to a customer based on the cluster in which the customer is placed. In one embodiment, the e-commerce system defines the clusters based on purchase history data for customers having sufficient purchase history data. The e-commerce system then places customers without sufficient purchase history data into one of the defined clusters based on demographic data for the customer and demographic data for the customers in the cluster.
1 . A method, the method comprising:
splitting a plurality of records into a first group and a second group;
generating a matrix associated with data of the first group, wherein the matrix comprises a Customer-Item (CI) matrix that is standardized column-wise using bin-quantiles standardization (BQS) to produce a standardized transaction space with values between 0 and 1;
applying principal component analysis to the standardized transaction space to obtain a reduced matrix for the first group;
generating, in accordance with a clustering large applications (CLARA) procedure, a plurality of samples of the first group according to the reduced matrix;
partitioning each of the plurality of samples of the first group to reduce dissimilarities;
applying a partitioning-around-medoids (PAM) algorithm using Euclidean distance to each sample;
computing for each medoid set a sum of distances of all points to a nearest medoid;
selecting the medoid set having the smallest sum of distances;
clustering the first group into a plurality of clusters according to the selected medoid set; and
placing a record of the second group into a particular cluster, of the plurality of clusters, according to demographic data of the record and demographic data of the particular cluster.
2 . The method of claim 1 , comprising validating the plurality of clusters prior to placing the record into the particular cluster.
3 . The method of claim 1 , comprising recalibrating a quantity of dimensions of the reduced matrix until a cluster validation determines the plurality of clusters are a valid clustering of the first group.
4 . The method of claim 1 , comprising recalibrating a quantity of clusters used by the clustering until a cluster validation determines the plurality of clusters are a valid clustering of the first group.
5 . The method of claim 1 , comprising:
generating silhouette numbers for the first group, and
determining, according to the silhouette numbers for the first group, that the plurality of clusters are a valid clustering of the first group.
6 . The method of claim 1 , wherein:
partitioning comprises calculating distances between records in the first group,
a larger distance corresponds to a greater dissimilarity, and
each cluster, in the plurality of clusters, is defined according to a smallest sum of distances.
7 . The method of claim 1 , wherein:
partitioning comprises determining a particular record according to a sum of dissimilarities between the particular record and all other records of the first group, and
the sum associated with the particular record is a minimum.
8 . A non-transitory computer readable medium, comprising a plurality of instructions, that in response to being executed, result in a computing device:
splitting a plurality of records into a first group and a second group;
generating a matrix associated with data of the first group, wherein the matrix comprises a Customer-Item (CI) matrix that is standardized column-wise using bin-quantiles standardization (BQS) to produce a standardized transaction space with values between 0 and 1;
applying principal component analysis to the standardized transaction space to obtain a reduced matrix for the first group;
generating, in accordance with a clustering large applications (CLARA) procedure, a plurality of samples of the first group according to the reduced matrix;
partitioning each of the plurality of samples of the first group to reduce dissimilarities;
applying a partitioning-around-medoids (PAM) algorithm using Euclidean distance to each sample;
computing for each medoid set a sum of distances of all points to a nearest medoid;
selecting the medoid set having the smallest sum of distances;
clustering the first group into a plurality of clusters according to the selected medoid set; and
placing a record of the second group into a particular cluster, of the plurality of clusters, according to demographic data of the record and demographic data of the particular cluster.
9 . The non-transitory computer readable medium of claim 8 , wherein in response to being executed, the plurality of instructions result in the computing device validating the plurality of clusters prior to placing the record into the particular cluster.
10 . The non-transitory computer readable medium of claim 8 , wherein in response to being executed, the plurality of instructions result in the computing device recalibrating a quantity of dimensions of the reduced matrix until a cluster validation determines the plurality of clusters are a valid clustering of the first group.
11 . The non-transitory computer readable medium of claim 8 , wherein in response to being executed, the plurality of instructions result in the computing device recalibrating a quantity of clusters used by the clustering until a cluster validation determines the plurality of clusters are a valid clustering of the first group.
12 . The non-transitory computer readable medium of claim 8 , wherein in response to being executed, the plurality of instructions result in the computing device:
generating silhouette numbers for the first group, and
determining, according to the silhouette numbers for the first group, that the plurality of clusters are a valid clustering of the first group.
13 . The non-transitory computer readable medium of claim 8 , wherein:
partitioning comprises calculating distances between records of the first group,
a larger distance corresponds to a greater dissimilarity, and
each cluster, in the plurality of clusters, is defined according to a smallest sum of distances.
14 . The non-transitory computer readable medium of claim 8 , wherein:
partitioning comprises determining a particular record according to a sum of dissimilarities between the particular record and all other records of the first group, and
the sum associated with the particular record is a minimum.
15 . A system, the system comprising:
one or more processors configured to:
split a plurality of records into a first group and a second group;
generate a matrix associated with data of the first group, wherein the matrix comprises a Customer-Item (CI) matrix that is standardized column-wise using bin-quantiles standardization (BQS) to produce a standardized transaction space with values between 0 and 1;
apply principal component analysis to the standardized transaction space to obtain a reduced matrix for the first group;
generate, in accordance with a clustering large applications (CLARA) procedure, a plurality of samples of the first group according to the reduced matrix;
partition each of the plurality of samples of the first group to reduce dissimilarities;
apply a partitioning-around-medoids (PAM) algorithm using Euclidean distance to each sample;
compute for each medoid set a sum of distances of all points to a nearest medoid;
select the medoid set having the smallest sum of distances;
cluster the first group into a plurality of clusters according to the selected medoid set; and
place a record of the second group into a particular cluster, of the plurality of clusters, according to demographic data of the record and demographic data of the particular cluster.
16 . The system of claim 15 , wherein the one or more processors are configured to validate the plurality of clusters prior to placing the record into the particular cluster.
17 . The system of claim 15 , wherein the one or more processors are configured to recalibrate a quantity of dimensions of the reduced matrix until a cluster validation determines the plurality of clusters are a valid clustering of the first group.
18 . The system of claim 15 , wherein the one or more processors are configured to recalibrate a quantity of clusters used by the clustering until a cluster validation determines the plurality of clusters are a valid clustering of the first group.
19 . The system of claim 15 , wherein the one or more processors are configured to:
generate silhouette numbers for the first group, and
determine, according to the silhouette numbers for the first group, that the plurality of clusters are a valid clustering of the first group.
20 . The system of claim 15 , wherein:
partitioning comprises calculating distances between records of the first group,
a larger distance corresponds to a greater dissimilarity, and
each cluster, in the plurality of clusters, is defined according to a smallest sum of distances.
21 . The system of claim 15 , wherein:
partitioning comprises determining a particular record according to a sum of dissimilarities between the particular record and all other records of the first group, and the sum associated with the particular record is a minimum.