Process and system for integrating information from disparate databases for purposes of predicting consumer behavior
A process and system for integrating information stored in at least two disparate databases. The stored information includes consumer transactional information. According to the process and system, at least one qualitative variable which is common to each database is identified, and then transformed into one or more quantitative variables. The consumer transactional information in each database is then converted into converted information in terms of the quantitative variables. Thereafter, an integrated database is formed for predicting consumer behavior by combining the converted information from the disparate databases.
1. A method of integrating and modeling information stored in a plurality of disparate databases, the method comprising:
identifying at least one qualitative variable which is common to each database of the plurality of disparate databases;
transforming the at least one qualitative variable into one or more quantitative variables;
using a processing device, converting a portion of the information stored in each database of the plurality of disparate databases according to the one or more quantitative variables to form converted information;
using a processing device, performing a cluster analysis using converted information from each database of the plurality of disparate databases to form a plurality of clusters;
using a processing device, linking through the plurality of clusters the plurality of disparate databases to form an integrated database; and
using a processing device, creating a behavioral model, using corresponding data of the plurality of clusters of the integrated database, for predicting individual behavior.
2. The process of claim 1 , further comprising the steps of:
selecting at least one discriminating subset of the at least one quantitative variable to create statistical drivers; and
creating clusters by assigning each consumer in the integrated database to at least one of the subsets.
3. The method of claim 2 , further comprising:
converting one or more clusters of the plurality of clusters into at least one supercluster; and
assigning the plurality of individuals to a corresponding cluster or supercluster using data from each database of the plurality of disparate databases.
4. The method of claim 3 , wherein the at least one qualitative variable is a merchant and the one or more quantitative variable comprises one or more of the following:
mean number of transactions per person for the merchant,
mean amount per transaction for the merchant,
mean household income of shoppers shopping at the merchant, and
mean proportion of the shoppers for a particular area of the merchant.
5. The method of claim 4 , further comprising:
prior to forming the integrated database, weighting data of the plurality of disparate databases to adjust for differences in size and in time encompassed.
6. The method of claim 4 , wherein the selecting step further comprises:
identifying one or more industries which have discriminating consumers and grouping selected merchants into the at least one discriminating subset.
7. The method of claim 1 , wherein the information stored in the plurality of disparate databases further comprises consumer transactional information and has instances of purchasing behavior by consumers.
8. The method of claim 7 , wherein at least one of the disparate databases includes joint account information for at least two consumers, and wherein the method further comprises:
determining a consumer of the at least two consumers who generated at least a portion of the consumer transactional information.
9. A system for integrating and modeling information stored in a plurality of disparate databases, the system comprising:
an integrating arrangement which:
identifies at least one qualitative variable which is common to each database of the plurality of disparate databases,
transforms the at least one qualitative variable into one or more quantitative variables,
converts a portion of the information stored in each database of the plurality of disparate databases according to the one or more quantitative variables to form converted information,
performs a cluster analysis using converted information from each database of the plurality of disparate databases to form a plurality of clusters,
links through the plurality of clusters the plurality of disparate databases to form an integrated database; and
creates a behavioral model, using corresponding data of the plurality of clusters of the integrated database, for predicting individual behavior.
10. The system of claim 9 , wherein the integrating arrangement selects at least one discriminating subset of the one or more quantitative variables to create one or more statistical drivers, and evaluates a plurality of individuals represented in the plurality of disparate databases using the one or more statistical drivers.
11. The system of claim 10 , wherein the integrating arrangement converts one or more clusters of the plurality of clusters into at least one supercluster, and assigns the plurality of individuals to a corresponding cluster or supercluster using data from each database of the plurality of disparate databases.
12. The system of claim 11 , wherein the at least one qualitative variable is a merchant and the one or more quantitative variable comprises one or more of the following:
mean number of transactions per person for the merchant,
mean amount per transaction for the merchant,
mean household income of shoppers shopping at the merchant, and
mean proportion of the shoppers for a particular area of the merchant.
13. The system of claim 12 , wherein the integrating arrangement weights data of the plurality of disparate databases to adjust for differences in size and in time encompassed prior to the formation of the integrated database.
14. The system of claim 12 , wherein the integrating arrangement selects the at least one discriminating subset by identifying one or more industries which have discriminating consumers and grouping selected merchants into the at least one discriminate subset.
15. The system of claim 9 , wherein the information stored in the plurality of disparate databases further comprises consumer transactional information and has instances of purchasing behavior by consumers.
16. The system of claim 9 , wherein at least one of said disparate databases includes joint account information for at least two consumers, and wherein the integrating database determines a consumer of the at least two consumers who generated at least a portion of the consumer transactional information.
17. A method for creating a behavioral model from information stored in a plurality of disparate databases, the method comprising:
determining a plurality of variables from each database, and converting the plurality of variables to form a plurality of statistical drivers, at least a portion of the plurality of statistical drivers common to each database of the plurality of disparate databases;
using a processing device, performing a first cluster analysis using corresponding data of the plurality of statistical drivers common to each database of the plurality of disparate databases to create a plurality of simultaneous cluster solutions across all databases of the plurality of disparate databases;
using a processing device, linking through at least one simultaneous cluster solution of the plurality of simultaneous cluster solutions the information stored in the plurality of disparate databases; and
validating at least one simultaneous cluster solution of the plurality of simultaneous cluster solutions as a discriminatory behavioral model for predicting individual behavior.
18. The method of claim 17 , further comprising:
converting the information stored in the plurality of disparate databases according to the plurality of statistical drivers to create the corresponding data of the plurality of statistical drivers.
19. The method of claim 17 , wherein the determination of the plurality of variables further comprises:
selecting at least one qualitative variable which is common to each database of the plurality of disparate databases; and
transforming the at least one qualitative variable from each database to a plurality of quantitative variables.
20. The method of claim 19 , further comprising:
performing a principal components analysis on the plurality of quantitative variables using the information stored in each database of the plurality of disparate databases to create the plurality of statistical drivers.
21. The method of claim 20 , further comprising:
standardizing the plurality of quantitative variables;
transforming the standardized plurality of quantitative variables to be substantially orthogonal; and
differentially weighting the orthogonal, standardized plurality of quantitative variables to form the plurality of statistical drivers.
22. The method of claim 17 , further comprising:
evaluating corresponding discrimination power of the plurality of statistical drivers using a second cluster analysis.
23. The method of claim 17 , wherein at least one database of the plurality of disparate databases stores behavioral and attitudinal information and wherein at least one database of the plurality of disparate databases stores consumer transactional information.
24. The method of claim 17 , wherein at least one database of the plurality of disparate databases stores behavioral and attitudinal information and wherein at least one database of the plurality of disparate databases stores media consumption information.
25. The method of claim 17 , further comprising:
describing each cluster of a plurality of clusters of the validated simultaneous cluster solution using information stored in at least one database of the plurality of disparate databases.
26. The method of claim 17 , further comprising:
creating a plurality of superclusters from the validated simultaneous cluster solution.
27. The method of claim 17 , wherein the validation step further comprises:
determining whether the at least one simultaneous cluster solution provides corresponding discrimination on a plurality of other variables which are not statistical drivers in the plurality of disparate databases.
28. The method of claim 17 , wherein the validation step further comprises:
determining whether the at least one simultaneous cluster solution provides corresponding discrimination separately within each database of the plurality of disparate databases.
29. A system for creating a behavioral model from information stored in a plurality of disparate databases, the system comprising:
a storage device storing data from one or more of the plurality of disparate databases; and
a processing device coupled to the storage device, the processing device adapted to determine a plurality of variables from each database and convert the plurality of variables to form a plurality of statistical drivers, at least a portion of the plurality of statistical drivers common to each database of the plurality of disparate databases; to perform a first cluster analysis using corresponding data of the plurality of statistical drivers common to each database of the plurality of disparate databases to create a plurality of simultaneous cluster solutions across all databases of the plurality of disparate databases; to link through at least one simultaneous cluster solution of the plurality of simultaneous cluster solutions the information stored in the plurality of disparate databases; and to validate at least one simultaneous cluster solution of the plurality of simultaneous cluster solutions as a discriminatory behavioral model for predicting individual behavior.
30. The system of claim 29 , wherein the processing device is further adapted to convert the information stored in the plurality of disparate databases according to the plurality of statistical drivers to create the corresponding data of the plurality of statistical drivers.
31. The system of claim 29 , wherein the processing device is further adapted to determine the plurality of variables by selecting at least one qualitative variable which is common to each database of the plurality of disparate databases; and transforming the at least one qualitative variable from each database to a plurality of quantitative variables.
32. The system of claim 31 , wherein the processing device is further adapted to perform a principal components analysis on the plurality of quantitative variables using the information stored in each database of the plurality of disparate databases to create the plurality of statistical drivers.
33. The system of claim 31 , wherein the processing device is further adapted to standardize the plurality of quantitative variables; to transform the standardized plurality of quantitative variables to be substantially orthogonal; and to differentially weight the orthogonal, standardized plurality of quantitative variables to form the plurality of statistical drivers.
34. The system of claim 31 , wherein the processing device is further adapted to evaluate corresponding discrimination power of the plurality of statistical drivers using a second cluster analysis.
35. The system of claim 29 , wherein at least one database of the plurality of disparate databases stores behavioral and attitudinal information and wherein at least one database of the plurality of disparate databases stores consumer transactional information.
36. The system of claim 29 , wherein at least one database of the plurality of disparate databases stores behavioral and attitudinal information and wherein at least one database of the plurality of disparate databases stores media consumption information.
37. The system of claim 29 , wherein the processing device is further adapted to describe each cluster of a plurality of clusters of the validated simultaneous cluster solution using information stored in at least one database of the plurality of disparate databases.
38. The system of claim 29 , wherein the processing device is further adapted to create a plurality of superclusters from the validated simultaneous cluster solution.
39. The system of claim 29 , wherein the processing device is further adapted to determine whether the at least one simultaneous cluster solution provides corresponding discrimination on a plurality of other variables which are not statistical drivers in the plurality of disparate databases.
40. The system of claim 29 , wherein the processing device is further adapted to determine whether the at least one simultaneous cluster solution provides corresponding discrimination separately within each database of the plurality of disparate databases.