Bias reduction using data fusion of household panel data and transaction data
View Patent ↗In embodiments of the present invention, a method is described for reducing bias by data fusion of a household panel data and a loyalty card data. In embodiments, a method is provided for receiving a consumer panel dataset in a data fusion facility, receiving a consumer point-of-sale dataset in a data fusion facility, receiving a dimension dataset in a data fusion facility, fusing the datasets received in the data fusion facility into a new panel dataset based at least in part on an encryption key, estimating a consumer behavior using a first model based on the consumer panel dataset, estimating a consumer behavior using a second model based only on those consumers present in both the consumer panel dataset and the consumer point-of-sale dataset, and refining the first model based at least on the results of the second model.
1. A method comprising:
using a computer, storing a panel dataset in a data fusion facility, the panel dataset including panel data obtained from inputs of consumers who are members of panels, and the panel data including household purchasing behavior for a set of pre-defined buyer and shopper groups;
using a computer, storing a fact dataset of consumer behavior from retailer point-of-sale data in the data fusion facility, wherein the retailer point-of-sale data includes transactional data for one or more retail locations;
fusing the fact dataset and the panel dataset received in the data fusion facility into a new dataset based on a key that associates the fact dataset with the panel dataset according to consumers identified to be present in the panel dataset and in the fact dataset;
storing loyalty card data for a number of retailers containing exact measurements of household purchases in one or more venues of the retailer;
generating a corrected dataset by correcting for bias in the new dataset using the loyalty card data;
generating a public view of the corrected dataset containing bias-corrected aggregated data adjusted to reduce bias according to the loyalty card data while obfuscating disaggregated data in the new dataset to disguise a most accurate form of the loyalty card data from the number of retailers; and
creating a private view of the corrected data set for one of a number of retailers containing the bias-corrected aggregated data while replacing estimated household-level purchases with loyalty card data for the one of the number of retailers.
2. The method of claim 1 , wherein the consumer behavior is a product purchase.
3. The method of claim 1 , wherein the source of the fact dataset is a retail sales dataset.
4. The method of claim 1 , wherein the source of the fact dataset is a syndicated sales dataset.
5. The method of claim 4 , wherein the syndicated sales dataset is a scanner dataset.
6. The method of claim 4 , wherein the syndicated sales dataset is an audit dataset.
7. The method of claim 4 , wherein the syndicated sales dataset is a combined scanner-audit dataset.
8. The method of claim 1 , wherein the source of the fact dataset is point-of-sale data.
9. The method of claim 1 , wherein the source of the fact dataset is a syndicated causal dataset.
10. The method of claim 1 , wherein the source of the fact dataset is an internal shipment dataset.
11. The method of claim 1 , wherein the source of the fact dataset is an internal financials dataset.
12. The method of claim 1 wherein the source of the fact dataset is a retail channel dataset with limited data coverage of channels including some but not all of the retailers for which measurements are reported.
13. The method of claim 12 further comprising estimating household-level purchases for a population using a population database.
14. The method of claim 1 further comprising sharing the private view with one or more partners of the retailer.
15. A computer program product comprising computer executable code embodied in a non-transitory computer readable medium that, when executing on one or more computing devices, performs the steps of:
using a computer, storing a panel dataset in a data fusion facility, the panel dataset including panel data obtained from inputs of consumers who are members of panels, and the panel data including household purchasing behavior for a set of pre-defined buyer and shopper groups;
using a computer, storing a fact dataset of consumer behavior from retailer point-of-sale data in the data fusion facility, wherein the retailer point-of-sale data includes transactional data for one or more retail locations;
fusing the fact dataset and the panel dataset received in the data fusion facility into a new dataset based on a key that associates the fact dataset with the panel dataset according to consumers identified to be present in the panel dataset and in the fact dataset;
storing loyalty card data for a retailer containing exact measurements of household purchases in one or more venues of the retailer;
generating a corrected dataset correcting for bias in the new dataset using the loyalty card data;
generating a public view of the corrected dataset containing bias-corrected aggregated data adjusted to reduce bias according to the loyalty card data while obfuscating disaggregated data in the new dataset to disguise a most accurate form of the loyalty card data from the retailer; and
displaying a private view of the corrected data set to the retailer, the private view containing the bias-corrected aggregated data while replacing estimated household-level purchases with loyalty card data for the retailer.
16. The computer program product of claim 15 wherein the fact dataset includes point-of-sale data for one or more retailers.
17. The computer program product of claim 15 wherein the consumer behavior is a product purchase.
18. The computer program product of claim 15 wherein the source of the fact dataset is a retail sales dataset.
19. The computer program product of claim 15 wherein the source of the fact dataset is a syndicated sales dataset.
20. The computer program product of claim 19 wherein the syndicated sales dataset is at least one of a scanner dataset, an audit dataset, and a combined scanner-audit dataset.