System and method for unsupervised abstraction of sensitive data for consortium sharing

1. A computer-implemented method for generating a standard customer profile in a data processing system comprising a processing device and a memory comprising instructions which are executed by the processing device, the method comprising:

receiving customer data from a plurality of computing devices over a network, the customer data including information for a plurality of customers to a plurality of entities;

performing, by the processing device, unsupervised learning on the customer data to produce a plurality of clusters of customers with a plurality of features in common;

determining, by the processing device, that one or more of the plurality of clusters each represent a standard customer and storing a plurality of standard customer profiles based on the determined standard customers, wherein the standard customer profiles comprise a plurality of data distributions for the plurality of features in common;

providing the plurality of standard customer profiles to each of the plurality of computing devices for generating synthetic transaction data based on the standard customer by:

providing the plurality of data distributions as a goal,

performing a plurality of iterations to generate synthetic transaction data, wherein the plurality of iterations is performed until a degree of similarity of the synthetic transaction data relative to the plurality of data distributions is higher than a predefined threshold,

in each iteration:

conducting an action including a plurality of simulated transactions,

comparing the action with the goal,

providing a feedback associated with the action based on a degree of similarity relative to the goal, and

adjusting a policy based on the feedback;

training, using the synthetic transaction data, a predictive model to identify an abnormal customer behavior; and

detecting, using the trained predictive model, the abnormal customer behavior in real transaction data.

2. The method of claim 1 , wherein the information for the plurality of customers comprises identifying information and transaction information.

3. The method of claim 1 , further comprising filtering the customer data prior to performing unsupervised learning.

4. The method of claim 3 , wherein the filtering comprises an RFM analysis to group customers.

5. The method of claim 1 , wherein the performing unsupervised learning comprises clustering customers based on a feature in common and repeating unsupervised learning to form sub-clusters of customers based on the plurality of features in common.

6. The method of claim 1 , wherein determining that a cluster represents a standard customer comprises applying one or more rules.

7. The method of claim 6 , wherein the one or more rules comprise a size determination indicating a minimum or maximum number of customers in a cluster that is determined to be a standard customer.

8. An abstraction system comprising a processing device and a memory comprising instructions which are executed by the processing device for generating a standard customer profile in a data processing system configured to:

receiving customer data from a plurality of computing devices over a network, the customer data including information for a plurality of customers to a plurality of entities;

performing, by the processing device, unsupervised learning on the customer data to produce a plurality of clusters of customers with a plurality of features in common;

determining, by the processing device, that one or more of the plurality of clusters each represent a standard customer and storing a plurality of standard customer profiles based on the determined standard customers, wherein the standard customer profiles comprise a plurality of data distributions for the plurality of features in common;

providing the plurality of standard customer profiles to each of the plurality of computing devices for generating synthetic transaction data based on the standard customer by:

providing the plurality of data distributions as a goal,

performing a plurality of iterations to generate synthetic transaction data, wherein the plurality of iterations is performed until a degree of similarity of the synthetic transaction data relative to the plurality of data distributions is higher than a predefined threshold,

in each iteration:

conducting an action including a plurality of simulated transactions,

comparing the action with the goal,

providing a feedback associated with the action based on a degree of similarity relative to the goal, and

adjusting a policy based on the feedback;

training, using the synthetic transaction data, a predictive model to identify an abnormal customer behavior; and

detecting, using the trained predictive model, the abnormal customer behavior in real transaction data.

9. The system of claim 8 , wherein the information for the plurality of customers comprises identifying information and transaction information.

10. The system of claim 8 , wherein the data processing system is further configured to:

filtering the customer data prior to performing unsupervised learning.

11. The system of claim 10 , wherein the filtering comprises an RFM analysis to group customers.

12. The system of claim 8 , wherein the performing unsupervised learning comprises clustering customers based on a feature in common and repeating unsupervised learning to form sub-clusters of customers based on the plurality of features in common.

13. The system of claim 8 , wherein determining that a cluster represents a standard customer comprises applying one or more rules.

14. The system of claim 13 , wherein the one or more rules comprise a size determination indicating a minimum or maximum number of customers in a cluster that is determined to be a standard customer.

15. A computer program product comprising software that when executed by a processor performs a method comprising:

receiving customer data from a plurality of computing devices over a network, the customer data including information for a plurality of customers to a plurality of entities;

performing, by the processor, unsupervised learning on the customer data to produce a plurality of clusters of customers with a plurality of features in common;

determining, by the processor, that one or more of the plurality of clusters each represent a standard customer and storing a plurality of standard customer profiles based on the determined standard customers, wherein the standard customer profiles comprise a plurality of data distributions for the plurality of features in common;

providing the plurality of standard customer profiles to each of the plurality of computing devices for generating synthetic transaction data based on the standard customer by:

providing the plurality of data distributions as a goal,

performing a plurality of iterations to generate synthetic transaction data, wherein the plurality of iterations is performed until a degree of similarity of the synthetic transaction data relative to the plurality of data distributions is higher than a predefined threshold,

in each iteration:

conducting an action including a plurality of simulated transactions,

comparing the action with the goal,

providing a feedback associated with the action based on a degree of similarity relative to the goal, and

adjusting a policy based on the feedback;

training, using the synthetic transaction data, a predictive model to identify an abnormal customer behavior; and

detecting, using the trained predictive model, the abnormal customer behavior in real transaction data.

16. The method of claim 15 , wherein the information for the plurality of customers comprises identifying information and transaction information.

17. The method of claim 15 , further comprising filtering the customer data prior to performing unsupervised learning.

18. The method of claim 17 , wherein the filtering comprises an RFM analysis to group customers.

19. The method of claim 15 , wherein the performing unsupervised learning comprises clustering customers based on a feature in common and repeating unsupervised learning to form clusters of customers based on the plurality of features in common.

20. The method of claim 15 , wherein determining that a cluster represents a standard customer comprises applying one or more rules.

21. The method of claim 20 , wherein the one or more rules comprise a size determination indicating a minimum or maximum number of customers in a cluster that is determined to be a standard customer.