Risk assessment using Poisson Shelves
Detecting fraudulent activity can be a complex, manual process. In this paper, we adapt statistical properties of count data in a novel algorithm to uncover records exhibiting high risk for fraud. Our method identifies shelves, partitioning data under the counts using a Student's t-distribution. We apply this methodology on a univariate dataset including cumulative results from phone calls to a customer service center. Additionally, we extend this technique to multivariate data, illustrating that the same method is applicable to both univariate and multivariate data.
1. A computer program product comprising computer executable code embodied in a non-transitory computer readable medium that, when executing on one more computing devices performs a method of performing risk assessment using count data, comprising:
accessing a probabilistic model in a library of known probabilistic models;
modeling said count data using a Poisson distribution using the probabilistic model running on a processor;
calculating a threshold value, ω, using the probabilistic model;
partitioning the count data based on the threshold ω, using the probabilistic model;
determining a number of surviving records from among the partitioned count data based on the threshold ω, using the probabilistic model;
identifying at least one shelf in the surviving records; and
defining outliers for risk assessment according to the at least one shelf.
2. The computer program product of claim 1 , wherein identifying the at least one shelf comprises identifying an abrupt variation in generative parameters of a sequence of data.
3. The computer program product of claim 1 , the method further comprising creating a visual representation of the surviving records for identifying the at least one shelf.
4. The computer program product of claim 3 , wherein the at least one shelf is user-identified based on the visual representation.
5. The computer program product of claim 4 , wherein identifying the at least one shelf comprises identifying an abrupt variation in generative parameters of a sequence of data.
6. The computer program product of claim 1 , wherein the at least one shelf if identified using concept drift detection.
7. The computer program product of claim 1 , wherein the threshold value ω is calculated according to:
ω=λ+ n ×stdev( X )=λ+ n√λ
where n is the number of standard deviation units in ω, λ is a mean of a Poisson distribution and X is a random variable having integer values.
8. The computer program product of claim 7 , further comprising determining a number of datapoints greater than the threshold value ω for a given number of standard deviation units n.
9. The computer program product of claim 1 , wherein the library of known probabilistic models is Python Pomegranate.
10. A method of performing risk assessment using count data using a probabilistic model running on computing device, comprising:
accessing a probabilistic model in a library of known probabilistic models;
modeling said count data using a Poisson distribution using the probabilistic model;
calculating a threshold value, ω, using the probabilistic model;
partitioning the count data based on the threshold ω, using the probabilistic model;
determining a number of surviving records from among the portioned count data based on the threshold ω, using the probabilistic model;
identifying at least one shelf in the surviving records; and
defining outliers for risk assessment according to the at least one shelf.
11. The method of claim 10 , wherein identifying the at least one shelf comprises identifying an abrupt variation in generative parameters of a sequence of data.
12. The method of claim 10 , the method further comprising creating a visual representation of the surviving records for identifying the at least one shelf.
13. The method of claim 12 , wherein the at least one shelf is user-identified based on the visual representation.
14. The method of claim 13 , wherein identifying the at least one shelf comprises identifying an abrupt variation in generative parameters of a sequence of data.
15. The method of claim 10 , wherein the threshold value ω is calculated according to:
ω=λ+ n ×stdev( X )=λ+ n√λ
where n is the number of standard deviation units in ω, λ is the mean of a Poisson distribution and X is a random variable having integer values.
16. The method of claim 15 , further comprising determining a number of datapoints greater than the threshold value ω for a given number of standard deviation units n.
17. The method of claim 10 , wherein the library of known probabilistic models is Python Pomegranate.