Multivariate risk assessment via Poisson Shelves
Detecting fraudulent activity can be a complex, manual process. In this paper, we adapt statistical properties of count data in a novel algorithm to uncover records exhibiting high risk for fraud. Our method identifies shelves, partitioning data under the counts using a Student's t-distribution. We apply this methodology on a univariate dataset including cumulative results from phone calls to a customer service center. Additionally, we extend this technique to multivariate data, illustrating that the same method is applicable to both univariate and multivariate data.
1. A non-transitory computer readable medium comprising instructions that, when executed by a processor of a processing system, cause the processing system to perform a method for identifying outliers for risk assessment using count data “Records” having y values, the method comprising:
computing a sample mean {circumflex over (λ)} of y values in Records;
calculating threshold value ω={circumflex over (λ)}+n√{circumflex over (λ)}, from n=0 to n max , where n is the number of standard deviation units in ω;
for every ω, computing a number of surviving records according to |{r∈Records: r[y]>ω}|;
identifying at least one shelf in surviving data by identifying an abrupt variation in generative parameters of a sequence of data; and
defining outliers for further risk assessment according to {r∈Records r[y]>{circumflex over (λ)}+n*√{circumflex over (λ)}}.
2. The non-transitory computer readable medium of claim 1 , the method further comprising creating a visual representation of the surviving data for identifying the at least one shelf.
3. The non-transitory computer readable medium of claim 1 , wherein identifying the at least one shelf comprises identifying an abrupt variation in the generative parameters of a sequence of data.
4. The non-transitory computer readable medium of claim 1 , wherein the at least one shelf is identified using concept drift detection.
5. The non-transitory computer readable medium of claim 1 , wherein the threshold value ω is calculated according to:
ω=λ+ n ×stdev( X )=λ+ n√λ
where n is the number of standard deviation units in ω, λ is the mean of the Poisson distribution and X is a random variable having integer values.
6. The non-transitory computer readable medium of claim 5 , the method further comprising determining a number of datapoints greater than the threshold ω for a given number of standard deviation units n.
7. The non-transitory computer readable medium of claim 1 , wherein identifying the at least one shelf comprises identifying an abrupt variation in the generative parameters of a sequence of data.
8. A method of identifying outliers for risk assessment using count data “Records” having y values, comprising:
computing a sample mean {circumflex over (λ)} of y values in Records;
calculating threshold value ω={circumflex over (λ)}+n√{circumflex over (λ)}, from n=0 to n max , where n is the number of standard deviation units in ω;
for every ω, computing a number of surviving records according to |{r∈Records: r[y]>ω}|;
identifying a shelf in surviving data by identifying an abrupt variation in generative parameters of a sequence of data; and
defining outliers for further risk assessment according to {r∈Records r[y]>{circumflex over (λ)}+n*√{circumflex over (λ)}}.
9. The method of claim 8 , the method further comprising creating a visual representation of the surviving data for identifying the at least one shelf.
10. The method of claim 8 , wherein the at least one shelf is user-identified based on the visual representation.
11. The method of claim 8 , wherein the threshold value co is calculated according to:
ω=λ+ n ×stdev( X )=λ+ n√λ
where n is the number of standard deviation units in ω, λ is the mean of the Poisson distribution and X is a random variable having integer values.
12. The method of claim 11 , further comprising determining a number of datapoints greater than the threshold ω for a given number of standard deviation units n.
13. The method of claim 8 , wherein identifying the at least one shelf comprises identifying an abrupt variation in the generative parameters of a sequence of data.
14. A system for identifying outliers for risk assessment using count data “Records” having y values, comprising:
a memory comprising executable instructions;
a processor configured to execute the executable instructions and cause the system to:
compute a sample mean {circumflex over (λ)} of y values in Records;
calculating threshold value ω={circumflex over (λ)}+n√{circumflex over (λ)}, from n=0 to n max , where n is the number of standard deviation units in ω;
for every ω, computing a number of surviving records according to |{r∈Records: r[y]>ω}|;
identify a shelf in surviving data by identifying an abrupt variation in generative parameters of a sequence of data; and
define outliers for further risk assessment according to {r∈Records: r[y]>{circumflex over (λ)}+n*√{circumflex over (λ)}}.
15. The system of claim 14 , the processor further configured to execute the executable instructions to cause the system to create a visual representation of the surviving data for identifying the at least one shelf.
16. The system of claim 14 , wherein identifying the at least one shelf comprises identifying an abrupt variation in the generative parameters of a sequence of data.
17. The system of claim 14 , wherein the at least one shelf if identified using concept drift detection.
18. The system of claim 14 , wherein the threshold value co is calculated according to:
ω=λ+ n ×stdev( X )=λ+ n√λ
where n is the number of standard deviation units in ω, λ is the mean of the Poisson distribution and X is a random variable having integer values.
19. The system of claim 18 , the processor further configured to execute the executable instructions to cause the system to determine a number of datapoints greater than the threshold ω for a given number of standard deviation units n.