Multivariate risk assessment via poisson shelves
Detecting fraudulent activity can be a complex, manual process. In this paper, we adapt statistical properties of count data in a novel algorithm to uncover records exhibiting high risk for fraud. Our method identifies shelves, partitioning data under the counts using a Student's t-distribution. We apply this methodology on a univariate dataset including cumulative results from phone calls to a customer service center. Additionally, we extend this technique to multivariate data, illustrating that the same method is applicable to both univariate and multivariate data.
1. A non-transitory computer readable medium comprising instructions that, when executed by a processor of a processing system, cause the processing system to perform a method for identifying outliers for risk assessment in multivariable count data having a first count variable x i and a second count variable y i , the method comprising:
calculating a mixture of independent Poissons:
ℙ
MixedPoi
(
x
)
=
∫
ℝ
++
d
g
(
λ
)
∏
i
=
1
d
ℙ
Poiss
(
x
i
|
λ
i
)
d
λ
where x=[ 1 , x 2 , . . . x d ], d is the dimension of data, λ=[λ 1 , λ 2 , . . . λ d ] is a vector where λ i is a parameter of the Poisson distribution for x i , and g(λ) is the mixing distribution
calculating a threshold value co for every combination of mixture component c and dimension i according to:
ω ic ={circumflex over (λ)} ic +n √{square root over ({circumflex over (λ)} ic )}
identifying at least one shelf in surviving data by identifying an abrupt variation in generative parameters of a sequence of data;
defining outliers in component c:
{ r ∈Records c :∃i s.t.r[y i ]>λ ic +n ic *√{square root over (λ ic )}}
where y i is the ith dimension of count data, and n* ic is the final n value of a first detected shelf in the ith dimension for component c; and
providing a visual representation of the outliers.
2. The non-transitory computer readable medium of claim 1 , wherein the at least one shelf is user-identified based on the visual representation.
3. The non-transitory computer readable medium of claim 1 , wherein the at least one shelf is identified using concept drift detection.
4. The non-transitory computer readable medium of claim 1 , wherein the threshold value ω is calculated according to:
ω=λ+ n x stdev( X )=λ+ n√λ
where n is the number of standard deviation units in co, A is the mean of the Poisson distribution and X is a random variable having integer values.
5. The non-transitory computer readable medium of claim 4 , further comprising determining a number of datapoints greater than the threshold co for a given number of standard deviation units n.
6. A method of identifying outliers for risk assessment in multivariable count data having a first count variable x i and a second count variable y i , the method comprising:
calculating a mixture of independent Poissons:
ℙ
MixedPoi
(
x
)
=
∫
ℝ
++
d
g
(
λ
)
∏
i
=
1
d
ℙ
Poiss
(
x
i
|
λ
i
)
d
λ
where x=[x 1 , x 2 , . . . x d ], d is the dimension of data, λ=[λ 1 , λ 2 , . . . λ d ] is a vector where λ i is a parameter of the Poisson distribution for x i , and g(λ) is the mixing distribution
calculating a threshold value co for every combination of mixture component c and dimension i according to:
ω ic ={circumflex over (λ)} ic +n √{square root over ({circumflex over (λ)} ic )}
identifying at least one shelf in surviving data by identifying an abrupt variation in generative parameters of a sequence of data;
defining outliers in component c:
{ r ∈Records c : ∃i s.t.r[ y i ]>λ ic +n ic *√{square root over (λ ic )}}
where y i is the ith dimension of count data, and n* ic is the final n value of a first detected shelf in the ith dimension for component c; and
identifying the outliers for further risk assessment.
7. The method of claim 6 , the method further comprising creating a visual representation of the surviving records for identifying the at least one shelf.
8. The method of claim 7 , wherein the at least one shelf is user-identified based on the visual representation.
9. The method of claim 6 , wherein the threshold value ω is calculated according to:
ω=λ+ n x stdev( X )=λ+ n√λ
where n is the number of standard deviation units in co, A is the mean of the Poisson distribution and X is a random variable having integer values.
10. The method of claim 9 , further comprising determining a number of datapoints greater than the threshold co for a given number of standard deviation units n.
11. The system of claim 9 , wherein executing the executable instructions further causes the system to determine a number of datapoints greater than the threshold co for a given number of standard deviation units n.
12. The system of claim 6 , wherein the threshold value ω is calculated according to:
ω=λ+ n x stdev( X )=λ+ n√λ
where n is the number of standard deviation units in ω, λ is the mean of the Poisson distribution and X is a random variable having integer values.
13. A system for identifying outliers for risk assessment in multivariable count data having a first count variable x i and a second count variable y i , comprising:
a memory comprising executable instructions;
a processor configured to execute the executable instructions and cause the system to:
calculate a mixture of independent Poissons:
ℙ
MixedPoi
(
x
)
=
∫
ℝ
++
d
g
(
λ
)
∏
i
=
1
d
ℙ
Poiss
(
x
i
|
λ
i
)
d
λ
where x=[x 1 , x 2 , . . . x d ], d is the dimension of data, λ=[λ 1 , λ 2 , . . . λ d ] is a vector where λ i is a parameter of the Poisson distribution for x i , and g(λ) is the mixing distribution
calculate a threshold value ω for every combination of mixture component c and dimension i according to:
ω ic ={circumflex over (λ)} ic +n √{square root over ({circumflex over (λ)} ic )}
identify at least one shelf in surviving data by identifying an abrupt variation in generative parameters of a sequence of data;
define outliers in component c:
{ r ∈Records c : ∃i s.t.r[ y i ]>λ ic +n ic *√{square root over (λ ic )}}
where y i is the ith dimension of count data, and n* ic is the final n value of a first detected shelf in the ith dimension for component c; and
identify the outliers for further risk assessment.
14. The system of claim 13 , wherein executing the executable instructions further causes the system to create a visual representation of the surviving records for identifying the at least one shelf.
15. The system of claim 14 , wherein the at least one shelf is user-identified based on the visual representation.