Malicious activity detection system capable of efficiently processing data accessed from databases and generating alerts for display in interactive user interfaces
Various systems and methods are provided that retrieve raw data from issuers, reorganize the raw data, analyze the reorganized data to determine whether the risky or malicious activity is occurring, and generate alerts to notify users of possible malicious activity. For example, the raw data is included in a plurality of tables. The system joins one or more tables to reorganize the data using several filtering techniques to reduce the processor load required to perform the join operation. Once the data is reorganized, the system executes one or more rules to analyze the reorganized data. Each rule is associated with a malicious activity. If any of the rules indicate that malicious activity is occurring, the system generates an alert for display to a user in an interactive user interface.
1. A computing system comprising:
a database storing first data;
a computer processor; and
a computer readable storage medium storing program instructions configured for execution by the computer processor in order to cause the computing system to:
select a behavior outlier rule;
cluster a portion of the first data into a first cluster based on a statistical measure;
apply the behavior outlier rule to the first cluster to identify a first outlier and a second outlier;
generate a first alert for the first outlier and a second alert for the second outlier;
receive an indication of one or more user actions taken with respect to at least one of the first alert or the second alert;
in response to receiving the indication of one or more user actions taken with respect to the at least one of the first alert or the second alert:
identify a percentage of outliers of the first cluster that are actioned; and
determine that the percentage is different from a threshold; and
in response to determining that the percentage is different from the threshold, modify the behavior outlier rule based on the one or more user actions such that a modified behavior outlier rule is applied to future clusters.
2. The computing system of claim 1 , wherein the program instructions further cause the computing system to validate the first cluster using historical transaction data.
3. The computing system of claim 1 , wherein the program instructions further cause the computing system to cluster a second portion of the first data into a second cluster.
4. The computing system of claim 1 , wherein the first outlier and the second outlier are the top two outliers in the first cluster.
5. The computing system of claim 1 , wherein the one or more user actions comprises closing the first alert.
6. The computing system of claim 1 , wherein the statistical measure comprises one of a mean, a mode, or a standard deviation.
7. The computing system of claim 3 , wherein the program instructions further cause the computing system to limit a number of clusters created from the first data using historical transaction data.
8. A computer-implemented method comprising:
as implemented by one or more computer systems comprising computer hardware and memory, the one or more computer systems configured with specific executable instructions,
selecting a behavior outlier rule;
clustering a portion of first data into a first cluster based on a statistical measure;
applying the behavior outlier rule to the first cluster to identify a first outlier and a second outlier;
generating a first alert for the first outlier and a second alert for the second outlier;
receiving an indication of one or more user actions taken with respect to at least one of the first alert or the second alert;
in response to receiving the indication of one or more user actions taken with respect to the at least one of the first alert or the second alert:
identifying a percentage of outliers of the first cluster that are actioned; and
determining that the percentage is different from a threshold; and
in response to determining that the percentage is different from the threshold, modifying the behavior outlier rule based on the one or more user actions such that a modified behavior outlier rule is applied to future clusters.
9. The computer-implemented method of claim 8 , further comprising validating the first cluster using historical transaction data.
10. The computer-implemented method of claim 8 , further comprising clustering a second portion of the first data into a second cluster.
11. The computer-implemented method of claim 8 , wherein the first outlier and the second outlier are the top two outliers in the first cluster.
12. The computer-implemented method of claim 8 , wherein the one or more user actions comprises closing the first alert.
13. The computer-implemented method of claim 8 , wherein the statistical measure comprises one of a mean, a mode, or a standard deviation.
14. The computer-implemented method of claim 10 , further comprising limiting a number of clusters created from the first data using historical transaction data.
15. A non-transitory computer-readable medium comprising one or more program instructions recorded thereon, the instructions configured for execution by a computing system comprising one or more processors in order to cause the computing system to:
select a behavior outlier rule;
cluster a portion of first data into a first cluster based on a statistical measure;
apply the behavior outlier rule to the first cluster to identify a first outlier and a second outlier;
generate a first alert for the first outlier and a second alert for the second outlier;
receive an indication of one or more user actions taken with respect to at least one of the first alert or the second alert;
in response to receiving the indication of one or more user actions taken with respect to the at least one of the first alert or the second alert:
identify a percentage of outliers of the first cluster that are actioned; and
determine that the percentage is different from a threshold; and
in response to determining that the percentage is different from the threshold, modify the behavior outlier rule based on the one or more user actions such that a modified behavior outlier rule is applied to future clusters.
16. The non-transitory computer-readable medium of claim 15 , wherein the instructions are further configured to cause the computing system to validate the first cluster using historical transaction data.
17. The non-transitory computer-readable medium of claim 15 , wherein the one or more user actions comprises closing the first alert.