Anomaly detection over high-dimensional space

1. A computer-implemented method comprising:

creating, by one or more computer processors, a binary cluster of events by bootstrapping a set of ground truths contained with a rule engine applied to a set of high-dimensional datapoints, wherein the binary cluster contains two clusters each containing a plurality of high-dimensional datapoints;

determining, by one or more computer processors, one or more peer groups for a set of unknown high-dimensional datapoints utilizing a trained multiclass classifier, wherein the high-dimensional datapoints are assigned to one or more peer groups by the trained multiclass classifier using an incremental learning algorithm in order to reduce system resources;

creating, by one or more computer processors, an activity distribution for each unknown high-dimensional datapoint associated with a user in the set of unknown high-dimensional datapoints and each peer group;

calculating, by one or more computer processors, a deviation percentage between the activity distribution of the user and each peer group associated with the user; and

responsive to exceeding a deviation threshold, classifying, by one or more computer processors, the user or associated high-dimensional datapoints as risky and performing an action.

2. The computer-implemented method of claim 1 , further comprising:

evaluating, by one or more computer processors, one or more unknown datapoints collected within a predetermined period utilizing a clustering classifier and the multiclass classifier;

comparing, by one or more computer processors, the evaluation of the clustering classifier and the evaluation of the multiclass classifier; and

responsive to detecting drift utilizing the comparison of evaluations from the clustering classifier and the multiclass classifier, utilizing, by one or more computer processors, evaluations from historical periods as a base for retraining the multiclass classifier.

3. The computer-implemented method of claim 1 , wherein creating the binary cluster of events by bootstrapping the set of ground truths contained with the rule engine applied to the set of high-dimensional datapoints, wherein the binary cluster contains two clusters each containing the plurality of high-dimensional datapoints, comprises:

identifying, by one or more computer processors, a set of rules as applied to a set of datapoints;

creating, by one or more computer processors, a labeled set of datapoints and associated binary choices utilizing the identified set of rules, wherein the binary choices are determined by one or more rule triggers;

correlating, by one or more computer processors, the set of high-dimensional datapoints within the labeled dataset with representation in a high-dimensional space, wherein the labeled dataset is directly represented in a space spanned by constituent datapoint attributes; and

clustering, by one or more computer processors, the correlated high-dimensional datapoints into the binary cluster.

4. The computer-implemented method of claim 1 , wherein the set of high-dimensional datapoints are events collected from access logs, authentication logs, account changes, network logs, and application logs.

5. The computer-implemented method of claim 1 , further comprising:

utilizing, by one or more computer processors, created activity distributions as feature vectors for the multiclass classifier.

6. The computer-implemented method of claim 1 , wherein the action is network restriction for the user.

7. The computer-implemented method of claim 1 , wherein the action is a notification.

8. A computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the stored program instructions comprising:

program instructions to create a binary cluster of events by bootstrapping a set of ground truths contained with a rule engine applied to a set of high-dimensional datapoints, wherein the binary cluster contains two clusters each containing a plurality of high-dimensional datapoints;

program instructions to determine one or more peer groups for a set of unknown high-dimensional datapoints utilizing a trained multiclass classifier, wherein the high-dimensional datapoints are assigned to one or more peer groups by the trained multiclass classifier using an incremental learning algorithm in order to reduce system resources;

program instructions to create an activity distribution for each unknown high-dimensional datapoint associated with a user in the set of unknown high-dimensional datapoints and each peer group;

program instructions to calculate a deviation percentage between the activity distribution of the user and each peer group associated with the user; and

program instructions to responsive to exceeding a deviation threshold, classify the user or associated high-dimensional datapoints as risky and performing an action.

9. The computer program product of claim 8 , wherein the program instructions, stored on the one or more computer readable storage media, further comprise:

program instructions to evaluate one or more unknown datapoints collected within a predetermined period utilizing a clustering classifier and the multiclass classifier;

program instructions to compare the evaluation of the clustering classifier and the evaluation of the multiclass classifier; and

program instructions to, responsive to detecting drift utilizing the comparison of evaluations from the clustering classifier and the multiclass classifier, utilize evaluations from historical periods as a base for retraining the multiclass classifier.

10. The computer program product of claim 8 , wherein the program instructions, to create the binary cluster of events by bootstrapping the set of ground truths contained with the rule engine applied to the set of high-dimensional datapoints, wherein the binary cluster contains two clusters each containing the plurality of high-dimensional datapoints, comprise:

program instructions to identify a set of rules as applied to a set of datapoints;

program instructions to create a labeled set of datapoints and associated binary choices utilizing the identified set of rules, wherein the binary choices are determined by one or more rule triggers;

program instructions to correlate the set of high-dimensional datapoints within the labeled dataset with representation in a high-dimensional space, wherein the labeled dataset is directly represented in a space spanned by constituent datapoint attributes; and

program instructions to cluster the correlated high-dimensional datapoints into the binary cluster.

11. The computer program product of claim 8 , wherein the set of high-dimensional datapoints are events collected from access logs, authentication logs, account changes, network logs, and application logs.

12. The computer program product of claim 8 , wherein the program instructions, stored on the one or more computer readable storage media, further comprise:

program instructions to utilize created activity distributions as feature vectors for the multiclass classifier.

13. The computer program product of claim 8 , wherein the action is network restriction for the user.

14. The computer program product of claim 8 , wherein the action is a notification.

15. A computer system comprising:

one or more computer processors;

one or more computer readable storage media; and

program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the stored program instructions comprising:

program instructions to create a binary cluster of events by bootstrapping a set of ground truths contained with a rule engine applied to a set of high-dimensional datapoints, wherein the binary cluster contains two clusters each containing a plurality of high-dimensional datapoints;

program instructions to determine one or more peer groups for a set of unknown high-dimensional datapoints utilizing a trained multiclass classifier, wherein the high-dimensional datapoints are assigned to one or more peer groups by the trained multiclass classifier using an incremental learning algorithm in order to reduce system resources;

program instructions to create an activity distribution for each unknown high-dimensional datapoint associated with a user in the set of unknown high-dimensional datapoints and each peer group;

program instructions to calculate a deviation percentage between the activity distribution of the user and each peer group associated with the user; and

program instructions to responsive to exceeding a deviation threshold, classify the user or associated high-dimensional datapoints as risky and performing an action.

16. The computer system of claim 15 , wherein the program instructions, stored on the one or more computer readable storage media, further comprise:

program instructions to evaluate one or more unknown datapoints collected within a predetermined period utilizing a clustering classifier and the multiclass classifier;

program instructions to compare the evaluation of the clustering classifier and the evaluation of the multiclass classifier; and

program instructions to, responsive to detecting drift utilizing the comparison of evaluations from the clustering classifier and the multiclass classifier, utilize evaluations from historical periods as a base for retraining the multiclass classifier.

17. The computer system of claim 15 , wherein the program instructions, to create the binary cluster of events by bootstrapping the set of ground truths contained with the rule engine applied to the set of high-dimensional datapoints, wherein the binary cluster contains two clusters each containing the plurality of high-dimensional datapoints, comprise:

program instructions to identify a set of rules as applied to a set of datapoints;

program instructions to create a labeled set of datapoints and associated binary choices utilizing the identified set of rules, wherein the binary choices are determined by one or more rule triggers;

program instructions to correlate the set of high-dimensional datapoints within the labeled dataset with representation in a high-dimensional space, wherein the labeled dataset is directly represented in a space spanned by constituent datapoint attributes; and

program instructions to cluster the correlated high-dimensional datapoints into the binary cluster.

18. The computer system of claim 15 , wherein the set of high-dimensional datapoints are events collected from access logs, authentication logs, account changes, network logs, and application logs.

19. The computer system of claim 15 , wherein the program instructions, stored on the one or more computer readable storage media, further comprise:

program instructions to utilize created activity distributions as feature vectors for the multiclass classifier.

20. The computer system of claim 15 , wherein the action is network restriction for the user.