Identifying insider-threat security incidents via recursive anomaly detection of user behavior

1. A computerized system for recursively detecting anomalies in monitored behavior of entities, the system comprising:

a storage unit to store monitored events, event deviations and parameters related to each event and to each event deviation;

a processing unit configured to:

(1) receive a plurality of input events, each input event comprising a predefined set of parameters,

(2) construct a plurality of baseline models, each baseline model representing a behavior of at least a subset of parameters of the received input events,

(3) receive an input event that occurred during an analyzed timeframe,

(4) compare parameters of the received input event to a corresponding baseline model in order to detect an event deviation,

(5) associate an event deviation score to the detected event deviation,

(6) using the detected event deviation as an input event, repeat at least (1)-(5) for each of one or more additional iterations until a predetermined condition relating to a number of detected event deviations is satisfied, the predetermined condition specifying detection of no more than a maximum number of event deviations for a particular designated time period, the one or more additional iterations being operative to reduce the number of detected event deviations relative to a number of detected event deviations in an initial iteration, and

(7) generate alerts indicating suspicious activity has been detected for a specified entity and a specified timeframe associated with the detected event deviation; and

a viewer application configured to:

receive alerts relating to one or more detected event deviations; and

display said alerts to a user of the viewer application on a display unit.

2. A computerized system for detecting anomalies in monitored behavior of entities, the system comprising:

a storage unit to store monitored events, event deviations and parameters related to each event and to each event deviation;

a processing unit configured to:

receive a plurality of event deviations and associated event deviation scores that were determined for a baseline time period, each event deviation comprising a predefined set of parameters, the event deviations being determined recursively over multiple iterations in which one or more event deviations detected in an initial one of the iterations are used as respective input events to a subsequent one of the iterations, the subsequent iteration being operative to produce a reduced number of detected event deviations relative to a number of detected event deviations in the initial iteration, the iterations continuing until a predetermined condition relating to a number of detected event deviations is satisfied, the predetermined condition specifying detection of no more than a maximum number of event deviations for a particular designated time period,

construct event deviation baseline models, each event deviation baseline model representing a behavior of a subset of selected parameters of the received event deviations,

receive an event deviation that occurred during an analyzed timeframe,

compare parameters of the event deviation to a corresponding event deviation baseline model in order to detect event anomalies, each event anomaly comprising one or more parameters,

associate an event anomaly score to each detected event anomaly, and

generate alerts indicating suspicious activity has been detected for a specified entity and a specified timeframe associated with the detected event anomaly; and

a viewer application configured to receive alerts relating to the detected event anomalies, and to display said alerts to a user of the viewer application on a display unit.

3. The system according to claim 2 , wherein the processing unit is further configured to:

receive a plurality of event anomalies and associated event anomaly scores that were determined for the baseline time period, wherein each event anomaly comprises a predefined set of parameters and a specified timeframe;

obtain an analyzed timeframe and a specified entity;

obtain a context feature set comprising a plurality of context features, wherein each context feature is based on at least one parameter derived from the event anomaly;

generate a reduced feature set from the context feature set;

calculate an aggregated anomaly value per specified entity per specified timeframe based on the reduced context feature set; and

generate a set of aggregated anomaly values per specified entity per specified timeframe.

4. The system according to claim 2 , wherein the processing unit is further configured to:

receive a plurality of aggregated anomaly values that were determined during the baseline time period, wherein each aggregated anomaly value is calculated based on context features per a specified timeframe;

construct an anomaly baseline model;

receive new aggregated anomaly values calculated for a specified timeframe;

compare the new aggregated anomaly values to the anomaly baseline model in order to identify an aggregated anomaly value deviation, wherein the aggregated anomaly value deviation comprises an entity and a specified timeframe; and

generate an alert indicating suspicious activity detected for the specified entity during the specified timeframe.

5. The system according to claim 3 , wherein the processing unit is configured to generate said reduced context feature set from the context feature set by:

calculating a correlation measure for every pair of context features in the context feature set;

defining a distance function between any two context features to determine correlating context features; and

performing clustering algorithm to group correlating context features.

6. The system according to claim 2 , wherein the analyzed timeframe is selected from the group consisting of sessions, minutes, hours, days and months.

7. The system according to claim 3 , wherein a context feature of the context feature set is calculated based on at least one of following parameters:

a number of events detected during a specified timeframe which include the context feature, a function of the context feature score in the specified timeframe, a number of a distinct occurrences in the specified timeframe, a number of a values measured for the context feature during the specified timeframe, a number of a new values measured for the context feature during the specified timeframe, a parameter type, a correlation measure between the context feature and another context feature, and a score calculated based on a stability over time of context features.

8. The system according to claim 7 , wherein a context feature of the context feature set is calculated based on received event data, a level of correlation between two or more context features, and the stability over time of context features.

9. The system according to claim 7 , wherein determining a correlation between two or more context features in the context feature set comprises: generating a correlation measure between the two or more context features based on the baseline time period, and determining a strength of the correlation.

10. The system according to claim 5 , wherein the processing unit is configured to group correlating context features into clusters by: calculating a distance function based on a correlation measure, and using the distance function to perform a clustering algorithm on the set of context features.

11. A computerized method for detecting anomalies in monitored behavior of entities, the method comprising:

storing monitored events, event deviations and parameters related to each event and to each event deviation;

receiving, by a processing unit, a plurality of event deviations and associated event deviation scores that were determined for a baseline time period, each event deviation comprising a predefined set of parameters, the event deviations being determined recursively over multiple iterations in which one or more event deviations detected in an initial one of the iterations are used as respective input events to a subsequent one of the iterations, the subsequent iteration being operative to produce a reduced number of detected event deviations relative to a number of detected event deviations in the initial iteration, the iterations continuing until a predetermined condition relating to a number of detected event deviations is satisfied, the predetermined condition specifying detection of no more than a maximum number of event deviations for a particular designated time period;

constructing, by the processing unit, event deviation baseline models, each event deviation baseline model representing a behavior of a subset of selected parameters of the received event deviations;

receiving, by the processing unit, an event deviation that occurred during an analyzed timeframe;

comparing, by the processing unit, parameters of the event deviation to a corresponding event deviation baseline model in order to detect event anomalies, each event anomaly comprising one or more parameters;

associating, by the processing unit, an event anomaly score to each detected event anomaly;

generating, by the processing unit, alerts indicating suspicious activity has been detected for an entity and a timeframe associated with the detected event anomaly;

receiving alerts, by a viewer application, relating to the detected event anomalies; and

displaying, by the viewer application, said alerts to a user of the viewer application on a di splay unit.

12. The method according to claim 11 , further comprising:

receiving, by the processing unit, a plurality of event anomalies and associated event anomaly scores that were determined for the baseline time period, wherein each anomaly comprises a predefined set of parameters and a specified timeframe;

obtaining, by the processing unit, an analyzed timeframe and a specified entity;

obtaining, by the processing unit, a context feature set comprising a plurality of context features, wherein each context feature is based on at least one parameter derived from the event anomaly;

generating, by the processing unit, a reduced feature set from the context feature set;

calculating, by the processing unit, an aggregated anomaly value per specified entity per specified timeframe based on the reduced context feature set; and

generating, by the processing unit, a set of aggregated anomaly values per entity per timeframe.

13. The method according to claim 11 , further comprising:

receiving, by the processing unit, a plurality of aggregated anomaly values that were determined during the baseline time period, wherein each aggregated anomaly value is calculated based on context features per a predetermined timeframe score;

constructing, by the processing unit, an anomaly baseline model;

receiving, by the processing unit, new aggregated anomaly values calculated for a specified timeframe;

comparing, by the processing unit, the new aggregated anomaly values to the anomaly baseline model in order to identifying an aggregated anomaly value deviation, wherein the aggregated anomaly value deviation comprises an entity and a specified timeframe; and

generating, by the processing unit, an alert indicating suspicious activity detected for the specified entity during the specified timeframe.

14. The method according to claim 12 , wherein generating said reduced context feature set from the context feature set comprises:

calculating a correlation measure for every pair of context features in the context feature set;

defining a distance function between any two context features; and

performing a clustering algorithm to group correlating context features.

15. The method according to claim 11 , further comprising selecting, by the processing unit, the analyzed timeframe from a group consisting of: sessions, minutes, hours, days and months.

16. The method according to claim 12 , further comprising calculating a context feature of the context feature set based on at least one of following parameters: a number of events detected during a specified timeframe which include the context feature, a function of the context feature score in the specified timeframe, a number of a distinct occurrences in the specified timeframe, a number of a values measured for the context feature during the specified timeframe, a number of a new values measured for the context feature during the specified timeframe, a parameter type, a correlation measurement between the context feature and another context feature, and a score calculated based on a stability over time of context features.

17. The method according to claim 16 , further comprising calculating a context feature of the context feature set based on received event data, a level of correlation between two or more context features, and the stability over time of context features.

18. The method according to claim 14 , wherein grouping correlating context features into clusters comprises: calculating a distance function based on a correlation measure, and using the distance function to perform a clustering algorithm on the set of context features.

19. The system according to claim 1 , wherein the processing unit is further configured to:

receive a plurality of event anomalies and associated event anomaly scores that were determined for a baseline time period, wherein each event anomaly comprises a predefined set of parameters and a specified timeframe;

obtain an analyzed timeframe and a specified entity;

obtain a context feature set comprising a plurality of context features, wherein each context feature is based on at least one parameter derived from the event anomaly;

generate a reduced feature set from the context feature set;

calculate an aggregated anomaly value per specified entity per specified timeframe based on the reduced context feature set; and

generate a set of aggregated anomaly values per specified entity per specified timeframe.

20. The system according to claim 1 , wherein the processing unit is further configured to:

receive a plurality of aggregated anomaly values that were determined during a baseline time period, wherein each aggregated anomaly value is calculated based on context features per a specified timeframe;

construct an anomaly baseline model;

receive new aggregated anomaly values calculated for a specified timeframe;

compare the new aggregated anomaly values to the anomaly baseline model in order to identify an aggregated anomaly value deviation, wherein the aggregated anomaly value deviation comprises an entity and a specified timeframe; and

generate an alert indicating suspicious activity detected for the specified entity during the specified timeframe.