Systems and methods for distributed rule-based correlation of events

1. A method for distributed correlation of events implemented by a physical processor of a master management device deployed on a computer hardware program platform executing computer-readable instructions, the method comprising:

determining, by the processor, a first event of a first subset of events is a first partial match of a distributed rule from a first slave management device, wherein the distributed rule is associated with the master management device;

determining, by the processor, a second event of a second subset of events is a second partial match of the distributed rule from a second slave management device;

receiving, by the processor, event summary data comprising properties of events determined to be partial matches of the distributed rule from the first and the second slave management devices;

receiving, at the processor, a first notification of the first partial match by the first event and a second notification of the second partial match by the second event, wherein the first notification comprises a set of properties of the first event, and the second notification comprises a set of properties of the second event;

aggregating, by the processor, the first notification and the second notification;

evaluating, by the processor, the distributed rule by aggregating, over a sliding time window, the set of properties of the first event and the set of properties of the second event;

determining, by the processor, based on the event summary data, and in response to the aggregation of the set of properties of the first event and the set of properties of the second event exceeding a threshold defined by the distributed rule, a complete match of the distributed rule based on the evaluation; and

generating, by the processor, a correlation event.

2. The method of claim 1 , further comprising:

receiving, at the processor, the second event;

evaluating, by the processor, the distributed rule using the set of properties of the second event; and

determining, by the processor, the second partial match of the distributed rule by the second event.

3. The method of claim 1 , wherein the first event is received at a second computing device, and wherein the notification of the first partial match by the first event is generated by the second computing device.

4. The method of claim 1 , wherein the set of properties of the first event of the first subset of events includes an event timestamp, an event identifier, a set of aggregation fields, or a set of join fields.

5. The method of claim 1 , wherein evaluating the distributed rule comprises:

determining, by the processor, whether each set of conditions of the distributed rule is satisfied based on the aggregation of the set of properties of the first event and the set of properties of the second event.

6. The method of claim 1 , further comprising:

detecting, by the processor, an update to a local copy of a data list; and

synchronizing, by the processor, a master copy of the data list with the local copy.

7. The method of claim 1 , wherein data validation is performed on a local copy of a data list prior to evaluating a lookup request on the local copy.

8. The method of claim 7 , wherein the lookup request is batched until the local copy of the data list is validated.

9. The method of claim 1 , further comprising performing an action, by the processor, specified in the distributed rule based on the correlation event.

10. The method of claim 9 , wherein performing the action, by the processor, comprises executing a pre-determined command or script, updating a data list, logging an alert, sending alerts to a console or notification designees, setting custom severity levels for the alert based on cumulative activity, adding a source to a list of suspicious sources, adding a target to a vulnerability list, or a combination thereof.

11. The method of claim 1 , further comprising each of the first and the second slave management devices operating on a plurality of events which it receives, wherein each of the first and the second slave management devices does not have information on a plurality of events received by other slave management devices.

12. A system for distributed correlation of events, the system comprising:

non-transitory memory; and

a first hardware processor of a master management device deployed on a computer hardware platform of a first computer system, in communication with the memory, and configured to:

determine a first event of a first subset of events is a first partial match of a distributed rule from a first slave management device, wherein the distributed rule is associated with the master management device;

determine a second event of a second subset of events is a second partial match of the distributed rule from a second slave management device;

receive event summary data comprising properties of events determined to be partial matches of the distributed rule from the first and the second slave management devices; and

generate a first notification of the first partial match and a second notification of the second partial match, wherein the first notification comprises a set of properties of the first event, and the second notification comprises a set of properties of the second event; and

a second hardware processor of a second computer system, in communication with the memory, and configured to:

receive the first and the second notifications;

aggregate the first and the second notifications;

evaluate the distributed rule by aggregating, over a sliding time window, the set of properties of the first event and the set of properties of the second event;

determine, based on the event summary data and in response to the aggregation of the set of properties of the first event and the set of properties of the second event exceeding a threshold defined by the distributed rule, a complete match of the distributed rule based on the evaluation; and

generate a correlation event.

13. The system of claim 12 , wherein the second processor is configured to evaluate the distributed rule by:

determining whether each set of conditions of the distributed rule is satisfied based on the aggregation of the set of properties of the first event and the set of properties of the second event.

14. The system of claim 12 , wherein the second processor is configured to:

detect an update to a local copy of a data list; and

synchronize a master copy of the data list with the local copy.

15. The system of claim 12 , wherein the correlation event comprises:

information associated with base events that trigger the distributed rule.

16. The system of claim 12 , wherein the master management device comprises information associated with a plurality of events received by the slave management devices with respect to the distributed rule.

17. A processor-readable non-transitory medium comprising code representing instructions that when executed at a hardware processor of a master management device deployed on a computer hardware platform cause the hardware processor to:

determine a first event of a first subset of events is a first partial match of a distributed rule from a first slave management device, wherein the distributed rule is associated with the master management device;

determine a second event of a second subset of events is a second partial match of the distributed rule from a second slave management device;

receive event summary data comprising properties of events determined to be partial matches of the distributed rule from the first and the second slave management devices;

receive a first notification of the first partial match and a second notification of the second partial match, wherein the notification comprises a set of properties of the first event, and the second notification comprises a set of properties of the second event;

aggregate the first and the second notifications;

evaluate the distributed rule by aggregating, over a sliding time window, the set of properties of the first event and the set of properties of the second event;

determine, based on the event summary data and in response to the aggregation of the set of properties of the first event and the set of properties of the second event exceeding a threshold defined by the distributed rule, a complete match of the distributed rule based on the evaluation; and

generate a correlation event.

18. The medium of claim 17 , wherein the executed instructions to evaluate the distributed rule comprise executed instructions to:

determine whether each set of conditions of the distributed rule is satisfied based on the aggregation of the set of properties of the first event and the set of properties of the second event.

19. The medium of claim 17 , wherein the executed instructions further comprise executed instructions to:

detect an update to a local copy of a data list; and

synchronize a master copy of the data list with the local copy.