Scalable security information and event management (SIEM) framework

1. A framework for security information and event management (SIEM), the framework comprising:

at least one processor;

memory, operatively connected to the at least one processor storing instructions that, when executed by the processor, cause the framework to perform operations:

a first data store;

one or more parsing mechanisms;

one or more correlation machines; and

one or more workflow engines,

wherein said framework performs SIEM on behalf of multiple subscribers to said framework, and

wherein said first data store is constructed and adapted to store log data obtained from at least some of said multiple subscribers, and

wherein said one or more parsing mechanisms are constructed and adapted to obtain log data from said first data store and to normalize said log, and

wherein said one or more correlations machines are constructed and adapted: to obtain normalized log data, and to apply one or more correlation rules to said normalized log data to determine one or more correlations, and to put information about said one or more correlations on a correlations queue; and

wherein said one or more workflow engines are constructed and adapted to obtain said information about said one or more correlations from said correlations queue and to determine ticket information based on said information about said one or more correlations, and to provide said ticket information to a subscriber of said multiple subscribers.

2. The framework of claim 1 , wherein a data router informs said one or more correlation machines that normalized log data is by putting a message on a parsed queue.

3. The framework of claim 2 , wherein the parsed message comprises a Simple Notification Service (SNS) message on the parsed queue.

4. The framework of claim 1 , wherein the first data obtains log data obtained from one or more log collection appliances (LCAs) associated with said multiple subscribers.

5. The framework of claim 4 , wherein each particular LCA is associated with a corresponding particular subscriber of said multiple subscribers.

6. The framework of claim 1 , wherein said first data store is constructed and adapted to store data for each subscriber separately from data from each other subscriber.

7. The framework of claim 1 , wherein a data router informs said one or more correlation machines of normalized log data present, based on subscriber-specific criteria.

8. The framework of claim 1 , wherein the one or more correlation machines include at least one subscriber-dedicated correlation machine.

9. The framework of claim 1 , wherein each subscriber has a corresponding correlation machine.

10. The framework of claim 1 , wherein the one or more workflow engines determine said ticket information based on said information about said one or more correlations, and on other information.

11. The framework of claim 10 , wherein the other information comprises information from one or more external systems.

12. The framework of claim 1 , wherein said ticket information is used to generate and/or cause automated intervention at the subscriber.

13. The framework of claim 12 , wherein said intervention is provided using one or more APIs on devices on the subscriber's network.

14. A method operable in a framework for security information and event management (SIEM), said framework supporting SIEM on behalf of multiple subscribers to said framework, the framework having: a first data store; one or more parsing mechanisms; one or more correlation machines; and one or more workflow engines,

wherein the method comprises:

storing, in said first data, encrypted, compressed log data obtained from at least some of said multiple subscribers;

said one or more parsing mechanisms obtaining data from said first data store and normalizing said log data,

said one or more correlations machines: obtaining normalized log data, and applying one or more correlation rules to said normalized log data to determine one or more correlations, and putting information about said one or more correlations on a correlations queue; and

said one or more workflow engines obtaining said information about said one or more correlations from said correlations queue and determining ticket information based on said information about said one or more correlations, and providing said ticket information to a subscriber of said multiple subscribers.

15. The method of claim 14 , wherein a data router informs said one or more correlation machines that normalized log data are present by putting a message on a parsed queue.

16. The method of claim 15 , wherein the parsed message comprises a Simple Notification Service (SNS) message on the parsed queue.

17. The method of claim 14 , wherein the first data obtains log data obtained from one or more log collection appliances (LCAs) associated with said multiple subscribers.

18. The method of claim 17 , wherein each particular LCA is associated with a corresponding particular subscriber of said multiple subscribers.

19. The method of claim 14 , wherein said first data store is constructed and adapted to store data for each subscriber separately from data from each other subscriber.

20. The method of claim 14 , wherein a data router informs said one or more correlation machines of normalized log data are present, based on subscriber-specific criteria.

21. The method of claim 14 , wherein the one or more correlation machines include at least one subscriber-dedicated correlation machine.

22. The method of claim 14 , wherein each subscriber has a corresponding correlation machine.

23. The method of claim 14 , wherein the one or more workflow engines determine said ticket information based on said information about said one or more correlations, and on other information.

24. The method of claim 23 , wherein the other information comprises information from one or more external systems.

25. The method of claim 14 , wherein said ticket information is used to generate and/or cause automated intervention at the subscriber.

26. The method of claim 25 , wherein said intervention is provided using one or more APIs on devices on the subscriber's network.

27. A non-transitory computer-readable medium with one or more computer programs stored therein that, when executed by one or more processors of a device, cause the one or more processors to perform the operations of the method of claim 14 .