Systems and methods for identifying attack patterns or suspicious activity in client networks

1. A system for identifying attack patterns or suspicious activity in computer networks relating to a plurality of clients, comprising:

a memory;

a normalizer configured to receive raw data from the computer networks and normalize the raw data into one or more structured data sets;

a profile builder that builds one or more baseline activity profiles for each client of the plurality of clients or entities associated therewith using the one or more structured data sets;

at least one primitive creator configured to generate primitives that are possibly indicative or suggestive of attack patterns or suspicious activity in the computer networks, wherein the at least one primitive creator extracts features from the one or more structured data sets, and create one or more primitives for each extracted feature that are found to occur below one or more selected frequency thresholds based on information in the one or more baseline activity profiles, each of the one or more selected frequency thresholds corresponding to one or more of the plurality of clients or entities; and

a compromise detector that receives primitives from the at least one primitive creator, wherein the compromise detector organizes the received primitives into groups according to prescribed grouping information and identifies combinations or sequences of primitives in the groups, and for each identified combination or sequence of primitives that meets one or more selected criteria, the compromise detector generates a compromise event to be provided to affected clients;

wherein, in response to a determination that the primitives received from the at least one primitive creator includes a number of primitives that exceed a prescribed number, match a specific combination or sequence of primitives that relate to known attack patterns or suspicious activities, are likely to correspond to one or more attack patterns or suspicious activities according to a prescribed probability, or combinations thereof, the compromise detector generates the compromise event.

2. The system of claim 1 , wherein the profile builder populates entity profiles for each entity associated with the clients with historical information related to the extracted features, and the profile builder populates the client profiles for each of the clients with historical information related to the extracted features.

3. The system of claim 2 , wherein the at least one primitive creator compares the extracted features to the historical information in a corresponding entity profile, and if the extracted features in comparison to the historical information in the entity profile meet a prescribed entity frequency threshold, the at least one primitive creator establishes a staged primitive to be investigated further for each identified primitive feature meeting the prescribed entity frequency threshold, and wherein the at least one primitive creator further compares the extracted features corresponding to each staged primitive to the historical information in a corresponding client profile, and if the identified primitives features corresponding to a staged primitive in comparison to the historical information in the client profile meet a prescribed client frequency threshold, the at least one primitive creator creates a corresponding primitive to be provided to the compromise detector.

4. The system of claim 1 , further comprising at least one additional primitive creator that creates primitives independent of historical client or entity information.

5. The system of claim 4 , wherein the at least one additional primitive creator includes an indicator of compromise primitive creator, a business email compromise primitive creator, a cloud account hijacking primitive creator, or combinations thereof.

6. The system of claim 1 , wherein the compromise detector groups the primitives received from the at least one primitive creator in session windows that are open for a prescribed time period.

7. The system of claim 1 , further comprising a client value machine learning system that receives client feedback information as one or more inputs, and generates one or more outputs that are provided to the compromise detector to facilitate suppression of compromise events below a certain probability threshold.

8. The system of claim 1 , wherein the raw data includes unstructured logs aggregated from the computer networks and wherein the normalizer normalizes the unstructured logs into the structured data sets, each having a prescribed schema.

9. A method for identifying attack patterns or suspicious activity in computer networks relating to a plurality of clients, comprising:

receiving data from the computer networks;

building one or more baseline activity profiles for each client of the plurality of clients or entities associated therewith;

extracting features from the received data for comparison with information in the one or more baseline activity profiles,

creating primitives that are possibly indicative or suggestive of attack patterns or suspicious activity in the computer networks for features that occur below a plurality of selected frequency thresholds based on information in the one or more baseline activity profiles, wherein each of the plurality of selected frequency thresholds correspond to one or more of the plurality of clients or entities;

organizing the primitives into groups according to prescribed grouping information, identifying combinations or sequences of primitives in the groups;

in response to a determination that an identified combination or sequence of primitives includes a number of primitives that exceed a prescribed number, match a specific combination or sequence of primitives that relate to known attack patterns or suspicious activities, are likely to correspond to one or more attack patterns or suspicious activities according to a prescribed probability, or combinations thereof, generating an event; and

notifying affected clients of each generated event to indicate an identified attack pattern or suspicious activity and facilitate investigation or remediation thereof.

10. The method of claim 9 , further comprising:

populating entity profiles for each entity associated with the clients with information related to the extracted features, and

populating client profiles for each of the clients with information related to the extracted features.

11. The method of claim 10 , further comprising:

comparing the extracted features to historical information in a corresponding entity profile;

establishing a staged primitive to be investigated further for each extracted feature meeting a prescribed entity frequency threshold in comparison to the historical information in the entity profile;

comparing the extracted features corresponding to each staged primitive to the historical information in a corresponding client profile; and

creating a corresponding primitive for each extracted feature corresponding to a staged primitive that meets a prescribed client frequency threshold in comparison to the historical information in the corresponding client profile.

12. The method of claim 10 , further comprising creating additional primitives independent of historical client or entity information.

13. The method of claim 10 , further comprising grouping the received primitives into session windows.

14. The method of claim 13 , further comprising generating events if the received primitives in the session windows include a number of primitives that exceed a prescribed number, match a specific combination or sequence of primitives that relate to known attack patterns or suspicious activities, or are likely to correspond to one or more attack patterns or suspicious activities according to a prescribed probability.

15. The method of claim 10 , further comprising:

aggregating unstructured logs from networks managed by the plurality of clients, and

normalizing the unstructured logs into normalized logs having a prescribed schema.

16. A system for identifying attack patterns or suspicious activity on computer networks related to a plurality of clients, comprising:

one or more processors and at least one memory having stored therein instructions that when executed by the one or more processors, cause the system to:

receive data from the computer networks;

build one or more baseline activity profiles for each client of the plurality of clients or entities associated therewith;

extract features from the data received from the computer networks for comparison with information in the one or more baseline activity profiles;

create primitives that are possibly indicative or suggestive of attack patterns or suspicious activity in the computer networks for features that occur below each of a plurality selected frequency thresholds based on information in the one or more baseline activity profiles, each of the plurality of selected frequency thresholds corresponding to one or more of the plurality of clients or entities;

organize the primitives into groups according to prescribed grouping information, identifying combinations or sequences of primitives in the groups;

in response to a determination that an identified combination or sequence of primitives includes a number of primitives that exceed a prescribed number, match a specific combination or sequence of primitives that relate to known attack patterns or suspicious activities, are likely to correspond to one or more attack patterns or suspicious activities according to a prescribed probability, or combinations thereof, generating an event; and

notify affected clients of each generated event indicating an identified attack pattern or suspicious activity and facilitate investigation or remediation thereof.