Systems and methods for anomaly detection on resource activity logs
The present disclosure includes systems and methods for anomaly detection on resource activity logs. Various embodiments include collecting resource activity data from a plurality of resources in a cloud environment, the resource activity data including information related to a plurality of events associated with the plurality of resources in the cloud environment; aggregating and performing one or more calculations on the resource activity data to represent the plurality of resources in vector form; determining a probability of a sequence of events to be executed by a resource of the plurality of resources based on the vector form of the resource; and determining an anomaly score for the sequence of events being executed by the resource based on the probability.
1. A method comprising steps of:
collecting resource activity data at a cloud edge via in-line real time monitoring from a plurality of resources in a cloud environment, the resource activity data including information related to a plurality of events associated with the plurality of resources in the cloud environment;
aggregating and performing one or more calculations on the resource activity data to represent the plurality of resources in vector form, wherein the aggregating is performed via a supervised Machine Learning (ML) model and includes grouping the plurality of events into a plurality of triples, wherein each of the plurality of triples includes 3 events executed by a resource of the plurality of resources in sequence;
determining a probability of a sequence of events to be executed by a resource of the plurality of resources based on the vector form of the resource; and
determining an anomaly score for the sequence of events being executed by the resource based on the probability.
2. The method of claim 1 , wherein the collecting resource activity data is performed for a preconfigured historical time span based on a training set period, and wherein the resource activity data is provided to the ML model for training.
3. The method of claim 1 , wherein the one or more calculations include calculating a number of occurrences that each of the plurality of resources executed each of the plurality of triples.
4. The method of claim 3 , wherein the sequence of events is a triple, and wherein determining the probability of the triple to be executed by the resource is based on the number of occurrences that the resource executed the triple.
5. The method of claim 1 , wherein the probability of a sequence of events to be executed by the plurality of resources is utilized to determine similarities between the plurality of resources in the cloud environment.
6. The method of claim 5 , wherein determining an anomaly score for the sequence of events being executed by the resource is based on the probability of the sequence of events being executed by the resource and the probability of the sequence of events being executed by one or more similar resources.
7. The method of claim 5 , wherein the steps further comprise:
providing via a Graphical User Interface (GUI) a graphical representation of the similarities between the plurality of resources in the cloud environment, wherein the GUI is adapted for a user to configure policies.
8. The method of claim 1 , wherein the steps further comprise:
performing inline monitoring of the cloud environment, wherein the inline monitoring includes collecting real-time resource activity data;
determining an anomaly score for a sequence of events executed by a resource; and
sending an alert based on the anomaly score being below a threshold.
9. The method of claim 1 , wherein the resource activity data includes any of a source Internet Protocol (IP) address, an event time, and an event name.
10. A non-transitory computer-readable medium comprising instructions that, when executed, cause one or more processors to perform steps of:
collecting resource activity data at a cloud edge via in-line real time monitoring from a plurality of resources in a cloud environment, the resource activity data including information related to a plurality of events associated with the plurality of resources in the cloud environment;
aggregating and performing one or more calculations on the resource activity data to represent the plurality of resources in vector form, wherein the aggregating is performed via a supervised Machine Learning (ML) model and includes grouping the plurality of events into a plurality of triples, wherein each of the plurality of triples includes 3 events executed by a resource of the plurality of resources in sequence;
determining a probability of a sequence of events to be executed by a resource of the plurality of resources based on the vector form of the resource; and
determining an anomaly score for the sequence of events being executed by the resource based on the probability.
11. The non-transitory computer-readable medium of claim 10 , wherein the collecting resource activity data is performed for a preconfigured historical time span.
12. The non-transitory computer-readable medium of claim 10 , wherein the one or more calculations include calculating a number of occurrences that each of the plurality of resources executed each of the plurality of triples.
13. The non-transitory computer-readable medium of claim 12 , wherein the sequence of events is a triple, and wherein determining the probability of the triple to be executed by the resource is based on the number of occurrences that the resource executed the triple.
14. The non-transitory computer-readable medium of claim 10 , wherein the probability of a sequence of events to be executed by the plurality of resources is utilized to determine similarities between the plurality of resources in the cloud environment.
15. The non-transitory computer-readable medium of claim 14 , wherein determining an anomaly score for the sequence of events being executed by the resource is based on the probability of the sequence of events being executed by the resource and the probability of the sequence of events being executed by one or more similar resources.
16. The non-transitory computer-readable medium of claim 14 , wherein the steps further comprise:
providing a graphical representation of the similarities between the plurality of resources in the cloud environment.
17. The non-transitory computer-readable medium of claim 10 , wherein the steps further comprise:
performing inline monitoring of the cloud environment, wherein the inline monitoring includes collecting real-time resource activity data;
determining an anomaly score for a sequence of events executed by a resource; and
sending an alert based on the anomaly score being below a threshold.
18. The non-transitory computer-readable medium of claim 10 , wherein the resource activity data includes any of a source Internet Protocol (IP) address, an event time, and an event name.