Application traffic and runtime behavior learning and enforcement
Systems and methods for learning behavioral activity correlations. A method includes intercepting a plurality of requests, wherein each of the plurality of requests is directed to a respective destination entity of a plurality of destination entities; creating a request queue by queueing the plurality of requests; inspecting contents of the plurality of requests; separately forwarding each intercepted request to its respective destination entity based on the request queue; monitoring runtime output of each of the plurality of destination entities, wherein the runtime output includes behavioral activities of the plurality of destination entities; and training a machine learning model based on the contents of the plurality of requests the runtime output of each of the plurality of destination entities, wherein the machine learning model is trained to output request-output correlations between groups of requests and subsequent behavioral activities.
1 . A method comprising:
intercepting a first plurality of requests, wherein each of the first plurality of requests is directed to a corresponding one of a plurality of entities;
buffering the first plurality of requests in a queue;
based on forwarding requests in the first plurality of requests in the queue to respective entities in the plurality of entities,
monitoring a first plurality of events at the respective entities in the plurality of entities; and
correlating requests in the first plurality of requests with corresponding events in the first plurality of events;
training a machine learning model on the correlated requests and corresponding events to learn first correlations that indicate events in the first plurality of events and corresponding requests in the first plurality of requests that caused those events in the first plurality of events
inputting a second plurality of requests and behavioral data for a second plurality of events at entities in the plurality of entities into the trained machine learning model to obtain second correlations as output, wherein the second correlations indicate events in the second plurality of events and corresponding requests in the second plurality of requests that caused those events in the second plurality of events; and
determining at least one of allowed and forbidden ones of the second correlations.
2 . The method of claim 1 , further comprising:
detecting one or more impermissible events at a first entity in the plurality of entities based on monitoring events at the plurality of entities;
obtaining a second plurality of requests in a time period preceding the detected one or more impermissible events;
inputting the second plurality of requests and the one or more impermissible events into the trained machine learning model to determine a subset of the second plurality of requests correlated with the one or more impermissible events; and
indicating mitigation action at the first entity based, at least in part, on the subset of the second plurality of requests and the one or more impermissible events.
3 . The method of claim 2 , further comprising training a firewall model to detect permissible and impermissible events at each of the plurality of entities, wherein detecting the one or more impermissible events at the first entity comprises inputting metadata for events at the first entity into the firewall model.
4 . The method of claim 3 , wherein training the firewall model to detect permissible and impermissible events comprises training the firewall model according to one or more policies comprising one or more lists of permissible and impermissible request-to-event correlations.
5 . The method of claim 1 , wherein the events at the plurality of entities comprise at least one of filesystem events, process events, network events, and queries to databases.
6 . A non-transitory computer readable medium having stored thereon program code comprising instructions to:
intercept a first plurality of requests, wherein each of the first plurality of requests is directed to a corresponding one of a plurality of entities;
buffer the first plurality of requests in a queue;
based on forwarding requests in the first plurality of requests in the queue to respective entities in the plurality of entities,
monitor a first plurality of events at the respective entities in the plurality of entities; and
correlate requests in the first plurality of requests with corresponding events in the first plurality of events;
train a machine learning model on the correlated requests and corresponding events to learn first correlations that indicate events in the first plurality of events and corresponding requests in the first plurality of requests that caused those events in the first plurality of events;
input a second plurality of requests and behavioral data for a second plurality of events at entities in the plurality of entities into the trained machine learning model to obtain second correlations as output, wherein the second correlations indicate events in the second plurality of events and corresponding requests in the second plurality of requests that caused those events in the second plurality of events; and
determine at least one of allowed and forbidden ones of the second correlations.
7 . A system comprising:
a processor; and
a machine-readable medium having instructions stored thereon that are executable by the processor to cause the system to,
intercept a first plurality of requests, wherein each of the first plurality of requests is directed to a corresponding one of a plurality of entities;
buffer the first plurality of requests in a queue;
based on forwarding requests in the first plurality of requests in the queue to respective entities in the plurality of entities,
monitor a first plurality of events at the respective entities in the plurality of entities; and
correlate requests in the first plurality of requests with corresponding events in the first plurality of events;
train a machine learning model on the correlated requests and corresponding events to learn first correlations that indicate events in the first plurality of events and corresponding requests in the first plurality of requests that caused those events in the first plurality of events;
input a second plurality of requests and behavioral data for a second plurality of events at entities in the plurality of entities into the trained machine learning model to obtain second correlations as output, wherein the second correlations indicate events in the second plurality of events and corresponding requests in the second plurality of requests that caused those events in the second plurality of events; and
determining at least one of allowed and forbidden ones of the second correlations.
8 . The system of claim 7 , wherein the machine-readable medium further has instructions stored thereon executable by the processor to cause the system to:
detect one or more impermissible events at a first entity in the plurality of entities based on monitoring events at the plurality of entities;
obtain a second plurality of requests in a time period preceding the detected one or more impermissible events;
input the second plurality of requests and the one or more impermissible events into the trained machine learning model to determine a subset of the second plurality of requests correlated with the one or more impermissible events; and
indicate mitigation action at the first entity based, at least in part, on the subset of the second plurality of requests and the one or more impermissible events.
9 . The system of claim 8 , wherein the machine-readable medium further has instructions stored thereon executable by the processor to cause the system to train a firewall model to detect permissible and impermissible events at each of the plurality of entities, wherein the instructions executable by the processor to cause the system to detect the one or more impermissible events at the first entity comprise instructions to input metadata for events at the first entity into the firewall model.
10 . The system of claim 9 , wherein the instructions executable by the processor to cause the system to train the firewall model to detect permissible and impermissible events comprise instructions executable by the processor to cause the system to train the firewall model according to one or more policies comprising one or more lists of permissible and impermissible request-to-event correlations.
11 . The computer readable medium of claim 6 , wherein the program code further comprises instructions to:
detect one or more impermissible events at a first entity in the plurality of entities based on monitoring events at the plurality of entities;
obtain a second plurality of requests in a time period preceding the detected one or more impermissible events;
input the second plurality of requests and the one or more impermissible events into the trained machine learning model to determine a subset of the second plurality of requests correlated with the one or more impermissible events; and
indicate mitigation action at the first entity based, at least in part, on the subset of the second plurality of requests and the one or more impermissible events.
12 . The computer readable medium of claim 11 , wherein the program code further comprises instructions to train a firewall model to detect permissible and impermissible events at each of the plurality of entities, wherein the instructions to detect the one or more impermissible events at the first entity comprise instructions to input metadata for events at the first entity into the firewall model.
13 . The computer readable medium of claim 12 , wherein the instructions to train the firewall model to detect permissible and impermissible events comprise instructions to train the firewall model according to one or more policies comprising one or more lists of permissible and impermissible request-to-event correlations.
14 . A method comprising:
grouping metadata of subsets of a first plurality of requests directed to a protected entity, wherein the subsets of requests are differentiated by called process and wherein metadata of each request within a subset of requests indicates a same process;
buffering subsets of requests grouped by called process in a queue;
based on processing subsets of requests in the queue at the protected entity, correlating events in a first plurality of events at the protected entity with respect to each called process with corresponding subsets of the first plurality of requests;
training a machine learning model on the correlated requests and corresponding events at the protected entity, wherein the machine learning model is trained to learn first correlations that indicate events in the first plurality of events and corresponding requests in a first plurality of requests that caused those events in the first plurality of events;
inputting a second plurality of requests and behavioral data for a second plurality of events at the protected entity into the trained machine learning model to obtain second correlations as output, wherein the second correlations indicate events in the second plurality of events and corresponding requests in the second plurality of requests that caused those events in the second plurality of events; and
determining at least one of allowed and forbidden ones of the second correlations.
15 . The method of claim 14 , further comprising:
detecting one or more impermissible events at a protected entity based on monitoring events at the protected entity;
obtaining a second plurality of requests directed to the protected entity in a time period preceding the detected one or more impermissible events;
grouping metadata of subsets of the second plurality of requests, wherein the subsets of requests are differentiated by called process and metadata of each request within a subset of requests indicates a same process;
inputting the second plurality of requests and the one or more impermissible events into the trained machine learning model to determine a subset of the second plurality of requests correlated with the one or more impermissible events; and
indicating mitigation action at the protected entity based, at least in part, on the subset of the second plurality of requests and the one or more impermissible events.
16 . The method of claim 15 , further comprising training a firewall model to detect permissible and impermissible events at the protected entity, wherein detecting the one or more impermissible events at the protected entity comprises inputting metadata for events at the protected entity into the firewall model.
17 . The method of claim 16 , wherein training the firewall model to detect permissible and impermissible events comprises training the firewall model according to one or more policies comprising one or more lists of permissible and impermissible request-to- event correlations.
18 . The method of claim 14 , wherein the events at the protected entity comprise at least one of filesystem events, process events, network events, and queries to databases.
19 . A non-transitory computer readable medium having stored thereon program code comprising instructions to:
group metadata of subsets of a first plurality of requests directed to a protected entity, wherein the subsets of requests are differentiated by called process and wherein metadata of each request within a subset of requests indicates a same process;
buffer subsets of requests grouped by called process in a queue;
based on processing subsets of requests in the queue at the protected entity, correlate events in a first plurality of events at the protected entity with respect to each called process with corresponding subsets of the first plurality of requests;
train a machine learning model on the correlated requests and corresponding events at the protected entity, wherein the machine learning model is trained to learn first correlations that indicate events in the first plurality of events and corresponding requests in the first plurality of requests that caused those events in the first plurality of events;
input a second plurality of requests and behavioral data for a second plurality of events at the protected entity into the trained machine learning model to obtain second correlations as output, wherein the second correlations indicate events in the second plurality of events and corresponding requests in the second plurality of requests that caused those events in the second plurality of events; and
determine at least one of allowed and forbidden ones of the second correlations.
20 . The computer readable medium of claim 19 , wherein the program code further comprises instructions to:
detect one or more impermissible events at a protected entity based on monitoring events at the protected entity;
obtain a second plurality of requests directed to the protected entity in a time period preceding the detected one or more impermissible events;
group metadata of subsets of the second plurality of requests, wherein the subsets of requests are differentiated by called process and metadata of each request within a subset of requests indicates a same process;
input the second plurality of requests and the one or more impermissible events into the trained machine learning model to determine a subset of the second plurality of requests correlated with the one or more impermissible events; and
indicate mitigation action at the protected entity based, at least in part, on the subset of the second plurality of requests and the one or more impermissible events.
21 . The computer readable medium of claim 20 , wherein the program code further comprises instructions to train a firewall model to detect permissible and impermissible events at the protected entity, wherein the instructions to detect the one or more impermissible events at the protected entity comprise instructions to input metadata for events at the protected entity into the firewall model.
22 . The computer readable medium of claim 21 , wherein the instructions to train the firewall model to detect permissible and impermissible events comprise instructions to train the firewall model according to one or more policies comprising one or more lists of permissible and impermissible request-to-event correlations.
23 . A system comprising:
a processor; and
a machine-readable medium having instructions stored thereon that are executable by the processor to cause the system to,
group metadata of subsets of a first plurality of requests directed to a protected entity, wherein the subsets of requests are differentiated by called process and wherein metadata of each request within a subset of requests indicates a same process;
buffer subsets of requests grouped by called process in a queue;
based on processing subsets of requests in the queue at the protected entity, correlate events in a first plurality of events at the protected entity with respect to each called process with corresponding subsets of the first plurality of requests;
train a machine learning model on the correlated requests and corresponding events at the protected entity, wherein the machine learning model is trained to learn first correlations that indicate events in the first plurality of events and corresponding requests in the first plurality of requests that caused those events in the first plurality of events;
input a second plurality of requests and behavioral data for a second plurality of events at the protected entity into the trained machine learning model to obtain second correlations as output, wherein the second correlations indicate events in the second plurality of events and corresponding requests in the second plurality of requests that caused those events in the second plurality of events; and
determine at least one of allowed and forbidden ones of the second correlations.
24 . The system of claim 23 , wherein the machine-readable medium further has instructions stored thereon executable by the processor to cause the system to:
detect one or more impermissible events at a protected entity based on monitoring events at the protected entity;
obtain a second plurality of requests directed to the protected entity in a time period preceding the detected one or more impermissible events;
group metadata of subsets of the second plurality of requests, wherein the subsets of requests are differentiated by called process and metadata of each request within a subset of requests indicates a same process;
input the second plurality of requests and the one or more impermissible events into the trained machine learning model to determine a subset of the second plurality of requests correlated with the one or more impermissible events; and
indicate mitigation action at the protected entity based, at least in part, on the subset of the second plurality of requests and the one or more impermissible events.
25 . The system of claim 24 , wherein the machine-readable medium further has instructions stored thereon executable by the processor to cause the system to train a firewall model to detect permissible and impermissible events at the protected entity, wherein the instructions executable by the processor to cause the system to detect the one or more impermissible events at the protected entity comprise instructions to input metadata for events at the protected entity into the firewall model.
26 . The system of claim 25 , wherein the instructions executable by the processor to cause the system to train the firewall model to detect permissible and impermissible events comprise instructions executable by the processor to cause the system to train the firewall model according to one or more policies comprising one or more lists of permissible and impermissible request-to-event correlations.