Adaptive network monitoring with tuneable elastic granularity
Embodiments are directed to monitoring network traffic using network computers. Monitoring triggers associated with one or more conditions and one or more actions may be provided. A monitoring engine may monitor information that is associated with network traffic associated with networks based on an inspection detail level. The monitoring engine may compare the monitored information to the conditions associated with the monitoring triggers. The monitoring engine may activate one or more monitoring triggers based on a result of the comparison. The monitoring engine may modify the inspection detail level based on the actions associated with the activated monitoring triggers to increase the amount of the information monitored by the monitoring engine. An analysis engine may provide analysis of the network traffic based on the monitored information.
1. A method for monitoring network traffic using one or more network computers, wherein execution of instructions by the one or more network computers perform the method comprising:
instantiating a monitoring engine to perform actions, including:
providing one or more monitoring triggers, wherein each monitoring trigger is associated with one or more conditions and one or more actions;
monitoring information that is associated with network traffic associated with one or more networks based on an inspection detail level;
comparing the monitored information to the one or more conditions associated with the one or more monitoring triggers;
adaptively activating one or more of the one or more monitoring triggers based on a result of the comparison; and
modifying the inspection detail level based on the one or more actions associated with the one or more activated monitoring triggers and an available amount of one or more of compute, data storage or network resources, wherein the modification of the inspection detail level initiates or stops deep packet detail level inspection of packets captured in an amount of the monitored information for the one or more activated monitoring triggers provided by the monitoring engine; and
instantiating an analysis engine to perform actions, including, providing analysis of the network traffic based on the inspected packets of the amount of monitored information.
2. The method of claim 1 , wherein comparing the monitored information to the one or more conditions associated with the one or more monitoring triggers, further comprises, instantiating a machine learning engine to evaluate the monitored information based on one or more machine learning models, wherein a result of the evaluation is included in the comparison.
3. The method of claim 1 , further comprising, instantiating an alert engine to perform actions, including:
providing one or more alert triggers, wherein each alert trigger is associated with one or more conditions and one or more actions;
activating one or more of the one or more alert triggers based on the result of the comparison between the information to the one or more conditions associated with the one or more alert triggers; and
providing one or more alerts based on the one or more activated alert triggers.
4. The method of claim 1 , wherein the monitoring engine performs actions further comprising:
providing a resource budget that is associated with the one or more networks and the one or more activated monitoring triggers; and
when a total resource cost associated with the one or more activated monitoring triggers exceeds the resource budget, allocating a portion of the resource budget to one or more of the one or more activated monitoring triggers based on one or more of one or more priority scores that are associated with each of the one or more monitoring triggers, or one or more machine learning evaluations performed by a machine learning engine.
5. The method of claim 1 , wherein the monitoring engine performs actions further comprising:
providing a resource budget that is associated with the one or more networks and the one or more activated monitoring triggers, wherein the portion of the resource budget that is allocated to the one or more of the one or more monitoring triggers is based on a resource cost that is associated with each of the one or more monitoring triggers; and
de-activating a remainder of the one or more activated monitoring triggers that are excluded, wherein allocating the resource cost associated with each deactivated monitoring trigger to the resource budget exceeds the resource budget.
6. The method of claim 1 , wherein increasing the amount of the information monitored by the monitoring engine, further comprises performing one or more of: collecting more data, collecting different data, collecting data for additional agents, capturing network packets, or capturing increased portions of network packets.
7. The method of claim 1 , wherein the actions of the one or more activated monitoring triggers further comprise increasing the inspection detail based on one or more occurrences of a file access event, wherein additional information is analyzed to determine whether the one or more occurrences of file access event is a malicious attack on a file server.
8. The method of claim 1 , wherein the actions of the one or more activated monitoring triggers further comprising:
identifying monitored information that is associated with an application; and
when a behavior of a user in communication with the application matches one or more of behavior corresponding to a malicious state machine, or behavior classified as malicious by a machine learning engine, increasing the inspection detail.
9. A processor readable non-transitory storage media that includes instructions for monitoring network traffic using one or more network monitoring computers, wherein execution of the instructions by the one or more network computers perform the method comprising:
instantiating a monitoring engine to perform actions, including:
providing one or more monitoring triggers, wherein each monitoring trigger is associated with one or more conditions and one or more actions;
monitoring information that is associated with network traffic that is associated with one or more networks based on an inspection detail level;
comparing the monitored information to the one or more conditions associated with the one or more monitoring triggers;
adaptively activating one or more of the one or more monitoring triggers based on a result of the comparison; and
modifying the inspection detail level based on the one or more actions associated with the one or more activated monitoring triggers and an available amount of one or more of compute, data storage or network resources, wherein the modification of the inspection detail level initiates or stops deep packet detail level inspection of packets captured in an amount of the monitored information for the one or more activated monitoring triggers provided by the monitoring engine.
10. The media of claim 9 , wherein comparing the monitored information to the one or more conditions associated with the one or more monitoring triggers, further comprises, instantiating a machine learning engine to evaluate the monitored information based on one or more machine learning models, wherein a result of the evaluation is included in the comparison.
11. The media of claim 9 , further comprising:
instantiating an alert engine to perform actions, including:
providing one or more alert triggers, wherein each alert trigger is associated with one or more conditions and one or more actions;
activating one or more of the one or more alert triggers based on the result of the comparison between the information to the one or more conditions associated with the one or more alert triggers; and
providing one or more alerts based on the one or more activated alert triggers.
12. The media of claim 9 , wherein the monitoring engine performs actions further comprising:
providing a resource budget that is associated with the one or more networks and the one or more activated monitoring triggers; and
when a total resource cost associated with the one or more activated monitoring triggers exceeds the resource budget, allocating a portion of the resource budget to one or more of the one or more activated monitoring triggers based on one or more of one or more priority scores that are associated with each of the one or more monitoring triggers, or one or more machine learning evaluations performed by a machine learning engine.
13. The media of claim 9 , wherein the monitoring engine performs actions further comprising:
providing a resource budget that is associated with the one or more networks and the one or more activated monitoring triggers, wherein the portion of the resource budget that is allocated to the one or more of the one or more monitoring triggers is based on a resource cost that is associated with each of the one or more monitoring triggers; and
de-activating a remainder of the one or more activated monitoring triggers that are excluded, wherein allocating the resource cost associated with each deactivated monitoring trigger to the resource budget exceeds the resource budget.
14. The media of claim 9 , wherein increasing the amount of the information monitored by the monitoring engine, further comprises performing one or more of: collecting more data, collecting different data, collecting data for additional agents, capturing network packets, or capturing increased portions of network packets.
15. The media of claim 9 , wherein the actions of the one or more activated monitoring triggers further comprise increasing the inspection detail based on one or more occurrences of a file access event, wherein additional information is analyzed to determine whether the one or more occurrences of file access event is a malicious attack on a file server.
16. The media of claim 9 , wherein the actions of the one or more activated monitoring triggers further comprising:
identifying monitored information that is associated with an application; and
when a behavior of a user in communication with the application matches one or more of behavior corresponding to a malicious state machine, or behavior classified as malicious by a machine learning engine, increasing the inspection detail.
17. A system for monitoring network traffic in a network:
one or more network computers, comprising:
a transceiver that communicates over the network;
a memory that stores at least instructions; and
one or more processors that execute instructions that perform actions, including:
instantiating a monitoring engine to perform actions, including:
providing one or more monitoring triggers, wherein each monitoring trigger is associated with one or more conditions and one or more actions;
monitoring information that is associated with network traffic that is associated with one or more networks based on an inspection detail level;
comparing the monitored information to the one or more conditions associated with the one or more monitoring triggers;
adaptively activating one or more of the one or more monitoring triggers based on a result of the comparison; and
modifying the inspection detail level based on the one or more actions associated with the one or more activated monitoring triggers and an available amount of one or more of compute, data storage or network resources, wherein the modification of the inspection detail level initiates or stops deep packet detail level inspection of packets captured in an amount of the monitored information for the one or more activated monitoring triggers provided by the monitoring engine; and
instantiating an analysis engine to perform actions, including, providing analysis of the network traffic based on the inspected packets of the amount of monitored information; and
one or more client computers, comprising:
a transceiver that communicates over the network;
a memory that stores at least instructions; and
one or more processors that execute instructions that perform actions, including:
providing one or more portions of the network traffic.
18. The system of claim 17 , wherein comparing the monitored information to the one or more conditions associated with the one or more monitoring triggers, further comprises, instantiating a machine learning engine to evaluate the monitored information based on one or more machine learning models, wherein a result of the evaluation is included in the comparison.
19. The system of claim 17 , further comprising, instantiating an alert engine to perform actions, including:
providing one or more alert triggers, wherein each alert trigger is associated with one or more conditions and one or more actions;
activating one or more of the one or more alert triggers based on the result of the comparison between the information to the one or more conditions associated with the one or more alert triggers; and
providing one or more alerts based on the one or more activated alert triggers.
20. The system of claim 17 , wherein the monitoring engine performs actions further comprising:
providing a resource budget that is associated with the one or more networks and the one or more activated monitoring triggers; and
when a total resource cost associated with the one or more activated monitoring triggers exceeds the resource budget, allocating a portion of the resource budget to one or more of the one or more activated monitoring triggers based on one or more of one or more priority scores that are associated with each of the one or more monitoring triggers, or one or more machine learning evaluations performed by a machine learning engine.
21. The system of claim 17 , wherein the monitoring engine performs actions further comprising:
providing a resource budget that is associated with the one or more networks and the one or more activated monitoring triggers, wherein the portion of the resource budget that is allocated to the one or more of the one or more monitoring triggers is based on a resource cost that is associated with each of the one or more monitoring triggers; and
de-activating a remainder of the one or more activated monitoring triggers that are excluded, wherein allocating the resource cost associated with each deactivated monitoring trigger to the resource budget exceeds the resource budget.
22. The system of claim 17 , wherein increasing the amount of the information monitored by the monitoring engine, further comprises performing one or more of: collecting more data, collecting different data, collecting data for additional agents, capturing network packets, or capturing increased portions of network packets.
23. The system of claim 17 , wherein the actions of the one or more activated monitoring triggers further comprise increasing the inspection detail based on one or more occurrences of a file access event, wherein additional information is analyzed to determine whether the one or more occurrences of file access event is a malicious attack on a file server.
24. A network computer for monitoring communication over a network between two or more computers, comprising:
a transceiver that communicates over the network;
a memory that stores at least instructions; and
one or more processors that execute instructions that perform actions, including:
instantiating a monitoring engine to perform actions, including:
providing one or more monitoring triggers, wherein each monitoring trigger is associated with one or more conditions and one or more actions;
monitoring information that is associated with network traffic that is associated with one or more networks based on an inspection detail level;
comparing the monitored information to the one or more conditions associated with the one or more monitoring triggers;
adaptively activating one or more of the one or more monitoring triggers based on a result of the comparison; and
modifying the inspection detail level based on the one or more actions associated with the one or more activated monitoring triggers and an available amount of one or more of compute, data storage or network resources, wherein the modification of the inspection detail level initiates or stops deep packet detail level inspection of packets captured in an amount of the monitored information for the one or more activated monitoring triggers provided by the monitoring engine; and
instantiating an analysis engine to perform actions, including, providing analysis of the network traffic based on the inspected packets of the amount of monitored information.
25. The network computer of claim 24 , wherein comparing the monitored information to the one or more conditions associated with the one or more monitoring triggers, further comprises, instantiating a machine learning engine to evaluate the monitored information based on one or more machine learning models, wherein a result of the evaluation is included in the comparison.
26. The network computer of claim 24 , further comprising, instantiating an alert engine to perform actions, including:
providing one or more alert triggers, wherein each alert trigger is associated with one or more conditions and one or more actions;
activating one or more of the one or more alert triggers based on the result of the comparison between the information to the one or more conditions associated with the one or more alert triggers; and
providing one or more alerts based on the one or more activated alert triggers.
27. The network computer of claim 24 , wherein the monitoring engine performs actions further comprising:
providing a resource budget that is associated with the one or more networks and the one or more activated monitoring triggers; and
when a total resource cost associated with the one or more activated monitoring triggers exceeds the resource budget, allocating a portion of the resource budget to one or more of the one or more activated monitoring triggers based on one or more of one or more priority scores that are associated with each of the one or more monitoring triggers, or one or more machine learning evaluations performed by a machine learning engine.
28. The network computer of claim 24 , wherein the monitoring engine performs actions further comprising:
providing a resource budget that is associated with the one or more networks and the one or more activated monitoring triggers, wherein the portion of the resource budget that is allocated to the one or more of the one or more monitoring triggers is based on a resource cost that is associated with each of the one or more monitoring triggers; and
de-activating a remainder of the one or more activated monitoring triggers that are excluded, wherein allocating the resource cost associated with each deactivated monitoring trigger to the resource budget exceeds the resource budget.
29. The network computer of claim 24 , wherein increasing the amount of the information monitored by the monitoring engine, further comprises performing one or more of: collecting more data, collecting different data, collecting data for additional agents, capturing network packets, or capturing increased portions of network packets.
30. The network computer of claim 24 , wherein the actions of the one or more activated monitoring triggers further comprising:
identifying monitored information that is associated with an application; and
when a behavior of a user in communication with the application matches one or more of behavior corresponding to a malicious state machine, or behavior classified as malicious by a machine learning engine, increasing the inspection detail.