Detecting network anomalies by probabilistic modeling of argument strings with markov chains
Systems, methods, and media for detecting network anomalies are provided. In some embodiments, a training dataset of communication protocol messages having argument strings is received. The content and structure associated with each of the argument strings is determined and a probabilistic model is trained using the determined content and structure of each of the argument strings. A communication protocol message having an argument string that is transmitted from a first processor to a second processor across a computer network is received. The received communication protocol message is compared to the probabilistic model and then it is determined whether the communication protocol message is anomalous.
1. A method for detecting network anomalies, the method comprising:
receiving, using a hardware processor, a communication protocol message having an argument string that is transmitted from a first processor to a second processor across a computer network;
determining, using the hardware processor, whether the communication protocol message is anomalous based on n-grams in the argument string by applying a probabilistic model that uses at least one Markov chain to generate a score that indicates a deviation of the argument string of the communication protocol message from previously received communication protocol messages and determining whether the score is greater than a threshold value; and
performing, using the hardware processor, a predetermined action in response to determining that the communication protocol message is anomalous.
2. The method of claim 1 , wherein the at least one Markov chain is specified by a gram size and wherein the probabilistic model was trained based on the gram size.
3. The method of claim 1 , wherein the at least one Markov chain is specified by a mixture size that specifies a number of Markov chains to use in the probabilistic model and wherein the at least one Markov chain is a mixture of Markov chains specified by the mixture size.
4. The method of claim 1 , further comprising:
receiving a plurality of packets transmitted from the first processor to the second processor across the computer network;
assembling the plurality of packets to form the communication protocol message; and
extracting the argument string from the communication protocol message.
5. The method of claim 1 , wherein the communication protocol message is a Hypertext Transfer Protocol (HTTP) request message.
6. The method of claim 1 , wherein the probabilistic model was trained based on content and structure of the argument string included in each of a plurality of communication protocol messages.
7. The method of claim 6 , wherein the content and the structure associated with each of the argument strings further comprises one or more variable names, a corresponding argument value for each of the variable names, and a layout with respect to each of the variable names.
8. The method of claim 1 , wherein the at least one Markov chain comprises one or more Markov models of transition probabilities in n-grams of the communication protocol message.
9. The method of claim 1 , wherein the predetermined action comprises issuing an alert.
10. The method of claim 1 , further comprises calculating a normality score for the received communication protocol message using the probabilistic model.
11. A system for detecting network anomalies, the system comprising:
a hardware processor that is configured to:
receive a communication protocol message having an argument string that is transmitted from a first processor to a second processor across a computer network;
determine whether the communication protocol message is anomalous based on n-grams in the argument string by applying a probabilistic model that uses at least one Markov chain to generate a score that indicates a deviation of the argument string of the communication protocol message from previously received communication protocol messages and determining whether the score is greater than a threshold value; and
perform a predetermined action in response to determining that the communication protocol message is anomalous.
12. The system of claim 11 , wherein the at least one Markov chain is specified by a gram size and wherein the probabilistic model was trained based on the gram size.
13. The system of claim 11 , wherein the at least one Markov chain is specified by a mixture size that specifies a number of Markov chains to use in the probabilistic model and wherein the at least one Markov chain is a mixture of Markov chains specified by the mixture size.
14. The system of claim 11 , wherein the hardware processor is further configured to:
receive a plurality of packets transmitted from the first processor to the second processor across the computer network;
assemble the plurality of packets to form the communication protocol message; and
extract the argument string from the communication protocol message.
15. The system of claim 11 , wherein the communication protocol message is a Hypertext Transfer Protocol (HTTP) request message.
16. The system of claim 11 , wherein the probabilistic model was trained based on content and structure of the argument string included in each of a plurality of communication protocol messages.
17. The system of claim 16 , wherein the content and the structure associated with each of the argument strings further comprises one or more variable names, a corresponding argument value for each of the variable names, and a layout with respect to each of the variable names.
18. The system of claim 11 , wherein the at least one Markov chain comprises one or more Markov models of transition probabilities in n-grams of the communication protocol message.
19. The system of claim 11 , wherein the predetermined action comprises issuing an alert.
20. The system of claim 11 , wherein the hardware processor further configured to calculate a normality score for the received communication protocol message using the probabilistic model.
21. A non-transitory computer-readable medium containing computer-executable instructions that, when executed by a processor, cause the processor to perform method for detecting network anomalies, the method comprising:
receiving, using a hardware processor, a communication protocol message having an argument string that is transmitted from a first processor to a second processor across a computer network;
determining, using the hardware processor, whether the communication protocol message is anomalous based on n-grams in the argument string by applying a probabilistic model that uses at least one Markov chain to generate a score that indicates a deviation of the argument string of the communication protocol message from previously received communication protocol messages and determining whether the score is greater than a threshold value; and
performing, using the hardware processor, a predetermined action in response to determining that the communication protocol message is anomalous.