System and method for detecting network activity of interest
A network activity detection system is trained to detect network activities of interest such as threats by malicious computer data. The training involves distilling the characteristics of known network activities of interest (e.g., intrusion by computer viruses, exploits, worms, or the like) into a minimal set of meta-expressions. At run-time, the network activity detection system combines the minimal set of meta-expressions with efficient computer algorithms for evaluating meta-expressions to detect known network activities of interest, as well as their unknown variants, among an unknown set of network activity. The network activity detection system may produce appropriate responses upon the detection of network activities of interest.
1. A computer-implemented method for detecting a network activity of interest, the method comprising:
obtaining, by one or more processors, a plurality of network packets from a network, wherein the obtained plurality of network packets includes the network activity of interest;
creating, by the one or more processors, a plurality of combined packets from the plurality of network packets, wherein the plurality of combined packets includes:
a first combined packet, and
a second combined packet different from the first combined packet;
creating, by the one or more processors, a first sequence by converting content of at least a portion of the first combined packet;
creating, by the one or more processors, a second sequence by converting content of at least a portion of the second combined packet;
determining, by the one or more processors, a similarity metric between the first sequence and the second sequence;
creating, by the one or more processors, a third sequence based on the similarity metric, wherein the third sequence is a meta-expression that corresponds to the presence of the network activity of interest in the network traffic; and
storing the meta-expression, wherein the stored meta-expression is used to detect the presence of the network activity of interest.
2. The computer-implemented method of claim 1 , wherein:
the network activity of interest is a threat based on a computer virus.
3. The computer-implemented method of claim 1 , wherein the creating of the first combined packet comprises:
identifying a first network packet, of the plurality of network packets, representing a request;
identifying a second network packet, of the plurality of network packets, representing an acknowledgement, wherein the second network packet: occurs after the first network packet, and represents an acknowledgement corresponding to the first network packet; and
identifying a third network packet of the plurality of network packets, wherein the third network packet occurs between the first network packet and the second network packet.
4. The computer-implemented method of claim 1 , wherein:
creating the first sequence includes converting content of at least a portion of the first combined packet into a first plurality of integers, wherein the first sequence includes the first plurality of integers; and
creating the second sequence includes converting content of at least a portion of the second combined packet into a second plurality of integers, wherein the second sequence includes the second plurality of integers.
5. The computer-implemented method of claim 4 , wherein:
Determining the similarity metric between the first sequence and the second sequence is based on a distance function.
6. The computer-implemented method of claim 5 , wherein the third sequence comprises a third plurality of integers common to the first sequence and the second sequence, in a first order.
7. The computer-implemented method of claim 6 , wherein:
the similarity metric represents a difference between a position of an integer in the first sequence and a position of the integer in the second sequence, and
the integer is included in the third sequence if the difference is within a first pre-determined distance.
8. The computer-implemented method of claim 7 , wherein:
the first predetermined distance is four.
9. The computer-implemented method of claim 7 , wherein:
the first, second, and third plurality of integers are 8-bit integers.
10. The computer-implemented method of claim 9 , wherein:
the converting content of the at least a portion of the first combined packets into the first plurality of integers comprises converting bitwise the content of the at least a portion of the first combined packet into the first plurality of integers.
11. A networking device for detecting a network activity of interest, comprising:
a network port for connecting to a network infrastructure, wherein the network port is adapted to obtain a plurality of network packets, wherein the obtained plurality of network packets includes the network activity of interest;
a processor connected to the network port, wherein the processor is adapted to:
create a plurality of combined packets from the plurality of network packets, wherein the plurality of combined packets includes:
a first combined packet, and
a second combined packet different from the first combined packet;
create a first sequence by converting content of at least a portion of the first combined packet;
create a second sequence by converting content of at least a portion of the second combined packet;
determine a similarity metric between the first sequence and the second sequence;
create a third sequence based on the similarity metric, wherein the third sequence is a meta-expression that corresponds to the presence of the network activity of interest in the network traffic; and
a memory connected to the processor, wherein the memory is adapted to store the meta-expression, wherein the stored meta-expression is used to detect the presence of the network activity of interest.
12. The networking device of claim 11 , wherein:
the network activity of interest is a threat based on a computer virus.
13. The networking device of claim 11 , wherein the processor connected to the network port, is further adapted to:
identify a first network packet, of the plurality of network packets, representing a request;
identify a second network packet, of the plurality of network packets, representing an acknowledgement, wherein the second network packet: occurs after the first network packet, and represents an acknowledgement corresponding to the first network packet; and
identify a third network packet of the plurality of network packets, wherein the third network packet occurs between the first network packet and the second network packet.
14. The networking device of claim 11 , wherein the processor connected to the network port, is further adapted to:
create the first sequence by converting content of at least a portion of the first combined packet into a first plurality of integers, wherein the first sequence includes the first plurality of integers; and
create the second sequence by converting content of at least a portion of the second combined packet into a second plurality of integers, wherein the second sequence includes the second plurality of integers.
15. The networking device of claim 14 , wherein the processor connected to the network port, is further adapted to:
Determine the similarity metric between the first sequence and the second sequence is based on a distance function.
16. The networking device of claim 15 , wherein the third sequence comprises a third plurality of integers common to the first sequence and the second sequence, in a first order.
17. The networking device of claim 16 , wherein:
the similarity metric represents a difference between a position of an integer in the first sequence and a position of the integer in the second sequence, and
the integer is included in the third sequence if the difference is within a first pre-determined distance.
18. The networking device of claim 14 , wherein:
the first, second, and third plurality of integers are 8-bit integers.
19. The networking device of claim 14 , wherein:
the converting content of the at least a portion of the first combined packets into the first plurality of integers comprises converting bitwise the content of the at least a portion of the first combined packet into 8-bit integers.
20. A non-transitory computer-readable storage medium having computer-executable instructions for detecting a network activity of interest, computer-executable instructions comprising instructions for:
obtaining, by one or more processors, a plurality of network packets from a network, wherein the obtained plurality of network packets includes the network activity of interest;
creating, by the one or more processors, a plurality of combined packets from the plurality of network packets, wherein the plurality of combined packets includes:
a first combined packet, and
a second combined packet different from the first combined packet;
creating, by the one or more processors, a first sequence by converting content of at least a portion of the first combined packet;
creating, by the one or more processors, a second sequence by converting content of at least a portion of the second combined packet;
determining, by the one or more processors, a similarity metric between the first sequence and the second sequence;
creating, by the one or more processors, a third sequence based on the similarity metric, wherein the third sequence is a meta-expression that corresponds to the presence of the network activity of interest in the network traffic; and
storing the meta-expression, wherein the stored meta-expression is used to detect the presence of the network activity of interest.