Stack-HAC for machine learning based botnet detection
Discussed herein are devices, systems, and methods for detecting anomalous or malicious processes based on a network session. A method includes receiving a network session, implementing a stacked hierarchical agglomerative clustering (HAC) algorithm that operates multiple HAC algorithms to identify respective clusters to which the network session maps, each HAC algorithm of the StackHAC algorithm operates using a different linkage function and distance pair, appending the respective clusters from the multiple HAC algorithms to a feature vector representing the network session resulting in an augmented feature space, and determining, using a classifier or clustering model that operates using the augmented feature space as input, whether each of the network sessions is associated with a network intrusion.
1. A device of a computer network, the device comprising:
at least one memory including instructions stored thereon; and
processing circuitry configured to execute the instructions, the instructions, when executed, cause the processing circuitry to perform operations comprising:
receiving a network session;
completing the network session to alter text to numerical values and fill-in any empty fields of the network session;
extracting features of the network session;
implementing a stacked hierarchical agglomerative clustering (HAC) algorithm that operates multiple HAC algorithms to identify respective clusters to which the network session maps, each HAC algorithm of the StackHAC algorithm operates on the extracted features to cluster the network session using a different linkage function and distance pair, and each HAC algorithm outputting a cluster assignment;
appending each of the cluster assignments from each of the HAC algorithms to a feature vector representing the network session resulting in a combined augmented feature vector; and
determining, using a classifier or clustering model that operates using the combined augmented feature vector as input, whether the network session is associated with a network intrusion.
2. The device of claim 1 , wherein the network intrusion includes a botnet.
3. The device of claim 1 , wherein the linkage functions include two or more of Ward, average, simple, or complete linkage.
4. The device of claim 1 , wherein the distances include two or more of L 1 , L 2 , or cosine.
5. The device of claim 1 , wherein data of the network session includes three or more of an input interface index, an output interface index, timestamps for flow start and finish times, number of bytes and packets observed in a corresponding flow, source and destination internet protocol (IP) address, internet control message protocol (ICMP) type and code, IP protocol, type of service value, source and destination port numbers for transmission control protocol (TCP), unigram data protocol (UDP), or stream control transmission protocol (SCTP), union of all TCP flags observed over a life of the flow, IP address of an immediate next-hop along a route to a destination, or source and destination IP masks.
6. A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations for network intrusion detection, the operations comprising:
receiving a network session;
completing the network session to alter text to numerical values and fill-in any empty fields of the network session;
extracting features of the network session;
implementing a stacked hierarchical agglomerative clustering (HAC) algorithm that operates multiple HAC algorithms to identify respective clusters to which the network session maps, each HAC algorithm of the StackHAC algorithm operates on the extracted features to cluster the network session using a different linkage function and distance pair, and each HAC algorithm outputting a cluster assignment;
appending each of the cluster assignments from each of the HAC algorithms to a feature space representing the reduced and modified feature space of the network sessions resulting in a combined augmented or enhanced feature space; and
determining, using a classifier or clustering model that operates using the combined augmented feature vector as input, whether the network session is associated with a network intrusion.
7. The non-transitory machine-readable medium of claim 6 , wherein the network intrusion includes a botnet.
8. The non-transitory machine-readable medium of claim 6 , wherein the linkage functions include two or more of Ward, average, simple, or complete linkage.
9. The non-transitory machine-readable medium of claim 6 , wherein the distances include two or more of L 1 , L 2 , or cosine.
10. The non-transitory machine-readable medium of claim 6 , wherein data of the network session includes three or more of an input interface index, an output interface index, timestamps for flow start and finish times, number of bytes and packets observed in a corresponding flow, source and destination internet protocol (IP) address, internet control message protocol (ICMP) type and code, IP protocol, type of service value, source and destination port numbers for transmission control protocol (TCP), unigram data protocol (UDP), or stream control transmission protocol (SCTP), union of all TCP flags observed over a life of the flow, IP address of an immediate next-hop along a route to a destination, or source and destination IP masks.
11. A method for network intrusion detection, the method comprising:
receiving a network session;
completing the network session to alter text to numerical values and fill-in any empty fields of the network session;
extracting features of the network session;
implementing a stacked hierarchical agglomerative clustering (HAC) algorithm that operates multiple HAC algorithms to identify respective clusters to which the network session maps, each HAC algorithm of the StackHAC algorithm operates on the extracted features to cluster the network session using a different linkage function and distance pair, and each HAC algorithm outputting a cluster assignment;
appending each of the cluster assignments from each of the HAC algorithms to a feature vector representing the network session resulting in a combined augmented feature vector;
and determining, using a classifier or clustering model that operates using the combined augmented feature vector as input, whether the network session is associated with a network intrusion.
12. The method of claim 11 , wherein the network intrusion includes a botnet.
13. The method of claim 11 , wherein the linkage functions include two or more of Ward, average, simple, or complete linkage.
14. The method of claim 11 , wherein the distances include two or more of L 1 , L 2 , or cosine.
15. The method of claim 11 , wherein data of the network session includes three or more of an input interface index, an output interface index, timestamps for flow start and finish times, number of bytes and packets observed in a corresponding flow, source and destination internet protocol (IP) address, internet control message protocol (ICMP) type and code, IP protocol, type of service value, source and destination port numbers for transmission control protocol (TCP), unigram data protocol (UDP), or stream control transmission protocol (SCTP), union of all TCP flags observed over a life of the flow, IP address of an immediate next-hop along a route to a destination, or source and destination IP masks.