Hierarchical feature extraction for malware classification in network traffic

1. A method comprising:

receiving packet flow data comprising proxy logs at a computing device comprising a processor configured to process a feature extraction hierarchy comprising a plurality of feature extraction levels and a set of feature extraction functions, each of said feature extraction levels comprising a subset of said feature extraction functions, each of said feature extraction functions used to compute a feature vector;

computing at the processor, a first set of feature vectors for the packet flow data at a first feature extraction level of the feature extraction hierarchy using proxy log attributes;

inputting said first set of feature vectors computed from said first feature extraction level of the feature extraction hierarchy into a second feature extraction level of the feature extraction hierarchy at the processor, to compute a second set of feature vectors using a combination of results from said first feature extraction level; and

transmitting a final feature vector representing a fingerprint of the proxy logs to a classifier for use by the classifier to differentiate malicious traffic from legitimate traffic and identify the malicious traffic;

wherein each of said first and second feature extraction levels of the feature extraction hierarchy comprises said subset of feature extraction functions chosen for said feature extraction level from said set of feature extraction functions and builds feature vectors representing traffic behavior; and

wherein the feature extraction hierarchy allows for combination and aggregation of said feature extraction functions to provide an output of features to represent said traffic behavior and capture contextual information from related proxy log fields.

2. The method of claim 1 wherein the packet flow data comprises logs collected in a specified time window.

3. The method of claim 2 further comprising defining a vector of attributes representing one of the logs.

4. The method of claim 2 further comprising defining a matrix representing all of the logs.

5. The method of claim 1 wherein said final feature vector comprises a single feature vector representing traffic of all users in a specified time window.

6. The method of claim 1 further comprising:

constructing matrices representing traffic for a set of flows in a specified time window;

constructing a first data matrix for said first set of feature vectors using said matrices to represent a first attribute for the set of flows;

constructing a second data matrix for said second set of feature vectors using said first data matrix to represent a second attribute for the set of flows; and

generating said final feature vector based on a last data matrix.

7. The method of claim 1 wherein the classifier is configured to model a probability distribution of malicious and legitimate samples.

8. The method of claim 1 wherein said features comprise statistical features computed from HTTP (Hyper Text Transfer Protocol) traffic to represent traffic behavior.

9. An apparatus comprising:

a linecard configured to receive packet flow data comprising proxy logs:

a processor comprising a feature extraction hierarchy comprising a plurality of levels and a set of feature extraction functions, each of said levels comprising a subset of said feature extraction functions, each of said feature extraction functions used to compute a feature vector, and configured to compute a first set of feature vectors for the packet flow data at a first level of the feature extraction hierarchy using proxy log attributes, input said first set of feature vectors from said first level of the feature extraction hierarchy into a second level of the feature extraction hierarchy to compute a second set of feature vectors using a combination of results from said first level of the feature extraction hierarchy, and input a final feature vector representing a fingerprint of the proxy logs to a classifier to identify malicious traffic; and

memory for storing the packet flow data;

wherein each of said first and second levels of the feature extraction hierarchy comprises said subset of feature extraction functions chosen for said level from said set of feature extraction functions and builds feature vectors representing traffic behavior; and

wherein the feature extraction hierarchy allows for combination and aggregation of said feature extraction functions to provide an output of features to represent said traffic behavior and capture contextual information from related proxy log fields.

10. The apparatus of claim 9 wherein the packet flow data comprises logs collected in a specified time window.

11. The apparatus of claim 10 wherein the processor is further configured to define a vector of attributes representing one of the logs.

12. The apparatus of claim 10 wherein the processor is further configured to define a matrix representing all of the logs.

13. The apparatus of claim 9 wherein said final feature comprises a single feature vector representing traffic of all users in a specified time window.

14. The apparatus of claim 9 wherein the processor is further configured to:

construct matrices representing traffic for a set of flows in a specified time window;

construct a first data matrix for said first set of feature vectors using said matrices to represent a first attribute for the set of flows;

construct a second data matrix for said second set of feature vectors using said first data matrix to represent a second attribute for the set of flows; and

generate said final feature vector based on a last data matrix.

15. The apparatus of claim 9 wherein the classifier is configured to model a probability distribution of malicious and legitimate samples.

16. The apparatus of claim 9 wherein said features comprise statistical features computed from HTTP (Hyper Text Transfer Protocol) traffic to represent traffic behavior.

17. Logic encoded on one or more non-transitory computer readable media for execution and when executed operable to:

process packet flow data comprising proxy logs at a feature extraction hierarchy comprising a plurality of levels and a set of feature extraction functions, each of said levels comprising a subset of said feature extraction functions, each of said feature extraction functions used to compute a feature vector;

compute a first set of feature vectors for the packet flow data at a first level of the feature extraction hierarchy;

input said first set of feature vectors computed from said first level of the feature extraction hierarchy into a second level of the feature extraction hierarchy to compute a second set of feature vectors using a combination of results from said first level; and

transmit a final feature vector representing a fingerprint of the proxy logs to a classifier to identify malicious traffic;

wherein each of said first and second levels of the feature extraction hierarchy comprises said subset of feature extraction functions chosen for said level from said set of feature extraction functions and builds feature vectors representing traffic behavior; and

wherein the feature extraction hierarchy allows for combination and aggregation of said feature extraction functions to provide an output of features to represent said traffic behavior and capture contextual information from related proxy log fields.

18. The logic of claim 17 wherein the packet flow data comprises logs collected in a specified time window and further comprising defining a matrix comprising vectors of attributes for each of the logs.

19. The logic of claim 17 wherein said final feature comprises a single feature vector representing traffic of all users in a specified time window.

20. The logic of claim 17 further operable to:

construct matrices representing traffic for a set of flows in a specified time window;

construct a first data matrix for said first set of feature vectors using said matrices to represent a first attribute for the set of flows;

construct a second data matrix for said second set of features using said first data matrix to represent a second attribute for the set of flows; and

generate said final feature vector based on a last data matrix.