Provenance-based threat detection tools and stealthy malware detection

1. A computer implemented provenance-based threat detection method, comprising:

building a provenance graph including a plurality of paths using a processor device from provenance data obtained from one or more computer systems and/or networks, wherein the provenance graph is built by collecting the provenance data using hook functions that intercept operating system calls;

sampling the provenance graph to form a plurality of linear sample paths;

calculating a regularity score for each of the plurality of linear sample paths using a processor device;

selecting a subset of linear sample paths from the plurality of linear sample paths based on the regularity score;

embedding each of the subset of linear sample paths by converting each of the subset of linear sample paths into a numerical vector using a processor device;

detecting anomalies in the embedded paths to identify malicious process activities, wherein the anomalies in the embedded paths are detected using an anomaly detection model that is configured to identify malicious activity, wherein the anomaly detection model is selected from the group consisting of one-class support vector machine (OC-SVM) and Local Outlier Factor (LOF); and

terminating a process related to the embedded path having the identified malicious process activities.

2. The method as recited in claim 1 , wherein selecting a subset of linear sample paths addresses a dependency explosion problem.

3. The method as recited in claim 1 , wherein the anomaly detection model is trained using a benign training data set.

4. The method as recited in claim 3 , wherein embedding each of the plurality of paths is done using graph2vec or doc2vec.

5. A non-transitory computer readable storage medium comprising a computer readable program for a computer implemented provenance-based threat detection tool, wherein the computer readable program when executed on a computer causes the computer to perform the steps of:

building a provenance graph including a plurality of paths using a processor device from provenance data obtained from one or more computer systems and/or networks, wherein the provenance graph is built by collecting the provenance data using hook functions that intercept operating system calls;

sampling the provenance graph to form a plurality of linear sample paths;

calculating a regularity score for each of the plurality of linear sample paths using a processor device;

selecting a subset of linear sample paths from the plurality of linear sample paths based on the regularity score;

embedding each of the subset of linear sample paths by converting each of the subset of linear sample paths into a numerical vector using a processor device;

detecting anomalies in the embedded paths to identify malicious process activities, wherein the anomalies in the embedded paths are detected using an anomaly detection model that is configured to identify malicious activity, wherein the anomaly detection model is selected from the group consisting of one-class support vector machine (OC-SVM) and Local Outlier Factor (LOF); and

terminating a process related to the embedded path having the identified malicious process activities.

6. The method as recited in claim 5 , wherein selecting a subset of linear sample paths addresses a dependency explosion problem.

7. The computer readable program as recited in claim 5 , wherein the anomaly detection model is trained using a benign training data set.

8. The computer readable program as recited in claim 7 , wherein embedding each of the plurality of paths is done using graph2vec or doc2vec.

9. A system for provenance-based threat detection, comprising:

a computer system including:

random access memory configured to store a provenance-based threat detection tool;

one or more processor devices and an operating system having a kernel, wherein one or more hook functions operating in the kernel are configured to collect provenance data using the hook functions that intercept operating system calls; and

a database configured to store the provenance data collected by the one or more hook functions, wherein the provenance-based threat detection tool is configured to:

build a provenance graph including a plurality of paths using the one or more processor devices from provenance data obtained from the computer systems and/or a network;

sample the provenance graph to form a plurality of linear sample paths;

calculate a regularity score for each of the plurality of linear sample paths using the one or more processor devices;

select a subset of linear sample paths from the plurality of linear sample paths based on the regularity score;

embed each of the subset of linear sample paths by converting each of the subset of linear sample paths into a numerical vector using the one or more processor devices;

detect anomalies in the embedded paths to identify malicious process activities, wherein the anomalies in the embedded paths are detected using an anomaly detection model that is configured to identify malicious activity, wherein the anomaly detection model is selected from the group consisting of one-class support vector machine (OC-SVM) and Local Outlier Factor (LOF); and

terminate a process related to the embedded path having the identified malicious process activities.

10. The system as recited in claim 9 , wherein selecting a subset of linear sample paths addresses a dependency explosion problem.

11. The system as recited in claim 9 , wherein the anomaly detection model is trained using a benign training data set.