Machine learning powered cloud sandbox for malware detection
A cloud-based network security system (NSS) is described. The NSS uses a sandbox to safely detonate and extract information about a document and uses machine learning algorithms to analyze the information to predict whether the document contains malicious software. Specifically, during the detonation, static and dynamic information about the document is captured in the sandbox as well as character strings from images in the document. The dynamic information (and sometimes the static information) is input to an AI or machine learning model trained to provide an output indicating a prediction of whether the document contains malware. The character strings are compared with a batch of phishing keywords to generate a heuristic score. A validation engine combines the output from the AI or machine learning model and the heuristic score to classify the document as malicious or clean. Security policies can then be applied based on the classification.
1. A computer-implemented method, comprising:
intercepting, by a cloud-based network security system, a request to access a document;
obtaining, by the cloud-based network security system, the document;
detonating, by the cloud-based network security system, the document in a sandbox of the cloud-based network security system;
in response to the detonating, extracting, by the cloud-based network security system, dynamic information about the document;
extracting, by the cloud-based network security system, character strings from images in the document during the detonating in the sandbox;
providing, by the cloud-based network security system, the dynamic information as input to an artificial intelligence model trained to provide an output indicating a prediction of whether the document contains malware based on the input;
generating, by the cloud-based network security system, a heuristic score based on comparing the character strings extracted from the document to a batch of phishing keywords;
providing, by the cloud-based network security system, the output of the artificial intelligence model and the heuristic score as input to a verdict engine, wherein the verdict engine combines the output of the artificial intelligence model and the heuristic score to classify the document as one of malicious or clean; and
implementing, by the cloud-based network security system, a security policy based at least in part on the classification of the document.
2. The computer-implemented method of claim 1 , wherein the extracting the dynamic information about the document comprises:
analyzing behavior of the document during detonation in the sandbox; and
extracting data from the behavior, wherein the dynamic information about the document comprises:
a set of behavior features of the document exhibited during the detonating,
a size of a process tree spawned by the detonating, and
a set of signature features exhibited during the detonating.
3. The computer-implemented method of claim 2 , wherein the set of behavior features comprises at least one of: visited files, visited paths, and pathways explored by processes in the sandbox.
4. The computer-implemented method of claim 2 , wherein the set of signature features comprises:
a signature vector having a dimension for each of a plurality of software signatures, wherein a value of each dimension indicates whether the document invoked the respective software signature in the sandbox, and
severity scores for each of the software signatures the document invoked.
5. The computer-implemented method of claim 2 , wherein the providing the dynamic information as input comprises:
generating a feature vector representing at least a portion of the dynamic information; and
providing the feature vector as the input to the artificial intelligence model.
6. The computer-implemented method of claim 1 , wherein the artificial intelligence model comprises a gradient boosting tree algorithm.
7. The computer-implemented method of claim 1 , wherein the extracting the character strings from the images in the document comprises analyzing, by the cloud-based network security system in the sandbox, the document with optical character recognition.
8. The computer-implemented method of claim 1 , wherein the heuristic score comprises a count of all matches identified during the comparison of the character strings to the batch of the phishing keywords.
9. The computer-implemented method of claim 1 , wherein:
a heuristic rule is triggered based on the heuristic score exceeding a threshold heuristic value;
the output of the artificial intelligence model comprises a prediction score indicating the prediction; and
the verdict engine:
classifies document as malicious based on:
determining the heuristic rule is triggered; and
the prediction score exceeds a pre-defined prediction threshold value, and
classifies the document as clean based on:
determining the heuristic rule is not triggered; or
the prediction score does not exceed the pre-defined prediction threshold value.
10. The computer-implemented method of claim 1 , further comprising:
extracting static information about the document, wherein the providing the dynamic information as input to the artificial intelligence model comprises providing both the dynamic information and the static information as the input to the artificial intelligence model.
11. A cloud-based network security system, comprising:
one or more processors; and
one or more memories having stored thereon instructions that, upon execution by the one or more processors, cause the one or more processors to:
intercept a request to access a document;
obtain the document;
detonate the document in a sandbox of the cloud-based network security system;
in response to the detonating, extract dynamic information about the document;
extract character strings from images in the document during the detonating in the sandbox;
provide the dynamic information as input to an artificial intelligence model trained to provide an output indicating a prediction of whether the document contains malware based on the input;
generate a heuristic score based on comparing the character strings extracted from the document to a batch of phishing keywords;
provide the output of the artificial intelligence model and the heuristic score as input to a verdict engine, wherein the verdict engine combines the output of the artificial intelligence model and the heuristic score to classify the document as one of malicious or clean; and
implement a security policy based on the classification of the document.
12. The cloud-based network security system of claim 11 , wherein the instructions to extract the dynamic information about the document comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
analyze behavior of the document during detonation in the sandbox; and
extract data from the behavior, wherein the dynamic information about the document comprises:
a set of behavior features of the document exhibited during the detonating,
a size of a process tree spawned by the detonating, and
a set of signature features of the document exhibited during the detonating.
13. The cloud-based network security system of claim 12 , wherein the set of behavior features comprises at least one of: visited files, visited paths, and pathways explored by processes in the sandbox.
14. The cloud-based network security system of claim 12 , wherein the set of signature features comprises:
a signature vector having a dimension for each of a plurality of software signatures, wherein a value of each dimension indicates whether the document invoked the respective software signature in the sandbox, and
severity scores for each of the software signatures the document invoked.
15. The cloud-based network security system of claim 12 , wherein the instructions to provide the dynamic information as input comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
generate a feature vector representing the dynamic information; and
provide the feature vector as the input to the artificial intelligence model.
16. The cloud-based network security system of claim 11 , wherein the artificial intelligence model comprises a gradient boosting tree algorithm.
17. The cloud-based network security system of claim 11 , wherein the instructions to extract the character strings from the images in the document comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
analyze the document during the detonation in the sandbox with optical character recognition.
18. The cloud-based network security system of claim 11 , wherein the heuristic score comprises a count of all matches identified during the comparison of the character strings to the batch of phishing keywords.
19. The cloud-based network security system of claim 11 , wherein:
a heuristic rule is triggered based on the heuristic score exceeding a threshold heuristic value;
the output of the artificial intelligence model comprises a prediction score indicating the prediction; and
the verdict engine comprises instructions that, upon execution by the one or more processors, cause the one or more processors to:
classify document as malicious based on:
determining the heuristic rule is triggered; and
the prediction score exceeds a pre-defined prediction threshold value, and classify the document as clean based on:
determining the heuristic rule is not triggered; or
the prediction score does not exceed the pre-defined prediction threshold value.
20. The cloud-based network security system of claim 11 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
extract static data about the document; and
provide both the static information and the dynamic information as the input to the artificial intelligence model.