System and method for malware detection learning
Malware detection techniques that detect malware by identifying the C&C communication between the malware and the remote host, and distinguish between communication transactions that carry C&C communication and transactions of innocent traffic. The system distinguishes between malware transactions and innocent transactions using malware identification models, which it adapts using machine learning algorithms. However, the number and variety of malicious transactions that can be obtained from the protected network are often too limited for effectively training the machine learning algorithms. Therefore, the system obtains additional malicious transactions from another computer network that is known to be relatively rich in malicious activity. The system is thus able to adapt the malware identification models based on a large number of positive examples—The malicious transactions obtained from both the protected network and the infected network. As a result, the malware identification models are adapted with high speed and accuracy.
1. A method, comprising:
monitoring, by a network interface, a protected computer network that is protected from malicious software by a processor, the protected computer network comprising one or more infected computers, and an infected computer network that is known to be infected with malicious software and is not protected from malicious software by the processor, wherein the protected computer network and the infected computer network are different networks;
extracting, by the processor, from both the protected computer network and the infected computer network, first communication transactions that are known to be malicious;
extracting, by the processor, only from the protected computer network, second communication transactions that are not known to be malicious;
identifying, by the processor, one or more malicious communication transactions exchanged with the protected computer network, by processing the first and second communication transactions;
wherein identifying the malicious communication transactions comprises creating, by the processor, based on the extracted first and second communication transactions, one or more models that distinguish the malicious communication transactions from innocent communication transactions, and detecting the malicious communication transactions using the models.
2. The method according to claim 1 , wherein extracting the first communication transactions comprises selecting the first communication transactions depending on reputation levels of respective hosts participating in the communication transactions.
3. The method according to claim 1 , wherein extracting the first communication transactions comprises including a transaction in the first communication transactions if a host participating in the transaction appears on a blacklist.
4. The method according to claim 1 , wherein identifying the malicious communication transactions comprises generating alerts based on the models, receiving an analysis of the alerts from an operator, and detecting the malicious communication transactions based on the analysis of the alerts by the operator.
5. The method according to claim 4 , wherein identifying the malicious communication transactions comprises adapting the models based on the analysis of the alerts by the operator.
6. The method according to claim 5 , wherein extracting the first communication transactions comprises selecting the first communication transactions depending on reputation levels of respective hosts participating in the communication transactions, and comprising updating at least one of the reputation levels based on the analysis of the alerts by the operator.
7. The method according to claim 6 , and further comprising updating selection of the first communication transactions based on the updated reputation levels, and updating identification of the malicious communication transactions by processing the updated first communication transactions.
8. The method according to claim 1 , and further comprising extracting from the infected computer network one or more third communication transactions that are not known to be malicious, wherein identifying the malicious communication transactions comprises jointly processing the second and third communication transactions.
9. A system, comprising:
a network interface, which is configured to monitor a protected computer network that is to be protected from malicious software by a processor, the protected computer network comprising one or more infected computers, and an infected computer network that is known to be infected with malicious software and is not to be protected from malicious software by the processor, wherein the protected computer network and the infected computer network are different networks; and
a hardware computer system comprising the processor, which is configured to extract, from both the protected computer network and the infected computer network, first communication transactions that are known to be malicious, to extract, only from the protected computer network, second communication transactions that are not known to be malicious, and to identify one or more malicious communication transactions exchanged with the protected computer network, by processing the first and second communication transactions;
wherein the processor is configured to create, based on the extracted first and second communication transactions, one or more models that distinguish the malicious communication transactions from innocent communication transactions, and to detect the malicious communication transactions using the models.
10. The system according to claim 9 , wherein the processor is configured to select the first communication transactions depending on reputation levels of respective hosts participating in the communication transactions.
11. The system according to claim 9 , wherein the processor is configured to include a transaction in the first communication transactions if a host participating in the transaction appears on a blacklist.
12. The system according to claim 9 , wherein the processor is configured to generate alerts based on the models, to receive an analysis of the alerts from an operator, and to identify the malicious communication transactions based on the analysis of the alerts by the operator.
13. The system according to claim 12 , wherein the processor is configured to adapt the malware detection models based on the analysis of the alerts by the operator.
14. The system according to claim 12 , wherein the processor is configured to select the first communication transactions depending on reputation levels of respective hosts participating in the communication transactions, and to update at least one of the reputation levels based on the analysis of the alerts by the operator.
15. The system according to claim 14 , wherein the processor is configured to update selection of the first communication transactions based on the updated reputation levels, and to update identification of the malicious communication transactions by processing the updated first communication transactions.
16. The system according to claim 9 , wherein the processor is configured to extract from the infected computer network one or more third communication transactions that are not known to be malicious, and to identify the malicious communication transactions by jointly processing the second and third communication transactions.
17. A non-transitory computer-readable medium having stored thereon a sequence of instructions that when executed by a computing system causes, the computing system to perform the steps comprising:
monitoring a protected computer network that is to be protected from malicious software by the computing system, the protected computer network comprising one or more infected computers, and an infected computer network that is known to be infected with malicious software and is not to be protected from malicious software by the computing system, wherein the protected computer network and the infected computer network are different networks;
extracting from both the protected computer network and the infected computer network, first communication transactions that are known to be malicious;
extracting only from the protected computer network, second communication transactions that are not known to be malicious; and
identifying one or more malicious communication transactions exchanged with the protected computer network, by processing the first and second communication transactions;
wherein identifying the malicious communication transactions comprises creating, by the processor, based on the extracted first and second communication transactions, one or more models that distinguish the malicious communication transactions from innocent communication transactions, and detecting the malicious communication transactions using the models.
18. The non-transitory computer-readable medium according to claim 17 , wherein the step of identifying the malicious communication transactions comprises generating alerts based on the models, receiving an analysis of the alerts from an operator, and detecting the malicious communication transactions based on the analysis of the alerts by the operator.
19. The non-transitory computer-readable medium according to claim 18 , wherein the step of extracting the first communication transactions comprises selecting the first communication transactions depending on reputation levels of respective hosts participating in the communication transactions, and comprising updating at least one of the reputation levels based on the analysis of the alerts by the operator.
20. The non-transitory computer-readable medium according to claim 19 , having further instructions stored thereon that when executed by the computing system, cause the computing system to perform the additionally steps comprising updating selection of the first communication transactions based on the updated reputation levels, and updating identification of the malicious communication transactions by processing the updated first communication transactions.