Identifying malware communications with DGA generated domains by discriminative learning

1. A computer-implemented method comprising:

at a first computing device, obtaining a plurality of sample domain names and labeling each of the plurality of sample domain names as a domain generation algorithm (DGA) domain, a non-DGA domain or a suspicious domain;

training a classifier in a first stage based on the plurality of sample domain names without a proxy log;

obtaining a plurality of sample proxy logs including proxy logs of DGA domains and proxy logs of non-DGA domains;

training the classifier in a second stage based on the plurality of sample domain names and the plurality of sample proxy logs;

obtaining a plurality of live traffic proxy logs;

testing the classifier by classifying the plurality of live traffic proxy logs as DGA proxy logs; and

using the trained and tested classifier to identify network communication of a second computing device as malware network communication with DGA domains via a network interface unit of the second computing device.

2. The method of claim 1 , wherein training the classifier in the first stage comprises:

obtaining DGA domains and non-DGA domains from whitelists and blacklists;

obtaining suspicious domains from a domain contacted by an isolated malicious program; and

obtaining unknown DGA domains from output of a Domain Name System anomaly detection process.

3. The method of claim 1 , wherein training the classifier in the second stage comprises:

generating an artificial proxy log of a DGA domain by selecting a proxy log of a DGA domain comprising a first domain name from the plurality of sample proxy logs and replacing the first domain name with a second domain name classified as a DGA domain name; and

adding the artificial proxy log to the plurality of sample proxy logs.

4. The method of claim 3 , further comprising:

storing statistical training data calculated from the plurality of sample proxy logs and the corresponding domain names in a sample database, wherein the statistical data includes features calculated based on the domain names and flow based features; and

training the classifier based on the statistical training data.

5. The method of claim 4 , wherein testing the trained classifier comprises:

extracting statistical features from the live traffic proxy logs;

forming a first input feature vector from the statistical features extracted from the live traffic proxy logs; and

generating a test result by applying the trained classifier to the first input feature vector.

6. The method of claim 4 , wherein each of the plurality of sample proxy logs comprises a domain name in the form of a uniform resource locator (URL); and

wherein storing the statistical training data comprises:

parsing the URL into logical parts; and

calculating statistics for each logical part of the URL.

7. The method of claim 4 , further comprising:

obtaining proxy logs of the malware network communication;

extracting statistical features from the proxy logs of the malware network communication;

forming a second input feature vector from the statistical features extracted from the proxy logs of the malware network communication; and

identifying the network communication as the malware network communication with DGA domains by applying the trained and tested classifier to the second input feature vector.

8. An apparatus comprising:

one or more processors;

one or more memory devices in communication with the one or more processors; and

a network interface unit coupled to the one or more processors,

wherein the one or more processors are configured to:

obtain a plurality of sample domain names and labeling each of the plurality of sample domain names as a domain generation algorithm (DGA) domain, a non-DGA domain or a suspicious domain;

train a classifier in a first stage based on the plurality of sample domain names without a proxy log;

obtain a plurality of sample proxy logs including proxy logs of DGA domains and proxy logs of non-DGA domains;

train the classifier in a second stage based on the plurality of sample domain names and the plurality of sample proxy logs;

obtain a plurality of live traffic proxy logs;

test the classifier by classifying the plurality of live traffic proxy logs as DGA proxy logs; and

use the trained and tested classifier to identify network communication of another computing device as malware network communication with DGA domains via a network interface unit of the other computing device.

9. The apparatus of claim 8 , wherein the one or more processors are configured to train the classifier in the first stage by:

obtaining DGA domains and non-DGA domains from whitelists and blacklists;

obtaining suspicious domains from a domain contacted by an isolated malicious program; and

obtaining unknown DGA domains from output of a Domain Name System anomaly detection process.

10. The apparatus of claim 8 , wherein the one or more processors are configured to train the classifier in the second stage by:

generating an artificial proxy log of a DGA domain by selecting a proxy log of a DGA domain comprising a first domain name from the plurality of sample proxy logs and replacing the first domain name with a second domain name classified as a DGA domain name; and

adding the artificial proxy log to the plurality of sample proxy logs.

11. The apparatus of claim 10 , wherein the one or more processors are configured to:

store statistical training data calculated from the plurality of sample proxy logs and the corresponding domain names in a sample database, wherein the statistical data includes features calculated based on the domain names and flow based features; and

train the classifier based on the statistical training data.

12. The apparatus of claim 10 , wherein the one or more processors are configured to test the trained classifier by:

extracting statistical features from the live traffic proxy logs;

forming a first input feature vector from the statistical features extracted from the live traffic proxy logs; and

generating a test result by applying the trained classifier to the first input feature vector.

13. The apparatus of claim 10 , wherein each of the plurality of sample proxy logs comprises a domain name in the form of a uniform resource locator (URL), and wherein the one or more processors are configured to store the statistical training data by:

parsing the URL into logical parts; and

calculating statistics for each logical part of the URL.

14. The apparatus of claim 10 , wherein the one or more processors are configured to:

obtain proxy logs of the malware network communication;

extract statistical features from the proxy logs of the malware network communication;

form a second input feature vector from the statistical features extracted from the proxy logs of the malware network communication; and

identify the network communication as the malware network communication with DGA domains by applying the trained and tested classifier to the second input feature vector.

15. One or more computer readable non-transitory storage media encoded with software comprising computer executable instructions that when executed by one or more processors, cause the one or more processors to:

obtain a plurality of sample domain names and labeling each of the plurality of sample domain names as a domain generation algorithm (DGA) domain, a non-DGA domain or a suspicious domain;

train a classifier in a first stage based on the plurality of sample domain names without a proxy log;

obtain a plurality of sample proxy logs including proxy logs of DGA domains and proxy logs of non-DGA domains;

train the classifier in a second stage based on the plurality of sample domain names and the plurality of sample proxy logs;

obtain a plurality of live traffic proxy logs;

test the classifier by classifying the plurality of live traffic proxy logs as DGA proxy logs; and

use the trained and tested classifier to identify network communication of another computing device as malware network communication with DGA domains via a network interface unit of the other computing device.

16. The computer readable non-transitory storage media of claim 15 , wherein the executable instructions further cause the one or more processors to train the classifier in the first stage by:

obtaining DGA domains and non-DGA domains from whitelists and blacklists;

obtaining suspicious domains from a domain contacted by an isolated malicious program; and

obtaining unknown DGA domains from output of a Domain Name System anomaly detection process.

17. The apparatus of claim 15 , wherein the executable instructions further cause the one or more processors to train the classifier in the second stage by:

generating an artificial proxy log of a DGA domain by selecting a proxy log of a DGA domain comprising a first domain name from the plurality of sample proxy logs and replacing the first domain name with a second domain name classified as a DGA domain name; and

adding the artificial proxy log to the plurality of sample proxy logs.

18. The computer readable non-transitory storage media of claim 17 , wherein the executable instructions further cause the one or more processors to:

store statistical training data calculated from the plurality of sample proxy logs and the corresponding domain names in a sample database; wherein the statistical data includes features calculated based on the domain names and flow based features; and

train the classifier based on the statistical training data.

19. The computer readable non-transitory storage media of claim 17 , wherein the executable instructions further cause the one or more processors to test the trained classifier by:

extracting statistical features from the live traffic proxy logs;

forming a first input feature vector from the statistical features extracted from the live traffic proxy logs; and

generating a test result by applying the trained classifier to the first input feature vector.

20. The computer readable non-transitory storage media of claim 17 , wherein each of the plurality of sample proxy logs comprises a domain name in the form of a uniform resource locator (URL), and wherein the executable instructions further cause the one or more processors to:

store the statistical training data by parsing the URL into logical parts and calculating statistics for each logical part of the URL

obtain proxy logs of the malware network communication;

extract statistical features from the proxy logs of the malware network communication;

form a second input feature vector from the statistical features extracted from the proxy logs of the malware network communication; and

identify the network communication as the malware network communication with DGA domains by applying the trained and tested classifier to the second input feature vector.