Classifying locator generation kits
A method and system for classifying malicious locators where a processor is trained on a set of known malicious locators using a non-supervised learning procedure. Once trained, the processor may classify new locators as being generated by a particular generation kit.
1. A method for training a classifier for classifying malicious locators accessible through a network, the method comprising:
receiving a plurality of training malicious locators that each comprise the location of a known malicious network-accessible resource;
extracting, from each respective training malicious locator of the plurality of training malicious locators, a respective set of features;
performing, with respect to a first feature set comprising a combination of the respective set of features from the plurality of training malicious locators, feature set dimensionality reduction to reduce the dimensionality of the first feature set to a second feature set with fewer dimensions than the first feature set;
executing, using the second feature set a clustering procedure to identify latent class relationships among the plurality of training malicious locators, wherein, based on the latent class relationships, families of malicious locators, that respectively comprise a set of one or more malicious locators with similar characteristics, are identified;
assigning labels to the plurality of training malicious locators, wherein the assigning comprises labeling each respective training malicious locator as belonging to a particular family of the families of malicious locators, and wherein determination of the particular family for labeling a respective training malicious locator is based at least in part on respective features extracted from the respective training malicious locator that are in the second feature set; and
training a classification module to classify other locators, wherein the training comprises using the second feature set, the labels, and a machine learning procedure to train the classification module to classify a locator as belonging to a family of the families of malicious locators.
2. The method of claim 1 , wherein at least one locator of the plurality of training malicious locators is a uniform resource locator (URL).
3. The method of claim 1 , further comprising assigning a membership probability to each of the plurality of training malicious locators, the membership probability representing a probability a locator was generated by a specific source.
4. The method of claim 3 , wherein the label assigned to each of the plurality of training malicious locators is further based at least in part on a highest membership probability for each of the plurality of training malicious locators.
5. The method of claim 1 , wherein the extracted respective set of features extracted from a respective training malicious locator includes one or more of: locator string length, character frequency distribution, domain levels, number of directories, number of words, number of words from a predetermined list of words, number of vowels, or number of consonants in the respective training malicious locator.
6. The method of claim 1 , wherein:
the executing the clustering procedure comprises executing a supervised learning procedure; and
the machine learning procedure used to train the classification module comprises a supervised learning procedure.
7. The method of claim 1 , further comprising:
receiving, after the training, a particular locator;
determining a classification for the particular locator, wherein the determining the classification comprises:
classifying, using the classification module, the particular locator as belonging to a particular family of the families of malicious locators; and
presenting, via a user interface, the classification in the form of a notification.
8. The method of claim 7 , wherein the notification comprises at least one of:
a video-based notification;
an audio-based notification; or
a haptic-based notification.
9. A system for training a classifier for classifying malicious locators accessible through a network, the system comprising:
an interface to access a plurality of training malicious locators that each comprise the location of a known malicious network-accessible resource;
a network interface; and
a processor in communication with the medium interface and the network interface, the processor configured to:
extract, from each respective training malicious locator of the plurality of training malicious locators, a respective set of features;
perform, with respect to a first feature set comprising a combination of the respective set of features from the plurality of training malicious locators, feature set dimensionality reduction to reduce the dimensionality of the first feature set to a second feature set with fewer dimensions than the first feature set;
execute, using the second feature set, a clustering procedure to identify latent class relationships among the plurality of training malicious locators, wherein, based on the latent class relationships, families of malicious locators, that respectively comprise a set of one or more malicious locators with similar characteristics, are identified;
assign labels to the plurality of training malicious locators, wherein, to assign the labels, the processor is configured to label each respective training malicious locator as belonging to a particular family of the families of malicious locators, and wherein determination of the particular family for labeling a respective training malicious locator is based at least in part on respective features extracted from the respective training malicious locator that are in the second feature set; and
train a classification module to classify other locators, wherein the processor is configured to use the second feature set, the labels, and a machine learning procedure to train the classification module to classify a locator as belonging to a family of the families of malicious locators.
10. The system of claim 9 , wherein at least one locator of the plurality of training malicious locators is a uniform resource locator (URL).
11. The system of claim 9 , wherein the processor is configured to assign a membership probability to each of the plurality of training malicious locators, the membership probability representing a probability a locator was generated by a specific source.
12. The system of claim 11 , wherein the label assigned to each of the plurality of training malicious locators is further based at least in part on a highest membership probability for each of the plurality of training malicious locators.
13. The system of claim 9 , wherein the respective set of features extracted from a respective training malicious locator includes one or more of: locator string length, character frequency distribution, domain levels, number of directories, number of words, number of words from a predetermined list of words, number of vowels, or number of consonants in the respective training, malicious locator.
14. The system of claim 9 , wherein:
to execute the clustering procedure, the processor is configured to execute a supervised learning procedure; and
the machine learning procedure used to train the classification module comprises a supervised learning procedure.
15. The system of claim 9 , wherein the processor is further configured to:
receive, after the classification module is trained, a particular locator;
determine a classification for the particular locator, wherein, to determine the classification, the processor is configured to:
classify, using the classification module, the particular locator as belonging to a particular family of the families of malicious locators; and
present, via a user interface, the classification in the form of a notification.
16. The system of claim 15 , wherein the notification comprises at least one of:
a video-based notification;
an audio-based notification; or
a haptic-based notification.
17. One or more non-transitory computer-accessible storage media storing program instructions that, when executed on or across one or more processors, cause the one or more processors to:
receive a plurality of training malicious locators that each comprise the location of a known malicious network-accessible resource;
extract, from each respective training malicious locator of the plurality of training malicious locators, a respective set of features;
perform, with respect to a first feature set comprising a combination of the respective set of features from the plurality of training malicious locators, feature set dimensionality reduction to reduce the dimensionality of the first feature set to a second feature set with fewer dimensions than the first feature set;
execute, using the second feature set, a clustering procedure to identify latent class relationships among the plurality of training malicious locators, wherein, based on the latent class relationships, families of malicious locators, that respectively comprise a set of one or more malicious locators with similar characteristics, are identified;
assign labels to the plurality of training malicious locators, wherein, to assign the labels, the one or more processors are configured to label each respective training malicious locator as belonging to a particular family of the families of malicious locators, and wherein determination of the particular family for labeling a respective training malicious locator is based at least in part on respective features extracted from the respective training malicious locator that are in the second feature set; and
train a classification module to classify other locators, wherein the one or more processors are configured to use the second feature set, the labels, and a machine learning procedure to train the classification module to classify a locator as belonging to a family of the families of malicious locators.
18. The one or more non-transitory computer-accessible storage media of claim 17 , wherein:
to execute the clustering procedure, the one or more processors are configured to execute a supervised learning procedure; and
the machine learning procedure used to train the classification module comprises a supervised learning procedure.
19. The one or more non-transitory computer-accessible storage media of claim 17 , wherein the one or more processors are further configured to:
receive, after the classification module is trained, a particular locator;
determine a classification for the particular locator, wherein, to determine the classification, the one or more processors are configured to:
classify, using the classification module, the particular locator as belonging to a particular family of the families of malicious locators; and
present, via a user interface, the classification in the form of a notification.
20. The one or more non-transitory computer-accessible storage media of claim 19 , wherein the notification comprises at least one of:
a video-based notification;
an audio-based notification; or
a haptic-based notification.