Machine learning to determine command and control sites
Systems and methods include receiving a domain for a determination of a likelihood the domain is a command and control site; analyzing the domain with an ensemble of a plurality of trained machine learning models including a Uniform Resource Locator (URL) model that analyzes lexical features of a hostname of the domain and an artifact model that analyzes content features of a webpage associated with the domain; and combining results of the ensemble to predict the likelihood the domain is a command and control site.
1 . A method comprising the steps of:
receiving a domain for a determination of a likelihood the domain is a command and control site;
analyzing the domain with an ensemble of a plurality of trained machine learning models including a Uniform Resource Locator (URL) model that analyzes lexical features of a hostname of the domain and an artifact model that analyzes content features of a webpage associated with the domain;
combining, by a command-and-control (C2) model of the ensemble, a URL score generated by the URL model and an artifact score generated by the artifact model of the ensemble, the C2 model being a trained gradient-boosted decision tree model configured to receive the URL score and the artifact score as inputs and to output a final score to predict the likelihood the domain is a command and control site; and
performing, by an enforcement node, an action responsive to the likelihood the domain is the command and control site, wherein the action is one or more of adding the domain to blocked list and causing a block of the domain.
2 . The method of claim 1 , wherein the steps include
performing the receiving responsive to a determination by a domain reputation process of a likelihood the domain is malicious.
3 . The method of claim 1 , wherein the steps include
prior to the analyzing, training the URL model and the artifact model.
4 . The method of claim 3 , wherein the training includes
using labeled log data from a cloud-based system that performs monitoring of a plurality of users.
5 . The method of claim 4 , wherein the labeled log data is based on a content classification process.
6 . The method of claim 1 , wherein the ensemble further includes transaction patterns to the domain.
7 . The method of claim 1 , wherein the ensemble further includes an analysis of a reputation of the domain.
8 . The method of claim 1 , wherein the ensemble further includes a malware relation of the domain.
9 . A non-transitory computer-readable medium comprising instructions that, when executed, cause one or more processors to perform steps of:
receiving a domain for a determination of a likelihood the domain is a command and control site;
analyzing the domain with an ensemble of a plurality of trained machine learning models including a Uniform Resource Locator (URL) model that analyzes lexical features of a hostname of the domain and an artifact model that analyzes content features of a webpage associated with the domain;
combining, by a command-and-control (C2) model of the ensemble, a URL score generated by the URL model and an artifact score generated by the artifact model of the ensemble, the C2 model being a trained gradient-boosted decision tree model configured to receive the URL score and the artifact score as inputs and to output a final score to predict the likelihood the domain is a command and control site; and
perform, by a node, an action responsive to the likelihood the domain is the command and control site, wherein the action is one or more adding the domain to a blocked list and causing a block of the domain.
10 . The non-transitory computer-readable medium of claim 9 , wherein the steps include
performing the receiving responsive to a determination by a domain reputation process of a likelihood the domain is malicious.
11 . The non-transitory computer-readable medium of claim 9 , wherein the steps include
prior to the analyzing, training the URL model and the artifact model.
12 . The non-transitory computer-readable medium of claim 11 , wherein the training includes
using labeled log data from a cloud-based system that performs monitoring of a plurality of users.
13 . The non-transitory computer-readable medium of claim 12 , wherein the labeled log data is based on a content classification process.
14 . The non-transitory computer-readable medium of claim 9 , wherein the ensemble further includes transaction patterns to the domain.
15 . The non-transitory computer-readable medium of claim 9 , wherein the ensemble further includes an analysis of a reputation of the domain.
16 . The non-transitory computer-readable medium of claim 9 , wherein the ensemble further includes a malware relation of the domain.
17 . A cloud-based system comprising a plurality of interconnected nodes, each node comprises:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the node to:
receive a domain for a determination of a likelihood the domain is a command and control site;
analyze the domain with an ensemble of a plurality of trained machine learning models including a Uniform Resource Locator (URL) model that analyzes lexical features of a hostname of the domain and an artifact model that analyzes content features of a webpage associated with the domain;
combine, by a command-and-control (C2) model of the ensemble, a URL score generated by the URL model and an artifact score generated by the artifact model of the ensemble, the C2 model being a trained gradient-boosted decision tree model configured to receive the URL score and the artifact score as inputs and to output a final score to predict the likelihood the domain is a command and control site; and
perform an action responsive to the likelihood the domain is the command and control site, wherein the action is one or more of adding the domain to a blocked list and causing a block of the domain.
18 . The method of claim 1 , wherein the combining includes aggregating predictions of the plurality of trained machine learning models over a predetermined time period to increase a confidence level of the final score.
19 . The method of claim 1 , wherein performing the action includes automatically updating a blocklist database stored at the node at an hourly or daily frequency with domains predicted as command and control sites based on the final score.
20 . The method of claim 1 , wherein analyzing the domain with the ensemble of the plurality of trained machine learning models further includes aggregating transaction data by company identifier, user identifier, hostname, request, response, and user agent over a predefined time for generating input features for the plurality of trained machine learning models.