Malicious software detection in a computing system
A computer system identifies malicious Uniform Resource Locator (URL) data items from a plurality of unscreened data items that have not been previously identified as associated with malicious URLs. The system can execute a number of pre-filters to identify a subset of URLs in the plurality of data items that are likely to be malicious. A scoring processor can score the subset of URLs based on a plurality of input vectors using a suitable machine learning model. Optionally, the system can execute one or more post-filters on the score data to identify data items of interest. Such data items can be fed back into the system to improve machine learning or can be used to provide a notification that a particular resource within a local network is infected with malicious software.
1. A computer system for detecting malicious software, the computer system comprising:
one or more computer-readable storage devices including computer executable instructions; and
one or more processors configured to execute the computer executable instructions to cause the computer system to:
access connection records that include respective locational references to computerized resources external to a local network which computerized devices within the local network have accessed or attempted to access;
access a set of recently accessed domain names, wherein the set of recently accessed domain names are determined based on a set of locational references in a set of communications involving the local network from a recent period of time, wherein domain names in communications outside of the recent period of time are not included in the set of recently accessed domain names; and
perform one or more filtering operations on the connection records, wherein the one or more filtering operations include:
parsing the respective locational references associated with the connection records to identify domain names; and
for each of the identified domain names, if the identified domain name is not included in the set of recently accessed domain names, including any connection records associated with the identified domain name in a first subset of connection records, wherein domain names not included in the set of recently accessed domain names are identified as more likely malicious or unwanted than domain names included in the set of recently accessed domain names.
2. The computer system of claim 1 , wherein the one or more processors are configured to execute the computer executable instructions to further cause the computer system to:
score at least a portion of the first subset of connection records using a machine learning model incorporating a factor relating to the identified domain names; and
perform one or more additional filtering operations on the scored portion of the first subset of connection records to identify a second subset of the first subset of connection records.
3. The computer system of claim 2 , wherein:
the one or more additional filtering operations identify, within the scored portion of the first subset of connection records, the second subset of the first subset of connection records associated with one or more locational references, and the second subset of the first subset of connection records are more likely to be malicious than identified domain names associated with connection records in the first subset of connection records that are not included in the second subset of the first subset of connection records.
4. The computer system of claim 2 , wherein the machine learning model incorporates a plurality of factors based on at least one of the one or more filtering operations.
5. The computer system of claim 2 , wherein the machine learning model comprises at least one of: a Support Vector Machine model, a Neural Network model, a Decision Tree model, a Naïve Bayes model, or a Logistic Regression model.
6. The computer system of claim 1 , wherein the one or more filtering operations include:
a filtering operation based on a determination that an identified domain name does not satisfy a threshold position in a list of domain names satisfying a ranking condition based on Internet traffic data, identifying connection records associated with the identified domain name to be in the first subset of connection records.
7. The computer system of claim 1 , wherein the one or more filtering operations include:
identifying, within the connection records, a second subset of connection records associated with a first locational reference;
determining first n-gram distribution data indicating a frequency of each n-length character combination that occurs as an n-length window moves across at least parts of one or more filepaths associated with the second subset of connection records, the n-lengths being two or more characters;
accessing second n-gram distribution data representing an expected n-gram distribution associated with likely non-malicious locational references;
comparing the first n-gram distribution data with the second n-gram distribution data to determine a variance; and
at least in part in response to the variance exceeding a threshold, determining that the first locational reference is likely to compromise security.
8. The computer system of claim 1 , wherein the one or more processors are configured to execute the computer executable instructions to further cause the computer system to:
transmit an indicator for display, the indicator indicating the identified domain names are likely to compromise security.
9. The computer system of claim 1 , wherein the one or more filtering operations include a filtering operation based on a determination that the domain names are not included in a plurality of dictionary words, identifying connection records to be in the first subset of the connection records.
10. The computer system of claim 1 , wherein the one or more filtering operations include a filtering operation comprising:
parsing the respective locational references associated with the connection records to identify filepaths; and
for each of the identified filepaths, based on a determination that the identified filepaths is in a plurality of filepaths associated with a set of locational references in a set of communications involving the local network from a period of time, identifying connection records associated with the identified filepaths to be in the first subset of connection records.
11. The computer system of claim 1 , wherein the one or more filtering operations include a filtering operation comprising:
accessing a list of words associated with malicious locational references;
transmitting, to an Internet search engine providing an autocomplete function that automatically displays words to complete a query entered into the Internet search engine, a first query comprising a first domain name of the identified domain names;
receiving, from the Internet search engine, the words displayed in response to the first query; and
based on a determination that at least one of the words is in a list of words associated with malicious locational references, identifying connection records associated with the first domain name to be in the first subset of connection records.
12. The computer system of claim 1 , wherein the one or more filtering operations include a filtering operation based on registration dates of the identified domain names.
13. A computer-implemented method comprising:
by one or more processors configured to execute computer executable instructions:
accessing connection records that include respective locational references to computerized resources external to a local network which computerized devices within the local network have accessed or attempted to access;
accessing a set of recently accessed domain names, wherein the set of recently accessed domain names are determined based on a set of locational references in a set of communications involving the local network from a recent period of time, wherein domain names in communications outside of the recent period of time are not included in the set of recently accessed domain names; and
performing one or more filtering operations on the connection records, wherein the one or more filtering operations include:
parsing the respective locational references associated with the connection records to identify domain names; and
for each of the identified domain names, if the identified domain name is not included in the set of recently accessed domain names, including any connection records associated with the identified domain name in a first subset of connection records,
wherein domain names not included in the set of recently accessed domain names are identified as more likely malicious or unwanted than domain names included in the set of recently accessed domain names.
14. The computer-implemented method of claim 13 further comprising:
by the one or more processors configured to execute computer executable instructions:
scoring at least a portion of the first subset of connection records using a machine learning model incorporating a factor relating to the identified domain names; and
performing one or more additional filtering operations on the scored portion of the first subset of connection records to identify a second subset of the first subset of connection records.
15. The computer-implemented method of claim 14 , wherein:
the one or more additional filtering operations identify, within the scored portion of the first subset of connection records, the second subset of the first subset of connection records associated with one or more locational references, and the second subset of the first subset of connection records are more likely to be malicious than identified domain names associated with connection records in the first subset of connection records that are not included in the second subset of the first subset of connection records.
16. The computer-implemented method of claim 14 , wherein the machine learning model incorporates a plurality of factors based on at least one of the one or more filtering operations.
17. The computer-implemented method of claim 14 , wherein the machine learning model comprises at least one of: a Support Vector Machine model, a Neural Network model, a Decision Tree model, a Naïve Bayes model, or a Logistic Regression model.
18. The computer-implemented method of claim 13 , wherein the one or more filtering operations include:
a filtering operation based on a determination that an identified domain name does not satisfy a threshold position in a list of domain names satisfying a ranking condition based on Internet traffic data, identifying connection records associated with the identified domain name to be in the first subset of connection records.
19. The computer-implemented method of claim 13 further comprising:
by the one or more processors configured to execute computer executable instructions:
transmitting an indicator for display, the indicator indicating the identified domain names are likely to compromise security.
20. The computer-implemented method of claim 13 , wherein the one or more filtering operations include a filtering operation comprising:
parsing the respective locational references associated with the connection records to identify filepaths; and
for each of the identified filepaths, based on a determination that the identified filepaths is in a plurality of filepaths associated with a set of locational references in a set of communications involving the local network from a period of time, identifying connection records associated with the identified filepaths to be in the first subset of connection records.