Ransomware detection accuracy based on machine learning analysis of filename extension patterns
Ransomware detection accuracy is improved by analyzing patterns of changes in filename extension counts, relative to each other, that occur in a file system over time. The disclosed approach is malware-agnostic and does not rely on recognizing malware extensions or on real-time monitoring of the target file system. Instead, during each successive backup job of the target file system, the disclosed technology counts different types of filename extensions and compares the counts to each other and to corresponding counts taken in earlier backup jobs. Preferably, the anomaly detection analysis uses machine learning to discern a behavior pattern of the file system, which indicates how filename extensions are distributed and how much they change between backup jobs over time.
1 . A system comprising:
a first computing device comprising one or more hardware processors and computer memory carrying computer programming instructions that, when executed by the one or more hardware processors, configure the first computing device to perform a first backup job of a file system that comprises primary data, wherein the first backup job comprises:
in a scan of the file system, count how many of each distinct filename extension were found;
store a count of each distinct filename extension found in the scan of the file system in a database configured at the first computing device;
cause a copy of the database to be stored at a second computing device;
after one or more backup copies of the primary data are generated:
identify, based on information in the database, a plurality of first counts, wherein each first count corresponds to a distinct filename extension found in the scan of the file system,
identify, based on information in the database, a plurality of second counts, wherein each second count corresponds to a distinct filename extension found in a scan of the file system that was performed in an earlier backup job before the first backup job,
determine a first number of potential renames of filename extensions as compared to the earlier backup job,
use a machine learning model to determine whether the first number of potential renames of filename extensions conforms to a pattern generated by the machine learning model for the file system, wherein the pattern is based on a history of filename extension counts taken in past backup jobs of the file system that precede the first backup job,
based at least in part on determining that the first number does not conform to the pattern, generate an anomaly alert before completing the first backup job.
2 . The system of claim 1 , wherein the first number of potential renames of filename extensions is based on changed counts among the plurality of first counts as compared to the plurality of second counts.
3 . The system of claim 1 , wherein the first number of potential renames of filename extensions is based on a difference between: (i) how many increases in filename extensions were counted in the plurality of first counts compared to the plurality of second counts, and (ii) how many additional files were counted in the plurality of first counts compared to the plurality of second counts.
4 . The system of claim 1 , wherein the first backup job further comprises:
based on determining that the first number does not conform to the pattern, determine whether the first number is greater than a pre-defined threshold percentage of the plurality of first counts, and
based on determining that the first number: does not conform to the pattern and is greater than the pre-defined threshold percentage of the plurality of first counts, generate the anomaly alert before completing the first backup job.
5 . The system of claim 4 , wherein the anomaly alert is further based on a determination that the first number of potential renames of filename extensions is also greater than a threshold number of potential renames.
6 . The system of claim 1 , wherein the first number of potential renames is normalized according to a time gap between the first backup job and the earlier backup job.
7 . The system of claim 1 further comprising the second computing device, which comprises one or more second hardware processors and computer memory carrying computer programming instructions that, when executed by the one or more second hardware processors, configure the second computing device to:
receive the copy of the database from the first computing device;
store the copy of the database at a data storage area configured at the second computing device; and
cause a backup copy of the copy of the database to be generated and stored at a secondary storage that is distinct from the second computing device and from the first computing device.
8 . The system of claim 7 , wherein the first backup job further comprises:
check an integrity of the database before storing filename extension counts in the database; and
based on failing the check of the integrity of the database, cause the backup copy of the copy of the database to be restored from the secondary storage to the first computing device.
9 . The system of claim 1 , wherein the first backup job further comprises:
check an integrity of the database before storing filename extension counts in the database.
10 . The system of claim 9 , wherein the first backup job further comprises:
based on failing the check of the integrity of the database, retrieve the copy of the database from the second computing device.
11 . The system of claim 1 , wherein the first computing device is further configured to:
transmit the anomaly alert to a third computing device, which comprises one or more third hardware processors and computer memory carrying computer programming instructions that, when executed by the one or more third hardware processors, configure the third computing device to:
manage backup jobs in the system, including the first backup job;
based on the anomaly alert, cause a graphical user interface to display a file extension trend of the file system that is based on the history of filename extension counts; and
manage a remedial action responsive to the anomaly alert.
12 . The system of claim 1 , wherein the first computing device is configured to execute a data agent that performs at least some of the first backup job.
13 . The system of claim 1 , wherein the first computing device is configured to execute a data agent and a media agent that collectively perform the first backup job,
wherein the data agent is configured to determine the first number and to use the machine learning model, and
wherein the media agent is configured to store the one or more backup copies of the primary data, in a backup format, at a secondary storage that is distinct from the first computing device.
14 . A system comprising:
a first computing device comprising one or more hardware processors and computer memory carrying computer programming instructions that, when executed by the one or more hardware processors, configure the first computing device to perform a first backup job of a file system that comprises primary data, wherein the first backup job comprises:
count how many of each distinct filename extension are found in a scan of the file system;
store a count of each distinct filename extension found in the scan of the file system in a database configured at the first computing device;
cause a copy of the database to be stored at a second computing device;
after one or more backup copies of the primary data are generated:
identify, based on information in the database, a plurality of first counts, wherein each first count corresponds to a distinct filename extension found in the scan of the file system,
identify, based on information in the database, a plurality of second counts, wherein each second count corresponds to a distinct filename extension found in a scan of the file system that was performed in an earlier backup job before the first backup job,
determine a first number of potential renames of filename extensions as compared to the earlier backup job, wherein the first number is based on a difference between: (i) how many increases in filename extensions were counted in the plurality of first counts compared to the plurality of second counts, and (ii) how many additional files were counted in the plurality of first counts compared to the plurality of second counts,
use a machine learning model to determine that the first number of potential renames of filename extensions does not conform to a pattern generated by the machine learning model for the file system, wherein the pattern is based on a history of filename extension counts taken in past backup jobs of the file system that precede the first backup job, and
generate an anomaly alert before completing the first backup job.
15 . The system of claim 14 , wherein the first backup job further comprises:
based on determining that the first number does not conform to the pattern, determine whether the first number is greater than a pre-defined threshold percentage of the plurality of first counts, and
based on determining that the first number: does not conform to the pattern and is greater than the pre-defined threshold percentage of the plurality of first counts, generate the anomaly alert before completing the first backup job.
16 . The system of claim 14 further comprising the second computing device, which comprises one or more second hardware processors and computer memory carrying computer programming instructions that, when executed by the one or more second hardware processors, configure the second computing device to:
receive the copy of the database from the first computing device;
store the copy of the database at a data storage area configured at the second computing device; and
cause a backup copy of the copy of the database to be generated and stored at a secondary storage that is distinct from the second computing device and from the first computing device.
17 . The system of claim 16 , wherein the first backup job further comprises:
perform an integrity check of the database before storing filename extension counts in the database; and
based on a failure of the integrity check, cause the backup copy of the copy of the database to be restored from the secondary storage to the first computing device.
18 . The system of claim 14 , wherein the first backup job further comprises:
perform an integrity check of the database before storing filename extension counts in the database; and
based on a failure of the integrity check, retrieve the copy of the database from the second computing device.
19 . The system of claim 14 , wherein the first computing device is further configured to:
transmit the anomaly alert to a third computing device, which comprises one or more third hardware processors and computer memory carrying computer programming instructions that, when executed by the one or more third hardware processors, configure the third computing device to:
manage backup jobs in the system, including the first backup job;
based on the anomaly alert, cause a graphical user interface to display a file extension trend of the file system that is based on the history of filename extension counts; and
manage a remedial action responsive to the anomaly alert.
20 . The system of claim 14 , wherein the first computing device is configured to execute a data agent and a media agent that collectively perform the first backup job,
wherein the data agent is configured to determine the first number and to use the machine learning model, and
wherein the media agent is configured to store the one or more backup copies of the primary data, in a backup format, at a secondary storage that is distinct from the first computing device.