Anomalous file detection
Systems and methods are provided for detecting an anomalous file in a dataset of files utilizing a set of computer resources executing instructions to establish computer processes. The computer processes include, for each file in the dataset, evaluating the file to produce a file profile, using the file profile to develop a score for the file as to its specific type, and processing the score in a machine learning model trained on anomalous file data to generate a probability that the file has an anomaly for the specific type.
1 . A computer-implemented method of detecting in a dataset of files an anomalous file that includes content that has been corrupted by a ransomware attack, the method utilizing a set of computer resources executing instructions to establish computer processes, wherein the computer processes comprise, for each given file in the dataset:
evaluating the given file to produce a file profile;
using the file profile to develop a score for the given file as a specific type by comparing the file profile against a benchmark for the specific type, wherein (i) the benchmark is a set of metrics, associated with the specific type, that uniquely identifies the given file as being of the specific type, (ii) particular constituents of the set of metrics used for determining the benchmark are selected based on a context in which the dataset of files is being used or has been generated, and (iii) the benchmark has been developed based on data for a selected collection of files of the specific file type;
processing the score in a machine learning model trained on post-ransomware attack anomalous file data to generate a probability that the given file has an anomaly for the specific type; and
determining based on the probability whether the given file includes content that has been corrupted by a ransomware attack.
2 . The computer-implemented method of claim 1 , wherein the benchmark is selected from the group consisting of an average entropy of a file of the specific type, a most common symbol in the file of the specific type, average frequency of occurrence of the most common symbol in the file of the specific type, an entropy variance, and combinations of the foregoing.
3 . The computer-implemented method of claim 2 , wherein evaluating the given file comprises scanning and using metadata of the given file to generate at least one metric to compare against the benchmark.
4 . The computer-implemented method of claim 3 , wherein using the file profile to develop the score comprises producing a collection of values from comparing the file profile and the benchmark to define the score.
5 . The computer-implemented method of claim 1 , wherein the score is selected from the group consisting of a percentage difference between a size of the given file and an average size of a benchmark, a percentile ranking between the size of the given file and the benchmark, a difference between an entropy of the given file and a benchmark entropy, and combinations of the foregoing.
6 . The computer-implemented method of claim 1 , wherein processing the score in the machine learning model includes processing the score trained on both files deemed normal and files deemed compromised by the ransomware.
7 . The computer-implemented method of claim 1 , wherein processing the score in the machine learning model comprises determining, using the probability, whether the given file has the anomaly for the specific type.
8 . The computer-implemented method of claim 1 , wherein the machine learning model is selected from the group consisting of a decision tree, a random forest, a neural network, a gradient boosting learning model, and combinations of the foregoing.