Scalable automated training framework
Techniques for implementing a scalable automated training framework for anomaly and ransomware detection are disclosed. In some embodiments, a computer system performs operations comprising: instantiating a plurality of virtual machines, each one of the virtual machines being loaded with a corresponding file system; simulating user actions and ransomware on the virtual machines, the simulating of user actions and ransomware on the virtual machines causing changes to the corresponding file systems of the virtual machines; for each one of the plurality of virtual machines, generating a corresponding metadata file based on one or more corresponding snapshots of the virtual machine, the one or more corresponding snapshots indicating the changes to the corresponding file system of the virtual machine; and training a ransomware detection model using a machine learning algorithm and training data, the training data being based on the corresponding metadata files of the virtual machines.
1 . A method, comprising:
executing, by a computer system that comprises one or more memories and one or more processors and is configured for performance of data backups and malware detection, a computer program that simulates a plurality of user actions on a plurality of virtual machines in the presence of simulated malware, wherein the plurality of virtual machines are associated with corresponding file systems, and wherein simulating the plurality of user actions causes a plurality of changes to the corresponding file systems associated with the plurality of virtual machines;
storing, by the computer system, training data in a metadata store of the computer system, wherein:
the training data is based at least in part on simulating the plurality of user actions on the plurality of virtual machines in the presence of the simulated malware, and
the training data comprises the plurality of changes to the corresponding file systems of the plurality of virtual machines; and
training a machine learning model for malware detection based at least in part on the training data.
2 . The method of claim 1 , wherein training the machine learning model for the malware detection comprises:
training the machine learning model for detection of a first type of malware from among a plurality of candidate malware types based at least in part on a subset of training data from among the training data, wherein the subset of training data is associated with the first type of malware.
3 . The method of claim 1 , further comprising:
obtaining, after executing the computer program that simulates the plurality of user actions on the plurality of virtual machines, a plurality of snapshots of the plurality of virtual machines, wherein the plurality of snapshots indicate the plurality of changes to the corresponding file systems associated with the plurality of virtual machines, and wherein the training data is based at least in part on the plurality of snapshots.
4 . The method of claim 3 , further comprising:
obtaining, for a virtual machine of the plurality of virtual machines, a first snapshot at a first point in time, the first snapshot comprising a first set of changes to a corresponding file system of the virtual machine at the first point in time;
obtaining, for the virtual machine, a second snapshot at a second point in time, the second snapshot comprising a second set of changes to the corresponding file system of the virtual machine at the second point in time; and
comparing the second set of changes included in the second snapshot with the first set of changes included in the first snapshot, wherein generating the training data is based at least in part on the comparing.
5 . The method of claim 1 , wherein executing the computer program to simulate the plurality of user actions comprises:
simulating manipulation, selection, or both of one or more elements of a user interface associated with the plurality of virtual machines.
6 . The method of claim 1 , wherein executing the computer program to simulate the plurality of user actions comprises:
simulating navigation of one or more resources of an operating system of a virtual machine of the plurality of virtual machines.
7 . The method of claim 1 , wherein executing the computer program to simulate the plurality of user actions comprises:
simulating navigation of the Internet via a web browser.
8 . The method of claim 1 , further comprising:
detecting, based at least in part on the machine learning model trained using the training data, whether malware is preset on the plurality of virtual machines.
9 . The method of claim 1 , further comprising:
detecting, based at least in part on the machine learning model trained using the training data, whether malware is preset on a computing device that is different than the plurality of virtual machines.
10 . The method of claim 1 , further comprising:
generating, based at least in part on simulating the plurality of user actions, a plurality of metadata files for the plurality of virtual machines, wherein the plurality of metadata files comprise the training data for the malware detection.
11 . The method of claim 1 , wherein the simulated malware comprises simulated ransomware on the plurality of virtual machines.
12 . The method of claim 1 , wherein the training data is associated with training the machine learning model that comprises a deep neural network, a logistic regression, or both.
13 . An apparatus configured for performance of data backups and malware detection, comprising:
one or more memories storing processor-executable code; and
one or more processors coupled with the one or more memories and individually or collectively operable to execute the code to cause the apparatus to:
execute, by the apparatus, a computer program that simulates a plurality of user actions on a plurality of virtual machines in the presence of simulated malware, wherein the plurality of virtual machines are associated with corresponding file systems, and wherein simulating the plurality of user actions causes a plurality of changes to the corresponding file systems associated with the plurality of virtual machines;
store, by the apparatus, training data in a metadata store of the apparatus, wherein:
the training data is based at least in part on simulating the plurality of user actions on the plurality of virtual machines in the presence of the simulated malware, and
the training data comprises the plurality of changes to the corresponding file systems of the plurality of virtual machines; and
train a machine learning model for malware detection based at least in part on the training data.
14 . The apparatus of claim 13 , wherein, to train the machine learning model, the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
train the machine learning model for detection of a first type of malware from among a plurality of candidate malware types based at least in part on a subset of training data from among the training data, wherein the subset of training data is associated with the first type of malware.
15 . The apparatus of claim 13 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
obtain, after executing the computer program that simulates the plurality of user actions on the plurality of virtual machines, a plurality of snapshots of the plurality of virtual machines, wherein the plurality of snapshots indicate the plurality of changes to the corresponding file systems associated with the plurality of virtual machines, and wherein the training data is based at least in part on the plurality of snapshots.
16 . The apparatus of claim 15 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
obtain, for a virtual machine of the plurality of virtual machines, a first snapshot at a first point in time, the first snapshot comprising a first set of changes to a corresponding file system of the virtual machine at the first point in time;
obtain, for the virtual machine, a second snapshot at a second point in time, the second snapshot comprising a second set of changes to the corresponding file system of the virtual machine at the second point in time; and
compare the second set of changes included in the second snapshot with the first set of changes included in the first snapshot, wherein generating
the training data is based at least in part on the comparing.
17 . The apparatus of claim 13 , wherein, to execute the computer program to simulate the plurality of user actions, the one or more processors are individually or collectively operable to execute the code to cause the apparatus to:
simulate manipulation, selection, or both of one or more elements of a user interface associated with the plurality of virtual machines.
18 . The apparatus of claim 13 , wherein, to execute the computer program to simulate the plurality of user actions, the one or more processors are individually or collectively operable to execute the code to cause the apparatus to:
simulate navigation of one or more resources of an operating system of a virtual machine of the plurality of virtual machines.
19 . A non-transitory computer-readable medium storing code, the code comprising instructions executable by one or more processors to:
execute, by a computer system that comprises one or more memories and one or more processors and is configured for performance of data backups and malware detection, a computer program that simulates a plurality of user actions on a plurality of virtual machines in the presence of simulated malware, wherein the plurality of virtual machines are associated with corresponding file systems, and wherein simulating the plurality of user actions causes a plurality of changes to the corresponding file systems associated with the plurality of virtual machines;
store, by the computer system, training data in a metadata store of the computer system, wherein:
the training data is based at least in part on simulating the plurality of user actions on the plurality of virtual machines in the presence of the simulated malware, and
the training data comprises the plurality of changes to the corresponding file systems of the plurality of virtual machines; and
train a machine learning model for malware detection based at least in part on the generated training data.
20 . The non-transitory computer-readable medium of claim 19 , wherein, to train the machine learning model, the instructions are executable by the one or more processors to:
train the machine learning model for detection of a first type of malware from among a plurality of candidate malware types based at least in part on a subset of training data from among the training data, wherein the subset of training data is associated with the first type of malware.