Real-time detection of system threats
Some examples relate generally to managing and storing data, and more specifically to the real-time detection of ransomware, system (or insider) threats, or the misappropriation of credentials by using file system audit events.
1. A method for detection of whether a file system is infected with malware, the method comprising:
accessing audit events in the file system for a time interval, the audit events including unique file operations and duplicative file operations within the time interval;
de-duplicating the audit events to remove the duplicative file operations and retain the unique file operations from the audit events;
generating time series data that comprises the unique file operations and is devoid of the duplicative file operations;
analyzing the time series data to determine whether a subset of the unique file operations includes delete instructions to delete files corresponding to the subset of the unique file operations;
comparing a pattern of the delete instructions in the time interval to a normal pattern of delete instructions;
determining, based at least in part on the comparing, that the delete instructions in the subset of the unique file operations are abnormal based at least in part on a deviation between the pattern of the delete instructions in the time interval and the normal pattern of delete instructions;
responsive to determining that the delete instructions in the subset of the unique file operations are abnormal, determining that the file system is infected with malware; and
generating an alert based at least in part on determining that the file system is infected with malware.
2. The method of claim 1 , wherein the audit events include information comprising, for each audit event, a user identity, a file name, a type of access, a timestamp, or any combination thereof.
3. The method of claim 1 , wherein determining that the file system is infected with malware is further based at least in part on determining that the unique file operations include instructions to encrypt copies of deleted files associated with the delete instructions.
4. The method of claim 1 , wherein de-duplicating the audit events is based at least in part on an identification of successive file operations that do not lead to a change in a file state.
5. The method of claim 1 , further comprising:
in a pre-analysis phase, identifying the duplicative file operations based at least in part on the duplicative file operations being successive file operations that maintain corresponding files in file states associated with corresponding prior file operations.
6. The method of claim 1 , further comprising:
generating a finite state machine including one or more file states, the one or more file states including a file open state, a file read state, a file write state, a file read/write state, a file close state, or any combination thereof; and
storing the one or more file states in the finite state machine in a key-value object store.
7. The method of claim 6 , wherein de-duplicating the audit events comprises identifying, as the duplicative file operations, file operations that maintain a file system state based at least in part on the finite state machine.
8. The method of claim 1 , wherein determining that the delete instructions in the subset of the unique file operations are abnormal comprises applying a set of machine learning models to the audit events, the set of machine learning models trained to determine the pattern or a number of the delete instructions and to compare the pattern or the number of the delete instructions to the normal pattern of delete instructions or a normal number of delete instructions based at least in part on features representing a normal or expected behavior of the file system.
9. The method of claim 1 , wherein determining that the delete instructions in the subset of the unique file operations are abnormal comprises applying Seasonal-Trend Decomposition Procedure Based on Loess (STL) decomposition to file delete audit events to remove seasonal and trend components and using a residue of the STL decomposition to generate the time series data, and performing an Exploratory Data Analysis (ESD) test on the time series data.
10. An apparatus for managing virtual machines, comprising:
at least one processor;
memory coupled with the at least one processor; and
instructions stored in the memory and executable by the at least one processor to cause the apparatus to:
access audit events in a file system for a time interval, the audit events including unique file operations and duplicative file operations within the time interval;
de-duplicate the audit events to remove the duplicative file operations and retain the unique file operations from the audit events;
generate time series data that comprises the unique file operations and is devoid of the duplicative file operations;
analyze the time series data to determine whether a subset of the unique file operations includes delete instructions to delete files corresponding to the subset of the unique file operations;
compare a pattern of the delete instructions in the time interval to a normal pattern of delete instructions;
determine, based at least in part on the comparing, that the delete instructions in the subset of the unique file operations are abnormal based at least in part on a deviation between the pattern of the delete instructions in the time interval and the normal pattern of delete instructions;
responsive to determining that the delete instructions in the subset of the unique file operations are abnormal, determine that the file system is infected with malware; and
generate an alert based at least in part on determining that the file system is infected with malware.
11. The apparatus of claim 10 , wherein the audit events include information comprising, for each audit event, a user id, a file name, a type of access, a timestamp, or any combination thereof.
12. The apparatus of claim 10 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
determine that the file system is infected with malware further based at least in part on determining that the unique file operations include instructions to encrypt copies of deleted files associated with the delete instructions.
13. The apparatus of claim 10 , wherein the instructions are executable by the at least one processor to cause the apparatus to de-duplicate the audit events based at least in part on an identification of successive file operations that do not lead to a change in a file state.
14. The apparatus of claim 10 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
in a pre-analysis phase, identify the duplicative file operations based at least in part on the duplicative file operations being successive file operations that maintain corresponding files in file states associated with corresponding prior file operations.
15. The apparatus of claim 10 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
generate a finite state machine including one or more file states, the one or more file states including a file open state, a file read state, a file write state, a file read/write state, a file close state, or any combination thereof; and
store the one or more file states in the finite state machine in a key-value object store.
16. The apparatus of claim 15 , wherein, to de-duplicate the audit events, the instructions are executable by the at least one processor to cause the apparatus to identify, as the duplicative file operations, file operations that maintain a file system state based at least in part on the finite state machine.
17. The apparatus of claim 10 , wherein, to determine that the delete instructions in the subset of the unique file operations are abnormal, the instructions are executable by the at least one processor to cause the apparatus to apply a set of machine learning models to the audit events, the set of machine learning models trained to determine the pattern or a number of the delete instructions and to compare the pattern or the number of the delete instructions to the normal pattern of delete instructions or a normal number of delete instructions based at least in part on features representing a normal or expected behavior of the file system.
18. A non-transitory computer-readable medium storing code for managing virtual machines, the code comprising instructions executable by at least one processor to:
access audit events in a file system for a time interval, the audit events including unique file operations and duplicative file operations within the time interval;
de-duplicate the audit events to remove the duplicative file operations and retain the unique file operations from the audit events;
generate time series data that comprises the unique file operations and is devoid of the duplicative file operations;
analyze the time series data to determine whether a subset of the unique file operations includes delete instructions to delete files corresponding to the subset of the unique file operations;
compare a pattern of the delete instructions in the time interval to a normal pattern of delete instructions;
determine, based at least in part on the comparing, that the delete instructions in the subset of the unique file operations are abnormal based at least in part on a deviation between the pattern of the delete instructions in the time interval and the normal pattern of delete instructions;
responsive to determining that the delete instructions in the subset of the unique file operations are abnormal, determine that the file system is infected with malware; and
generate an alert based at least in part on determining that the file system is infected with malware.
19. The non-transitory computer-readable medium of claim 18 , wherein the instructions are further executable by the at least one processor to determine that the file system is infected with malware further based at least in part on determining that the unique file operations include instructions to encrypt copies of deleted files associated with the delete instructions.
20. The non-transitory computer-readable medium of claim 18 , wherein the instructions are executable by the at least one processor to de-duplicate the audit events based at least in part on an identification of successive file operations that do not lead to a change in a file state.