Realtime detection of ransomware

1. A method for real-time detection of a ransomware infection or malicious code in file systems, the method comprising:

accessing audit events in a file system during a time interval, the audit events including unique file operations and duplicative file operations within the time interval;

in a pre-analysis phase, identifying the duplicative file operations based at least in part on the duplicative file operations being successive file operations that maintain corresponding files in file states associated with corresponding prior file operations, and de-duplicating the audit events to remove the duplicative file operations and generate time series data comprising the unique file operations devoid of the duplicative file operations;

in an analysis phase, analyzing the time series data to determine whether a subset of the unique file operations includes delete instructions to delete files corresponding to the subset of the unique file operations;

determining that the delete instructions in the subset of the unique file operations are abnormal in the time interval based at least in part on determining a pattern or number of the delete instructions in the time interval and comparing the pattern or number of the delete instructions to a normal pattern or number of delete instructions;

responsive to determining that the delete instructions in the subset of the unique file operations are abnormal, determining that the file system is infected with ransomware; and

generating an alert.

2. The method of claim 1 , wherein the audit events include information comprising, for each audit event, a user identity, a file name, a type of access, and a timestamp.

3. The method of claim 1 , further comprising determining whether the subset of the unique file operations includes instructions to encrypt copies of the deleted files corresponding to the subset of the unique file operations and to delete unencrypted original files.

4. The method of claim 1 , wherein removing the duplicative file operations in the de-duplication of the audit events is based at least in part on an identification of the successive file operations as not leading to a change in a file state.

5. The method of claim 1 , further comprising:

generating a finite state machine including one or more file states, the file states including a file open state, a file read state, a file write state, a file read/write state, and a file close state; and

storing the file states in the finite state machine in a key value object store.

6. The method of claim 1 , wherein determining whether the delete instructions in the subset of the unique file operations are abnormal comprises applying a set of machine learning models to the audit events, the set of machine learning models trained to determine the pattern or number of the unique file operations and to compare the pattern or number of the unique file operations to the normal pattern or number based on features representing a normal or expected behavior of the file system.

7. The method of claim 5 , wherein de-duplicating the audit events includes maintaining a file system state based on the finite state machine.

8. The method of claim 1 , wherein determining that the delete instructions in the subset of the unique file operations are abnormal comprises applying Seasonal-Trend Decomposition Procedure Based on Loess (STL) decomposition to file delete audit events to remove seasonal and trend components and using a residue of the decomposition to generate the time series data, and performing an Exploratory Data Analysis (ESD) test on the time series data.

9. A system for real-time detection of a ransomware infection or malicious code in file systems, the system comprising:

at least one processor for executing machine-readable instructions; and

a memory storing instructions configured to cause the at least one processor to perform operations comprising, at least:

accessing audit events in a file system during a time interval, the audit events including unique file operations and duplicative file operations within the time interval;

in a pre-analysis phase, identifying the duplicative file operations based at least in part on the duplicative file operations being successive file operations that maintain corresponding files in file states associated with corresponding prior file operations, and de-duplicating the audit events to remove the duplicative file operations and generate time series data comprising the unique file operations devoid of the duplicative file operations;

in an analysis phase, analyzing the time series data to determine whether a subset of the unique file operations includes delete instructions to delete files corresponding to the subset of the unique file operations;

determining that the delete instructions in the subset of the unique file operations are abnormal in the time interval based at least in part on determining a pattern or number of the delete instructions in the time interval and comparing the pattern or number of the delete instructions to a normal pattern or number of delete instructions;

responsive to determining that the delete instructions in the subset of the unique file operations are abnormal, determining that the file system is infected with ransomware; and

generating an alert.

10. The system of claim 9 , wherein the audit events include information comprising, for each audit event, a user identity, a file name, a type of access, and a timestamp.

11. The system of claim 9 , wherein the operations further comprise determining whether the subset of the unique file operations includes instructions to encrypt copies of the deleted files corresponding to the subset of the unique file operations and to delete unencrypted original files.

12. The system of claim 9 , wherein removing the duplicative file operations in the de-duplication of the audit events is based at least in part on an identification of the successive file operations as not leading to a change in a file state.

13. The system of claim 9 , wherein the operations further comprise:

generating a finite state machine including one or more file states, the file states including a file open state, a file read state, a file write state, a file read/write state, and a file close state; and

storing the file states in the finite state machine in a key value object store.

14. The system of claim 9 , wherein determining whether the delete instructions in the subset of the unique file operations are abnormal comprises applying a set of machine learning models to the audit events, the set of machine learning models trained to determine the pattern or number of the unique file operations and to compare the pattern or number of the unique file operations to the normal pattern or number based on features representing a normal or expected behavior of the file system.

15. The system of claim 13 , wherein de-duplicating the audit events includes maintaining a file system state based on the finite state machine.

16. The system of claim 9 , wherein determining that the delete instructions in the subset of the unique file operations are abnormal comprises applying Seasonal-Trend Decomposition Procedure Based on Loess (STL) decomposition to file delete audit events to remove seasonal and trend components and using a residue of the decomposition to generate the time series data, and performing an Exploratory Data Analysis (ESD) test on the time series data.

17. A non-transitory, machine-readable medium storing instructions which, when read by a machine, cause the machine to perform operations in a method for real-time detection of a ransomware infection or malicious code in file systems, the operations comprising, at least:

accessing audit events in a file system during a time interval, the audit events including unique file operations and duplicative file operations within the time interval;

in a pre-analysis phase, identifying the duplicative file operations based at least in part on the duplicative file operations being successive file operations that maintain corresponding files in file states associated with corresponding prior file operations, and de-duplicating the audit events to remove the duplicative file operations and generate time series data comprising the unique file operations devoid of the duplicative file operations;

in an analysis phase, analyzing the time series data to determine whether a subset of the unique file operations includes delete instructions to delete files corresponding to the subset of the unique file operations;

determining that the delete instructions in the subset of the unique file operations are abnormal in the time interval based at least in part on determining a pattern or number of the delete instructions in the time interval and comparing the pattern or number of the delete instructions to a normal pattern or number of delete instructions;

responsive to determining that the delete instructions in the subset of the unique file operations are abnormal, determining that the file system is infected with ransomware; and

generating an alert.

18. The medium of claim 17 , wherein the audit events include information comprising, for each audit event, a user identity, a file name, a type of access, and a timestamp.

19. The medium of claim 17 , wherein the operations further comprise determining whether the subset of the unique file operations includes instructions to encrypt copies of the deleted files corresponding to the subset of the unique file operations and to delete unencrypted original files.

20. The medium of claim 17 , wherein removing the duplicative file operations in the de-duplication of the audit events is based at least in part on an identification of the successive file operations as not leading to a change in a file state.