Automatically performing varied security scans on distributed files using machine learning techniques
Methods, apparatus, and processor-readable storage media for automatically performing varied security scans on distributed files using machine learning techniques are provided herein. An example computer-implemented method includes obtaining at least one input file from one of multiple source channels; identifying a data security scan operation, from a set of multiple data security scan operations, for the at least one input file by processing historical data attributed to the at least one input file using machine learning techniques; executing the identified data security scan operation on the at least one input file; generating a hash of the at least one input file and information pertaining to results of the executed data security scan operation; caching the generated hash in at least one cache; and performing automated actions based on the caching of the generated hash in the at least one cache.
1. A computer-implemented method comprising:
obtaining at least one input file from one of multiple source channels;
identifying a data security scan operation, from a designated set of multiple data security scan operations associated with multiple data security service providers, for the at least one input file by processing historical data attributed to the at least one input file using one or more machine learning techniques;
executing the identified data security scan operation on the at least one input file;
generating a hash of the at least one input file and information pertaining to results of the executed data security scan operation;
caching the generated hash in at least one cache; and
performing one or more automated actions based at least in part on the caching of the generated hash in the at least one cache;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
2. The computer-implemented method of claim 1 , wherein processing historical data attributed to the at least one input file using one or more machine learning techniques comprises processing historical data attributed to the at least one input file using at least one machine learning-based ensemble bagging algorithm.
3. The computer-implemented method of claim 1 , wherein generating a hash of the at least one input file and information pertaining to results of the executed data security scan operation comprises generating the hash using at least one secure hashing algorithm (SHA).
4. The computer-implemented method of claim 1 , wherein performing one or more automated actions comprises comparing data from an additional input file to the data within the at least one cache prior to executing a data security scan operation on the additional input file.
5. The computer-implemented method of claim 1 , wherein processing historical data attributed to the at least one input file comprises processing one or more historical data security scan metrics derived from previous data security scan operations related to the at least one input file.
6. The computer-implemented method of claim 1 , wherein processing historical data attributed to the at least one input file comprises processing one or more data features of the at least one input file.
7. The computer-implemented method of claim 6 , wherein the one or more data features of the at least one input file comprise at least one of file type, file size, and file source channel.
8. The computer-implemented method of claim 1 , wherein processing historical data attributed to the at least one input file using one or more machine learning techniques comprises leveraging chaining of at least one regressor and at least one classifier.
9. The computer-implemented method of claim 1 , wherein generating a hash comprises attributing a unique identifier of the at least one input file corresponding to the generated hash.
10. The computer-implemented method of claim 1 , further comprising:
performing at least one of a dynamic addition of one or more data security scan operations to the a set of multiple data security scan operations and a dynamic removal of one or more data security scan operations from the a set of multiple data security scan operations.
11. The computer-implemented method of claim 1 , further comprising:
exposing at least a portion of the method as an application programming interface.
12. The computer-implemented method of claim 1 , further comprising:
training the one or more machine learning techniques based at least in part on results of the executed data security scan operation.
13. A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device:
to obtain at least one input file from one of multiple source channels;
to identify a data security scan operation, from a designated set of multiple data security scan operations associated with multiple data security service providers, for the at least one input file by processing historical data attributed to the at least one input file using one or more machine learning techniques;
to execute the identified data security scan operation on the at least one input file;
to generate a hash of the at least one input file and information pertaining to results of the executed data security scan operation;
to cache the generated hash in at least one cache; and
to perform one or more automated actions based at least in part on the caching of the generated hash in the at least one cache.
14. The non-transitory processor-readable storage medium of claim 13 , wherein processing historical data attributed to the at least one input file using one or more machine learning techniques comprises processing historical data attributed to the at least one input file using at least one machine learning-based ensemble bagging algorithm.
15. The non-transitory processor-readable storage medium of claim 13 , wherein generating a hash of the at least one input file and information pertaining to results of the executed data security scan operation comprises generating the hash using at least one SHA.
16. The non-transitory processor-readable storage medium of claim 13 , wherein performing one or more automated actions comprises comparing data from an additional input file to the data within the at least one cache prior to executing a data security scan operation on the additional input file.
17. An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured:
to obtain at least one input file from one of multiple source channels;
to identify a data security scan operation, from a designated set of multiple data security scan operations associated with multiple data security service providers, for the at least one input file by processing historical data attributed to the at least one input file using one or more machine learning techniques;
to execute the identified data security scan operation on the at least one input file;
to generate a hash of the at least one input file and information pertaining to results of the executed data security scan operation;
to cache the generated hash in at least one cache; and
to perform one or more automated actions based at least in part on the caching of the generated hash in the at least one cache.
18. The apparatus of claim 17 , wherein processing historical data attributed to the at least one input file using one or more machine learning techniques comprises processing historical data attributed to the at least one input file using at least one machine learning-based ensemble bagging algorithm.
19. The apparatus of claim 17 , wherein generating a hash of the at least one input file and information pertaining to results of the executed data security scan operation comprises generating the hash using at least one SHA.
20. The apparatus of claim 17 , wherein performing one or more automated actions comprises comparing data from an additional input file to the data within the at least one cache prior to executing a data security scan operation on the additional input file.