Aggregating input/output operation features extracted from storage devices to form a machine learning vector to check for malware
Provided are a computer program product, system, and method for aggregating input/output operation features extracted from storage devices to form a machine learning vector to check for malware. Feature extraction functions are generated for the storage devices, indicating I/O operation features for the storage devices to gather. The feature extraction functions are communicated to the storage devices. The feature extraction functions transmitted to the storage devices cause the storage devices to gather information on I/O operation features, identified in the feature extraction functions, from the storage devices and transmit the information on the I/O operation features to the storage controller. The information on the I/O operation features are received from the storage devices. Information based on the received information on the I/O operation features are inputted into a machine learning model to output indication whether data in the storage devices contains malware.
1 . A computer program product for gathering I/O operation features from a plurality of storage devices, wherein the computer program product comprises a computer readable storage medium, implemented in a storage controller, having computer readable program code embodied therein that when executed performs operations, the operations comprising:
generating feature extraction functions, for the storage devices, indicating I/O operation features for the storage devices to gather, wherein the I/O operation features comprise statistics and characteristics of read and write requests at the storage devices predictive of whether malware is contained in the storage devices;
communicating the feature extraction functions to the storage devices, wherein the feature extraction functions transmitted to the storage devices cause each storage device of the storage devices receiving a feature extraction function to perform:
gathering information on I/O operation features, identified in the feature extraction functions, from the storage device;
aggregating the gathered information by volume and I/O operation feature into aggregated information by volume and I/O operation feature for the I/O operation features gathered for volumes at the storage device; and
transmitting the aggregated information by volume and I/O operation feature for the volumes and the I/O operation features to the storage controller;
for the volumes, aggregating the aggregated information for a given volume from the storage devices into system-wide aggregated information having the I/O operation features for the given volume; and
inputting the system-wide aggregated information for the given volume into a machine learning model to output indication whether the given volume contains malware.
2 . The computer program product of claim 1 , wherein at least two of the feature extraction functions include different subsets of at least one I/O operation feature to control at least two of the storage devices to gather and aggregate information on different I/O operation features, and transmit to the storage controller.
3 . The computer program product of claim 1 , wherein at least two of the feature extraction functions indicate a same I/O operation feature for at least two of the storage devices to gather information on the same I/O operation feature, and transmit to the storage controller.
4 . The computer program product of claim 1 , wherein the feature extraction functions communicated to the storage devices evenly distribute the I/O operation features to the storage devices and assign the I/O operation features to at least two of the storage devices.
5 . The computer program product of claim 1 , wherein the I/O operation features comprise at least a plurality of features of a set of features consisting of: entropy of data in a storage device, a compression ratio of the data in a storage device; logical block addresses (LBAs) to which I/O operations are directed; an I/O type; I/O size; I/O request rate; number of rewrites; and read and write heat of regions of the storage device.
6 . The computer program product of claim 1 , wherein the feature extraction functions cause the storage devices to gather information on I/O operation features for different volumes for which data is stored at the storage devices.
7 . The computer program product of claim 1 , wherein there are at least two groups of storage devices, each group of the groups of storage devices are associated with different time intervals, wherein the feature extraction functions sent to the storage devices in each group of the groups causes the storage devices in each group to gather information on the I/O operation features for a time interval associated with each group,
wherein there are machine learning models associated with different time intervals, wherein the information on the I/O operation features received for a group of storage devices are inputted into a machine learning model associated with the time interval associated with the group of storage devices from which the information was received.
8 . The computer program product of claim 1 , wherein the operations further comprise:
forming a vector for each volume comprising the system-wide aggregated information having the I/O operation features for each volume, wherein the vector for each volume is inputted into the machine learning model to determine whether the volumes contain malware.
9 . The computer program product of claim 1 , wherein the aggregating, at each storage device, the gathered information on an I/O operation feature comprises calculating a mean and variance of the gathered information for the I/O operation feature.
10 . A system for gathering I/O operation features from a plurality of storage devices, comprising:
a processor; and
a computer readable storage medium, implemented in a storage controller, having computer readable program code embodied therein that when executed performs operations, the operations comprising:
generating feature extraction functions, for the storage devices, indicating I/O operation features for the storage devices to gather, wherein the I/O operation features comprise statistics and characteristics of read and write requests at the storage devices predictive of whether malware is contained in the storage devices;
communicating the feature extraction functions to the storage devices, wherein the feature extraction functions transmitted to the storage devices cause each storage device of the storage devices receiving a feature extraction function to perform:
gathering information on I/O operation features, identified in the feature extraction functions, from the storage device;
aggregating the gathered information by volume and I/O operation feature into aggregated information by volume and I/O operation feature for the I/O operation features gathered for volumes at the storage device; and
transmitting the aggregated information by volume and I/O operation feature for the volumes and the I/O operation features to the storage controller;
for the volumes, aggregating the aggregated information for a given volume from the storage devices into system-wide aggregated information having the I/O operation features for the given volume; and
inputting the system-wide aggregated information for the given volume into a machine learning model to output indication whether the given volume contains malware.
11 . The system of claim 10 , wherein at least two of the feature extraction functions include different subsets of at least one I/O operation feature to control at least two of the storage devices to gather and aggregate information on different I/O operation features, and transmit to the storage controller.
12 . The system of claim 10 , wherein at least two of the feature extraction functions indicate a same I/O operation feature for at least two of the storage devices to gather information on the same I/O operation feature, and transmit to the storage controller.
13 . The system of claim 10 , wherein the feature extraction functions communicated to the storage devices evenly distribute the I/O operation features to the storage devices and assign the I/O operation features to at least two of the storage devices.
14 . The system of claim 10 , wherein there are at least two groups of storage devices, each group of the groups of storage devices are associated with different time intervals, wherein the feature extraction functions sent to the storage devices in each group of the groups causes the storage devices in each group to gather information on the I/O operation features for a time interval associated with each group,
wherein there are machine learning models associated with different time intervals, wherein the information on the I/O operation features received for a group of storage devices are inputted into a machine learning model associated with the time interval associated with the group of storage devices from which the information was received.
15 . A computer implemented method for gathering I/O operation features from a plurality of storage devices, comprising:
generating feature extraction functions, for the storage devices, indicating I/O operation features for the storage devices to gather, wherein the I/O operation features comprise statistics and characteristics of read and write requests at the storage devices predictive of whether malware is contained in the storage devices;
communicating the feature extraction functions to the storage devices, wherein the feature extraction functions transmitted to the storage devices cause each storage device of the storage devices receiving a feature extraction function to perform:
gathering information on I/O operation features, identified in the feature extraction functions, from the storage device;
aggregating the gathered information by volume and I/O operation feature into aggregated information by volume and I/O operation feature for the I/O operation features gathered for volumes at the storage device; and
transmitting the aggregated information by volume and I/O operation feature for the volumes and the I/O operation features to a storage controller;
for the volumes, aggregating the aggregated information for a given volume from the storage devices into system-wide aggregated information having the I/O operation features for the given volume; and
inputting the system-wide aggregated information for the given volume into a machine learning model to output indication whether the given volume contains malware.
16 . The computer implemented method of claim 15 , wherein at least two of the feature extraction functions include different subsets of at least one I/O operation feature to control at least two of the storage devices to gather and aggregate information on different I/O operation features, and transmit to the storage controller.
17 . The computer implemented method of claim 15 , wherein at least two of the feature extraction functions indicate a same I/O operation feature for at least two of the storage devices to gather information on the same I/O operation feature, and transmit to the storage controller.
18 . The computer implemented method of claim 15 , wherein the feature extraction functions communicated to the storage devices evenly distribute the I/O operation features to the storage devices and assign the I/O operation features to at least two of the storage devices.
19 . The system of claim 10 , wherein the operations further comprise:
forming a vector for each volume comprising the system-wide aggregated information having the I/O operation features for each volume, wherein the vector for each volume is inputted into the machine learning model to determine whether the volumes contain malware.
20 . The computer implemented method of claim 15 , further comprising:
forming a vector for each volume comprising the system-wide aggregated information having the I/O operation features for each volume, wherein the vector for each volume is inputted into the machine learning model to determine whether the volumes contain malware.