SYSTEM AND METHOD FOR DETECTING EVENT ANOMALIES USING A NORMALIZATION MODEL ON A SET OF STORAGE DEVICES
A method for managing storage devices includes obtaining, by a storage device event manager, a set of storage device telemetry snapshots is associated with a set of storage devices, generating a telemetry summary correlation matrix using the set of storage device telemetry snapshots, performing, using the telemetry summary correlation matrix, a classification of each storage device in the set of storage devices to obtain a set of classification tags using a first portion of a set of features, obtaining a set of normality states for the set of storage devices using the set of classification tags and a second portion of the set of features, updating an event anomaly policy based on the set of normality states, and performing a remediation action on a storage device in the set of storage devices based on the event anomaly policy.
1 . A method for managing a plurality of storage devices, the method comprising:
obtaining, by a storage device event manager, a set of storage device telemetry snapshots associated with a set of storage devices;
generating a telemetry summary correlation matrix using the set of storage device telemetry snapshots;
performing, using the telemetry summary correlation matrix, a classification of each storage device in the set of storage devices to obtain a set of classification tags using a first portion of a set of features;
obtaining a set of normality states for the set of storage devices using the set of classification tags and a second portion of the set of features;
updating an event anomaly policy based on the set of normality states; and
performing a remediation action on a storage device in the set of storage devices based on the event anomaly policy.
2 . The method of claim 1 , wherein the set of normality states is further obtained using a normality model.
3 . The method of claim 2 , the method further comprising:
obtaining a second set of storage device telemetry snapshots, wherein the second set of storage device telemetry snapshots is associated with a second set of storage devices;
generating a second telemetry summary correlation matrix using the second set of storage device telemetry snapshots and using a set of variables;
performing a feature extraction on the set of variables to obtain the set of features;
performing a grouping on the second set of storage devices based on the first portion of the set of features and the second telemetry summary correlation matrix; and
generating the normality model based on the grouping and the second portion of the set of features.
4 . The method of claim 3 , wherein a storage device telemetry snapshot in the second set of storage devices comprises a variable in the set of variables as a function of time.
5 . The method of claim 1 , wherein the set of storage devices is grouped into storage device pools.
6 . The method of claim 1 , wherein the remediation action comprises at least one of: transferring data from the storage device to a second storage device, reducing a write rate of the storage device, and replacing the storage device.
7 . The method of claim 1 ,
wherein the first portion of the set of features comprises configuration variables and workload variables, and
wherein the second portion of the set of features comprises performance variables.
8 . A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for managing a plurality of storage devices, the method comprising:
obtaining, by a storage device event manager, a set of storage device telemetry snapshots associated with a set of storage devices;
generating a telemetry summary correlation matrix using the set of storage device telemetry snapshots;
performing, using the telemetry summary correlation matrix, a classification of each storage device in the set of storage devices to obtain a set of classification tags using a first portion of a set of features;
obtaining a set of normality states for the set of storage devices using the set of classification tags and a second portion of the set of features;
updating an event anomaly policy based on the set of normality states; and
performing a remediation action on a storage device in the set of storage devices based on the event anomaly policy.
9 . The non-transitory computer readable medium of claim 8 , wherein the set of normality states is further obtained using a normality model.
10 . The non-transitory computer readable medium of claim 9 , the method further comprising:
obtaining a second set of storage device telemetry snapshots, wherein the second set of storage device telemetry snapshots is associated with a second set of storage devices;
generating a second telemetry summary correlation matrix using the second set of storage device telemetry snapshots and using a set of variables;
performing a feature extraction on the set of variables to obtain the set of features;
performing a grouping on the second set of storage devices based on the first portion of the set of features and the second telemetry summary correlation matrix; and
generating the normality model based on the grouping and the second portion of the set of features.
11 . The non-transitory computer readable medium of claim 10 , wherein a storage device telemetry snapshot in the second set of storage devices comprises a variable in the set of variables as a function of time.
12 . The non-transitory computer readable medium of claim 8 , wherein the set of storage devices is grouped into storage device pools.
13 . The non-transitory computer readable medium of claim 8 , wherein the remediation action comprises at least one of: transferring data from the storage device to a second storage device, reducing a write rate of the storage device, and replacing the storage device.
14 . The non-transitory computer readable medium of claim 8 ,
wherein the first portion of the set of features comprises configuration variables and workload variables, and
wherein the second portion of the set of features comprises performance variables.
15 . A system, comprising:
a processor; and
memory comprising instructions which, when executed by the processor, perform a method, the method comprising:
obtaining, by a storage device event manager, a set of storage device telemetry snapshots associated with a set of storage devices;
generating a telemetry summary correlation matrix using the set of storage device telemetry snapshots;
performing, using the telemetry summary correlation matrix, a classification of each storage device in the set of storage devices to obtain a set of classification tags using a first portion of a set of features;
obtaining a set of normality states for the set of storage devices using the set of classification tags and a second portion of the set of features;
updating an event anomaly policy based on the set of normality states; and
performing a remediation action on a storage device in the set of storage devices based on the event anomaly policy.
16 . The system of claim 15 , wherein the set of normality states is further obtained using a normality model.
17 . The system of claim 16 , the method further comprising:
obtaining a second set of storage device telemetry snapshots, wherein the second set of storage device telemetry snapshots is associated with a second set of storage devices;
generating a second telemetry summary correlation matrix using the second set of storage device telemetry snapshots and using a set of variables;
performing a feature extraction on the set of variables to obtain the set of features;
performing a grouping on the second set of storage devices based on the first portion of the set of features and the second telemetry summary correlation matrix; and
generating the normality model based on the grouping and the second portion of the set of features.
18 . The system of claim 17 , wherein a storage device telemetry snapshot in the second set of storage devices comprises a variable in the set of variables as a function of time.
19 . The system of claim 15 , wherein the remediation action comprises at least one of: transferring data from the storage device to a second storage device, reducing a write rate of the storage device, and replacing the storage device.
20 . The system of claim 15 ,
wherein the first portion of the set of features comprises configuration variables and workload variables, and
wherein the second portion of the set of features comprises performance variables.