Predicting drive failures
Predicting individual drive failures is achieved using machine learning models of drive behavior history based on samples of SMART data attributes collected over distinct time-periods. The drive behavior history is a historical feature added to drive features modeled based on a last sample of SMART data attributes. The drive behavior history feature is used in successive modeling of drive behavior history to increase accuracy in predicting an individual drive's failure over time. Consecutive individual drive failure predictions are aggregated to further increase accuracy in predicting an individual drive's failure. In one embodiment, the system models drive behavior history and other drive features using a machine learning model. Individual drives classified as predicted to fail within a certain period of time are incorporated into a drive replacement strategy that factors in a field-based replacement cost associated with the drive.
1. A computer-implemented method for predicting drive failures, the method comprising:
collecting any one or more samples of drive health indicators from a drive over a specified time period, wherein the samples of drive health indicators include one or more Self-Monitoring, Analysis and Reporting Technology (SMART) attributes obtained from the drive;
performing a first feature selection modeling of a last collected sample of SMART drive health indicators to generate a drive feature for the drive, the drive feature for modeling a drive health at a time of the last collected sample;
performing a second feature engineering modeling of collected samples of SMART drive health indicators over the specified time period to generate one or more drive behavior history features for the drive, the drive behavior history features for modeling the drive health over the specified time period; and
classifying the drive as more likely to experience failure than other drives, the classifying based on predicted drive failure probabilities representing the drive health, including:
the drive health at the time of the last collected sample as modeled by the drive feature, and
the drive health over the specified time period as modeled by the drive behavior history features.
2. The computer-implemented method of claim 1 , wherein the first and second modeling are performed using a machine learning model.
3. The computer-implemented method of claim 1 , wherein the drive health indicators are any one or more of attributes specified in the Self-Monitoring, Analysis and Reporting Technology (SMART) industrial standard for disk drives.
4. The computer-implemented method of claim 1 , wherein the drive behavior history features for the drive are derived from any function describing values of drive health indicators over the specified time period as obtained from the collected samples.
5. The computer-implemented method of claim 1 , further comprising:
performing consecutive modeling of last collected samples and collected samples over the specified time period;
aggregating the predicted drive failure probabilities resulting from the consecutive modeling; and
classifying the drive as more likely to experience failure than other drives, the classifying based on the drive health over the time period spanned by the consecutive modeling as modeled by the aggregated predicted drive failure probabilities.
6. The computer-implemented method of claim 1 , further comprising:
obtaining replacement cost data for the drive; and
generating a drive replacement strategy for the drive classified as more likely to experience failure based on the replacement cost data.
7. A data processing system comprising:
a distributed file system in which to store drive behavior datasets containing features modeling drive health of a plurality of drives operating in a storage system;
a processor in communication with the distributed file system and the plurality of drives operating in the storage system, the processor configured to:
collect any one or more samples of drive health indicators from a drive over a specified time period, wherein the samples of drive health indicators include one or more Self-Monitoring, Analysis and Reporting Technology (SMART) attributes obtained from the drive;
perform a first feature selection modeling of a last collected sample of SMART drive health indicators to generate a drive feature for the drive, the drive feature modeling a drive health at a time of the last collected sample;
perform a second feature engineering modeling of collected samples of SMART drive health indicators over the specified time period to generate one or more drive behavior history features for the drive, the drive behavior history features modeling the drive health over the specified time period; and
classify the drive as more likely to experience failure than other drives, the classifying based on predicted drive failure probabilities representing the drive health, including:
the drive health at the time of the last collected sample as modeled by the drive feature, and
the drive health over the specified time period as modeled by the drive behavior history features.
8. The data processing system of claim 7 , wherein the first and second modeling are performed using a machine learning model.
9. The data processing system of claim 7 , wherein the drive health indicators are any one or more of attributes specified in the Self-Monitoring, Analysis and Reporting Technology (SMART) industrial standard for disk drives.
10. The data processing system of claim 7 , wherein the drive behavior history features for the drive are derived from any function describing values of drive health indicators over the specified time period as obtained from the collected samples.
11. The data processing system of claim 7 , further comprising:
performing consecutive modeling of last collected samples and collected samples over the specified time period; and
aggregating the predicted drive failure probabilities resulting from the consecutive modeling; and
classifying the drive as more likely to experience failure than other drives, the classifying based on the drive health over the time period spanned by the consecutive modeling as modeled by the aggregated predicted drive failure probabilities.
12. The data processing system of claim 7 , further comprising:
obtaining replacement cost data for the drive; and
generating a drive replacement strategy for the drive classified as more likely to experience failure based on the replacement cost data.
13. A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for predicting drive failures, the operations comprising:
collect any one or more samples of drive health indicators from a drive over a specified time period, wherein the samples of drive health indicators include one or more Self-Monitoring, Analysis and Reporting Technology (SMART) attributes obtained from the drive;
perform a first feature selection modeling of a last collected sample of SMART drive health indicators to generate a drive feature for the drive, the drive feature modeling a drive health at a time of the last collected sample;
perform a second feature engineering modeling of collected samples of SMART drive health indicators over the specified time period to generate one or more drive behavior history features for the drive, the drive behavior history features modeling the drive health over the specified time period; and
classify the drive as more likely to experience failure than other drives, the classifying based on predicted drive failure probabilities representing the drive health, including:
the drive health at the time of the last collected sample as modeled by the drive feature, and
the drive health over the specified time period as modeled by the drive behavior history features.
14. The non-transitory computer-readable storage medium of claim 13 , wherein the first and second modeling are performed using a machine learning model.
15. The non-transitory computer-readable storage medium of claim 13 , wherein the drive health indicators are any one or more of attributes specified in the Self-Monitoring, Analysis and Reporting Technology (SMART) industrial standard for disk drives.
16. The non-transitory computer-readable storage medium of claim 13 , wherein the drive behavior history features for the drive are derived from any function describing values of drive health indicators over the specified time period as obtained from the collected samples.
17. The non-transitory computer-readable storage medium of claim 13 , the operations further comprising:
perform consecutive modeling of last collected samples and collected samples over the specified time period;
aggregate the predicted drive failure probabilities resulting from the consecutive modeling; and
classify the drive as more likely to experience failure than other drives, the classifying based on the drive health over the time period spanned by the consecutive modeling as modeled by the aggregated predicted drive failure probabilities.
18. The non-transitory computer-readable storage medium of claim 13 , the operations further comprising:
obtain replacement cost data for the drive; and
generate a drive replacement strategy for the drive classified as more likely to experience failure based on the replacement cost data.