Method and system for a storage (SSD) drive-level failure and health prediction leveraging machine learning on internal parametric data
Systems and methods for solid-state storage drive-level failure prediction and health metric are described. A plurality of host-write commands are received at a solid-state storage device. A number of drive-writes per day based on the on the plurality of host-write commands is determined. An aggregated amount of degradation to one or more internal non-volatile memory components based on the number of drive-writes per day is determined. Using a machine-learned model, a probability of failure value based on a set of parameter data and the aggregated amount of degradation to the non-volatile memory component is generated. An alert is generated, based on the probability of failure value or degradation threshold.
1. A method comprising:
identifying combinations of input parameter values of a machine-learned model, the combinations capable of resulting in a non-volatile memory component failure;
ranking a set of parameters of the machine-learned model between a high ranking and a low ranking, the ranking associated with the identifying;
selecting, from the set of parameters, a plurality of input parameters for the machine-learned model based on the ranking;
generating or modifying a set of weights to apply to input parameter values based on the ranking;
receiving a plurality of host-write commands at a solid-state storage device;
calculating, based on the plurality of host-write commands, a number of drive-writes per day;
calculating, based on the number of drive-writes per day, an aggregated amount of degradation to a non-volatile memory component of the solid-state storage device;
generating, using the machine-learned model, a probability of failure value based on data associated with the plurality of input parameters for the machine-learned model and the aggregated amount of degradation to the non-volatile memory component of the solid-state storage device; and
generating, based on the probability of failure value, an alert,
wherein:
the set of parameters comprises the plurality of input parameters for the machine-learned model;
the aggregated amount of degradation to the non-volatile memory component comprises a throughput acceleration coefficient;
the throughput acceleration coefficient is determined by dividing the number of drive-writes per day by a drive-writes per day specification;
the number of drive-writes per day is determined based on the plurality of host-write commands, a total amount of data written to the non-volatile memory component, and a data storage capacity of the non-volatile memory component;
the drive-writes per day specification is a pre-determined number of drive-writes per day for the non-volatile memory component to handle in order for the solid-state storage device to operate for a specific period of time; and
the solid-state storage device is configured to store the drive-writes per day specification.
2. The method of claim 1 , wherein the alert is generated in response to the probability of failure value satisfying a threshold probability of failure value.
3. The method of claim 1 , wherein the aggregated amount of degradation to the non-volatile memory component comprises average drive-writes per day over a lifetime of the non-volatile memory component.
4. The method of claim 1 , wherein the machine-learned model is implemented within a firmware of the solid-state storage device.
5. The method of claim 1 , wherein the data associated with the plurality of input parameters comprises:
non-volatile memory translation layer management events data;
controller initiated data movement for non-volatile memory endurance management event data;
non-volatile memory defect density data;
controller initiated data movement in non-volatile memory event data;
controller initiated data movement in non-volatile memory for managing defragmentation of the non-volatile memory event data;
non-volatile memory bit error rate data;
controller initiated non-volatile memory erase or rewrite operation management data; and
controller initiated event for non-volatile memory data movement and correction in response to errors data.
6. The method of claim 1 ,
wherein the selecting comprises selecting the plurality of input parameters for the machine-learned model having the high ranking,
wherein the machine-learned model is trained using training data from a population of data storage systems,
wherein the population of data storage systems includes a first set of data storage systems that has failed and a second set of data storage systems that has not failed, and
wherein the training data comprises data related to the first set of data storage systems from a first time stress is applied to the first set of data storage systems until the first set of data storage systems has failed.
7. The method of claim 6 ,
wherein the machine-learned model is trained using a unseen data set,
wherein the unseen data set is data from a third set of data storage systems,
wherein the third set of data storage systems is different from the first set of data storage systems and the second set of data storage systems,
wherein the unseen data set is provided to the machine-learned model in a same order of time as the unseen data set is generated, and
wherein the unseen data set is provided to the machine-learned model from a first instance of time that the unseen data set is generated until a last instance of time that the unseen data set is generated.
8. A solid-state storage device, comprising:
a controller;
a non-volatile memory component;
the controller configured to:
receive a plurality of host-write commands;
calculate, based on the plurality of host-write commands, a number of drive-writes per day;
calculate, based on the number of drive-writes per day, an aggregated amount of degradation to the non-volatile memory component;
generate, using a machine-learned model, a probability of failure value based on data associated with a plurality of input parameters for the machine-learned model and the aggregated amount of degradation to the non-volatile memory component; and
generate, based on the probability of failure value, an alert,
wherein the machine-learned model is associated with the following operations:
identifying combinations of input parameter values of a machine-learned model, the combinations capable of resulting in a non-volatile memory component failure;
ranking a set of parameters of the machine-learned model between a high ranking and a low ranking, the ranking associated with the identifying;
selecting, from the set of parameters, the plurality of input parameters for the machine-learned model based on the ranking; and
generating or modifying a set of weights to apply to input parameter values based on the ranking,
wherein the set of parameters comprises the plurality of input parameters for the machine-learned model, and
wherein the data associated with the plurality of input parameters comprises:
non-volatile memory translation layer management events data;
controller initiated data movement for non-volatile memory endurance management event data;
non-volatile memory defect density data;
controller initiated data movement in non-volatile memory event data;
controller initiated data movement in non-volatile memory for managing defragmentation of the non-volatile memory event data;
non-volatile memory bit error rate data;
controller initiated non-volatile memory erase or rewrite operation management data; and
controller initiated event for non-volatile memory data movement and correction in response to errors data.
9. The solid-state storage device of claim 8 , wherein the alert is generated based on the probability of failure value satisfying a threshold probability of failure value.
10. The solid-state storage device of claim 8 , wherein one or more parameters of the plurality of input parameters comprise output data from a proactive firmware algorithm.
11. The solid-state storage device of claim 8 , wherein one or more parameters of the plurality of input parameters comprise output data from a reactive firmware algorithm.
12. The solid-state storage device of claim 8 , wherein the aggregated amount of degradation to the non-volatile memory component is based on an average drive-writes over a lifetime of the non-volatile memory component.
13. The solid-state storage device of claim 8 , wherein the aggregated amount of degradation to the non-volatile memory component is based on a throughput acceleration coefficient of the non-volatility memory component,
wherein the controller is configured to determine the throughput acceleration coefficient by dividing the number of drive-writes per day by a drive-writes per day specification,
wherein the controller is configured to determine the number of drive-writes per day based on the plurality of host-write commands, a total amount of data written to the non-volatile memory component, and a data storage capacity of the non-volatile memory component,
wherein the drive-writes per day specification is a pre-determined number of drive-writes per day for the non-volatile memory component to handle in order for the solid-state storage device to operate for a specific period of time, and
wherein the solid-state storage device is configured to store the drive-writes per day specification.
14. The solid-state storage device of claim 8 , wherein the machine-learned model is based on a random decision forest, and
wherein the machine-learned model is implemented as a kernel within the solid-state storage device.
15. The solid-state storage device of claim 8 ,
wherein the selecting comprises selecting the plurality of input parameters for the machine-learned model having the high ranking,
wherein the machine-learned model is associated with training using training data from a population of data storage systems,
wherein the population of data storage systems includes a first set of data storage systems that has failed and a second set of data storage systems that has not failed, and
wherein the training data comprises data related to the first set of data storage systems from a first time stress is applied to the first set of data storage systems until the first set of data storage systems has failed.
16. The solid-state storage device of claim 15 ,
wherein the machine-learned model is associated with training using a unseen data set,
wherein the unseen data set is data from a third set of data storage systems,
wherein the third set of data storage systems is different from the first set of data storage systems and the second set of data storage systems,
wherein when the unseen data set is provided to the machine-learned model, the unseen data set is provided in a same order of time as the unseen data set is generated, and
wherein when the unseen data set is provided to the machine-learned model, the unseen data set is provided from a first instance of time that the unseen data set is generated until a last instance of time that the unseen data set is generated.
17. A solid-state storage device, comprising:
a non-volatile memory component;
means for receiving a plurality of host-write commands at a solid-state storage device;
means for calculating, based on the plurality of host-write commands, a number of drive-writes per day;
means for calculating, based on the number of drive-writes per day, an aggregated amount of degradation to the non-volatile memory component;
means for generating, using a machine-learned model, a probability of failure value based on data associated with a plurality of input parameters for the machine-learned model and the aggregated amount of degradation to the non-volatile memory component; and
means for generating, based on the probability of failure value, an alert,
wherein the machine-learned model is associated with the following operations:
identifying combinations of input parameter values of a machine-learned model, the combinations capable of resulting in a non-volatile memory component failure;
ranking a set of parameters of the machine-learned model between a high ranking and a low ranking, the ranking associated with the identifying;
selecting, from the set of parameters, the plurality of input parameters for the machine-learned model based on the ranking; and
generating or modifying a set of weights to apply to input parameter values based on the ranking,
wherein the set of parameters comprises the plurality of input parameters for the machine-learned model,
wherein the aggregated amount of degradation to the non-volatile memory component comprises a throughput acceleration coefficient,
wherein the throughput acceleration coefficient is determined by dividing the number of drive-writes per day by a drive-writes per day specification,
wherein the number of drive-writes per day is determined based on the plurality of host-write commands, a total amount of data written to the non-volatile memory component, and a data storage capacity of the non-volatile memory component,
wherein the drive-writes per day specification is a pre-determined number of drive-writes per day for the non-volatile memory component to handle in order for the solid-state storage device to operate for a specific period of time, and
wherein the solid-state storage device is configured to store the drive-writes per day specification.
18. The solid-state storage device of claim 17 , wherein the machine-learned model is implemented as a module within a firmware of the solid-state storage device.
19. The solid-state storage device of claim 17 , wherein the data associated with the plurality of input parameters comprises:
non-volatile memory translation layer management events data;
controller initiated data movement for non-volatile memory endurance management event data;
non-volatile memory defect density data;
controller initiated data movement in non-volatile memory event data;
controller initiated data movement in non-volatile memory for managing defragmentation of the non-volatile memory event data;
non-volatile memory bit error rate data;
controller initiated non-volatile memory erase or rewrite operation management data; and
controller initiated event for non-volatile memory data movement and correction in response to errors data.
20. The solid-state storage device of claim 17 ,
wherein the selecting comprises selecting the plurality of input parameters for the machine-learned model having the high ranking,
wherein the machine-learned model is associated with training using training data from a population of data storage systems,
wherein the population of data storage systems includes a first set of data storage systems that has failed and a second set of data storage systems that has not failed, and
wherein the training data comprises data related to the first set of data storage systems from a first time stress is applied to the first set of data storage systems until the first set of data storage systems has failed.