Apparatus and method for detecting poor component life zones in a datacenter
An information handling system determines zones within a datacenter that exhibit poor component life. Components of the information handling system are organized into zones, and provide error indications and have error thresholds and warranty durations. A component life manager receives the error indications, fits the error indications to curves, determines end of life estimates based upon the curves and the error thresholds, determines end of life dates based upon the end of life estimates and dates that the first error indication is received, determines losses of life based on the end of life estimates and the warranty durations, determines component life scores based upon the losses of life and the warranty durations, determines zone life scores based upon an average of the component life scores, determines a datacenter life score based upon an average of the component life scores, and provides an indication when a zone life score is below the datacenter life score by more than a threshold.
1. An information handling system for determining zones within a datacenter that exhibit poor component life, the information handling system comprising:
a plurality of components, each component configured to provide error indications associated with aging of the component, each component having a predefined error threshold and a predefined warranty duration, and the components being organized into a plurality of zones of the datacenter; and
a component life manager configured to;
receive from each component the error indications;
fit, for each component, the received error indications to a curve;
determine, for each component, an end of life estimate based upon the curve and the error threshold;
determine, for each component, an end of life date based upon the end of life estimate and a date that the first error indication is received;
determine, for each component, a loss of life based on the end of life estimate and the warranty duration;
determine, for each component, a component life score for the component based upon the loss of life and the warranty duration;
determine, for each zone, a zone life score based upon an average of the component life scores of the components of the zone;
determine a datacenter life score based upon an average of the component life scores;
determine that the datacenter life score for a particular zone is correlated to an operating parameter of the particular zone;
provide an indication for the particular zone when the associated zone life score is below the datacenter life score by more than a threshold; and
change the operating parameter of the particular zone when the associated zone life score is below the datacenter life score by more than the threshold.
2. The information handling system of claim 1 , wherein in fitting the received error indications to the curve for each component, the component life manager performs a regression analysis to fit the received error to the form:
# CE=Ae Bt −C
where # CE represents a number of error indications received at a given time t, and where A, B, and C are terms that are derived via the regression analysis.
3. The information handling system of claim 2 , wherein the component life manager determines the end of life estimate for each component as:
t
EOL
=
ln
(
#
CE
Threshold
-
C
A
)
B
where t EOL is the end of life estimate and # CE Threshold represents the error threshold.
4. The information handling system of claim 3 , wherein the component life manager determines the end of life date for each component as:
t Fail =t Error +t EOL
where t Fail is the end of life date and t Error is a date a first error indication is received.
5. The information handling system of claim 4 , wherein the component life manager determines the loss of life for each component as:
t LOL =t Warranty −t EOL
where t LOL is the loss of life date and t Warranty is the warranty duration.
6. The information handling system of claim 5 , wherein the component life manager determines the component life score for each component as:
PLMS
=
100
-
t
LOL
+
100
t
Warranty
where PLMS is the component life score.
7. The information handling system of claim 1 , wherein the components include dual in-line memory modules (DIMMs), and the error indications include correctable errors on the DIMMs.
8. The information handling system of claim 1 , wherein the components include graphics processing units (GPUs), and the error indications include retired page counts on the GPUs.
9. The information handling system of claim 1 , wherein the components include data storage devices, and the error indications include uncorrectable errors on the data storage devices, uncorrectable sector count on the data storage devices, and reallocated sector counts on the data storage devices.
10. The information handling system of claim 1 , wherein the components include network interface cards (NICs), and the error indications include lost packet counts on the NICs.
11. A method for determining zones within a datacenter that exhibit poor component life, the zones each including a plurality of components, each component configured to provide error indications associated with aging of the component, and each component having a predefined error threshold and a predefined warranty duration, the method comprising:
receiving, by a component life manager, the error indications for each component;
fitting the received error indications to a curve for each component;
determining an end of life estimate for each component based upon the respective curve and the error threshold;
determining an end of life date for each component based upon the respective end of life estimate and a date that the first error indication is received for each component;
determining a loss of life for each component based on the respective end of life estimate and warranty duration;
determining a component life score for each component based upon the respective loss of life and warranty duration;
determining, for each zone, a zone life score based upon an average of the component life scores of the components of the zone;
determining a datacenter life score based upon an average of the component life scores;
determining that the datacenter life score for a particular zone is correlated to an operating parameter of the particular zone;
providing an indication for the particular zone when the associated zone life score is below the datacenter life score by more than a threshold; and
changing the operating parameter of the particular zone when the associated zone life score is below the datacenter life score by more than the threshold.
12. The method of claim 11 , wherein the curve for each component is of the form:
# CE=Ae Bt −C
where # CE represents a number of error indications received at a given time t, and where A, B, and C are terms that are derived via the regression analysis.
13. The method of claim 12 , wherein the life estimate is for each component of the form:
t
EOL
=
ln
(
#
CE
Threshold
-
C
A
)
B
where t EOL is the end of life estimate and # CE Threshold represents the error threshold.
14. The method of claim 13 , wherein the end of life date for each component is of the form:
t Fail =t Error +t EOL
where t Fail is the end of life date and t Error a date a first error indication is received.
15. The method of claim 14 , wherein the loss of life for each component is of the form:
t LOL =t Warranty −t EOL
where t LOL is the loss of life date and t Warranty is the warranty duration.
16. The method of claim 15 , wherein the component life score for each component is of the form:
PLMS
=
100
-
t
LOL
+
100
t
Warranty
where PLMS is the component life score.
17. The method of claim 11 , wherein the components include dual in-line memory modules (DIMMs), and the error indications include correctable errors on the DIMMs.
18. The method of claim 11 , wherein the components include graphics processing units (GPUs), and the error indications include retired page counts on the GPUs.
19. The method of claim 11 , wherein the components include data storage devices, and the error indications include uncorrectable errors on the data storage devices, uncorrectable sector counts on the data storage devices, and reallocated sector counts on the data storage devices.
20. A method for determining zones within a datacenter that exhibit poor graphics processing unit (GPU) life, the zones each including a plurality of GPUs, each GPU configured to provide a retired page count, and each GPU having a predefined error threshold and a predefined warranty duration, the method comprising:
receiving, by a GPU life manager, the retired page count for each GPU;
fitting the received retired page counts to a curve for each GPU, wherein the curve for each GPU is of the form:
# CE=Ae Bt −C
where # CE represents a number of retired page counts received at a given time t, and where A, B, and C are terms that are derived via regression analysis;
determining an end of life estimate for each GPU based upon the respective curve and the error threshold;
determining an end of life date for each GPU based upon the respective end of life estimate and a date that the first retired page count is received for each GPU;
determining a loss of life for each GPU based on the respective end of life estimate and warranty duration;
determining a GPU life score for each GPU based upon the respective loss of life and warranty duration;
determining, for each zone, a zone life score based upon an average of the GPU life scores of the GPUs of the zone;
determining a datacenter life score based upon an average of the GPU life scores;
determining that the GPU life score for a particular zone is correlated to an operating parameter of the particular zone;
providing an indication for a particular zone when the associated zone life score is below the datacenter life score by more than a threshold; and
changing the operating parameter of the particular zone when the associated zone life score is below the datacenter life score by more than the threshold.