Apparatus and method for troubleshooting poor part life zones in a datacenter
An information handling system determines zones within a datacenter that exhibit poor component life. Components of the information handling system provide error indications and have error thresholds and warranty durations. The components organized into zones. A component life manager receives the error indications, fits the error indications to curves, determines end of life estimates based upon the curves and the error thresholds, determines end of life dates based upon the end of life estimates and dates that the first error indication is received, determines losses of life based on the end of life estimates and the warranty durations, determines component life scores based upon the losses of life and the warranty durations, determines zone life scores based upon an average of the component life scores, determines a datacenter life score based upon an average of the component life scores, determines that a zone has a zone life score that is below the datacenter life score by more than a threshold, determines a condition of the zone is associated with the low zone life score, and changes the condition of the zone to improve the zone life score of the zone.
1. An information handling system, for determining a component life for zones within a datacenter, the information handling system comprising:
a plurality of components, each component configured to provide error indications associated with aging of the component, each component having a predefined error threshold and a predefined warranty duration, and the plurality of components being organized into a plurality of zones of the datacenter; and
a component life manager configured to;
receive from each component the error indications;
fit, for each component, a curve to the received error indications;
determine, for each component, an end of life estimate based upon the curve and the predefined error threshold;
determine, for each component, an end of life date based upon the end of life estimate and a date that a first error indication is received;
determine, for each component, a loss of life based on the end of life estimate and the warranty duration;
determine, for each component, a component life score for the component based upon the loss of life and the warranty duration;
determine, for each particular zone of the plurality of zones, a zone life score based upon an average of the component life scores for each of the components of the particular zone;
determine a datacenter life score based upon an average of the zone life scores for each of the zones;
determine that a particular zone has an associated zone life score that is below the datacenter life score by more than a threshold;
determine that a physical condition of the particular zone is associated with the low zone life score of the particular zone; and
change the physical condition of the particular zone to improve the zone life score of the particular zone.
2. The information handling system of claim 1 , wherein in fitting the curve for each component to the received error indications, the component life manager performs a regression analysis to fit the received error to the form:
# CE=Ae Bt −C
where #CE represents a number of error indications received at a given time t, and where A, B, and C are terms that are derived via the regression analysis.
3. The information handling system of claim 2 , wherein the component life manager determines the end of life estimate for each component as:
t
EOL
=
ln
(
#
CE
Threshold
-
C
A
)
B
where t EOL is the end of life estimate and #CE Threshold represents the predefined error threshold.
4. The information handling system of claim 3 , wherein the component life manager determines the end of life date for each component as:
t Fail =t Error +t EOL
where t Fail is the end of life date and t Error is a date the first error indication is received.
5. The information handling system of claim 4 , wherein the component life manager determines the loss of life for each component as:
t LOL =t Warranty −t EOL
where t LOL is the loss of life date and t Warranty is the warranty duration.
6. The information handling system of claim 5 , wherein the component life manager determines the component life score for each component as:
PLMS
=
100
-
t
LOL
+
100
t
Warranty
where PLMS is the component life score.
7. The information handling system of claim 1 , wherein the physical condition includes a temperature of the particular zone.
8. The information handling system of claim 1 , wherein the physical condition includes a utilization level of a particular component of the particular zone.
9. The information handling system of claim 8 , wherein the particular component includes on of a dual in-line memory module (DIMM), a graphics processing unit (GPU), a network interface card (NIC), and a storage device.
10. The information handling system of claim 1 , wherein the condition includes a workload operating on processor of the particular zone.
11. A method for determining a component life for zones within a datacenter, the zones each including a plurality of components, each component configured to provide error indications associated with aging of the component, and each component having a predefined error threshold and a predefined warranty duration, the method comprising:
receiving, by a component life manager, the error indications for each component;
fitting a curve for each component to the received error indications;
determining an end of life estimate for each component based upon the respective curve and the predefined error threshold;
determining an end of life date for each component based upon the respective end of life estimate and a date that a first error indication is received for each component;
determining a loss of life for each component based on the respective end of life estimate and warranty duration;
determining a component life score for each component based upon the respective loss of life and warranty duration;
determining, for each zone, a zone life score based upon an average of the component life scores of each of the components of the zone;
determining a datacenter life score based upon an average of the zone life scores for each of the zones;
determining that a particular zone has an associated zone life score that is below the datacenter life score by more than a threshold;
determining that a physical condition of the particular zone is associated with the low zone life score of the particular zone; and
changing the physical condition of the particular zone to improve the zone life score of the particular zone.
12. The method of claim 11 , wherein the curve for each component is of the form:
# CE=Ae Bt −C
where #CE represents a number of error indications received at a given time t, and where A, B, and C are terms that are derived via the regression analysis.
13. The method of claim 12 , wherein the life estimate is for each component of the form:
t
EOL
=
ln
(
#
CE
Threshold
-
C
A
)
B
where t EOL is the end of life estimate and #CE Threshold represents the predefined error threshold.
14. The method of claim 13 , wherein the end of life date for each component is of the form:
t Fail =t Error +t EOL
where t Fail is the end of life date and t Error is a date the first error indication is received.
15. The method of claim 14 , wherein the loss of life for each component is of the form:
t LOL =t Warranty −t EOL
where t LOL is the loss of life date and t Warranty is the warranty duration.
16. The method of claim 15 , wherein the component life score for each component is of the form:
PLMS
=
100
-
t
LOL
+
100
t
Warranty
where PLMS is the component life score.
17. The method of claim 11 , wherein the physical condition includes a temperature of the particular zone.
18. The method of claim 11 , wherein the physical condition includes a utilization level of a particular component of the particular zone.
19. The method of claim 11 , wherein the condition includes a workload operating on processor of the particular zone.
20. A method for determining graphics processing unit (GPU) life for zones within a datacenter, the zones each including a plurality of GPUs, each GPU configured to provide a retired page count, and each GPU having a predefined error threshold and a predefined warranty duration, the method comprising:
receiving, by a GPU life manager, the retired page counts for each GPU;
fitting a curve for each GPU to the received retired page counts, wherein the curve for each GPU is of the form:
# CE=Ae Bt −C
where #CE represents a number of retired page counts received at a given time t, and where A, B, and C are terms that are derived via a regression analysis;
determining an end of life estimate for each GPU based upon the respective curve and the predefined error threshold;
determining an end of life date for each GPU based upon the respective end of life estimate and a date that the first retired page count is received for each GPU;
determining a loss of life for each GPU based on the respective end of life estimate and warranty duration;
determining a GPU life score for each GPU based upon the respective loss of life and warranty duration;
determining, for each zone, a zone life score based upon an average of the GPU life scores of the GPUs of the zone;
determining a datacenter life score based upon an average of the GPU life scores;
determining that a particular zone has an associated zone life score that is below the datacenter life score by more than a threshold;
determining that a physical condition of the particular zone is associated with the low zone life score of the particular zone; and
changing the physical condition of the particular zone to improve the zone life score of the particular zone.