Method and apparatus for optimizing profit in predictive systems
Techniques are disclosed for methods and apparatuses for forming for determining when to perform maintenance events. The technique comprises determining a first cost of false positives and a second cost of missed true positives. A Receive Operating Characteristic (ROC) of a prediction model is determined for the occurrence of an event. A survival function and prediction horizon is generated from the prediction model for the occurrence of an event. The operational area on the ROC is determined based on the first costs and second costs. A threshold is determined from the ROC and is applied to the survival function and prediction horizon. A maintenance event is triggered based on the threshold.
1. A computer implemented method for determining when to perform maintenance events, the method comprising:
determining a first cost of false positives and a second cost of missed true positives;
determining a Receive Operating Characteristic (ROC) of a prediction model for an occurrence of one or more maintenance events;
generating a survival function from the prediction model for the occurrence of an event;
determining an operational area on the ROC based on the first costs and second costs;
determining a threshold from the ROC, the operational area and an optimal point and applying the threshold to the survival function and prediction horizon; and
triggering a maintenance event based on the threshold.
2. The computer method of claim 1 , wherein the prediction model is determined for a prediction of a failure associated with the one or more maintenance events.
3. The computer method of claim 2 , further comprising:
determining a smoothed ROC, formed by applying a smoothing function to the ROC, that has a first derivative that is continuous and monotonically decreasing, and wherein the optimal point is determined on the ROC based on the first cost, the second cost, a cost ratio and a derivative of the smoothed ROC.
4. The computer method of claim 3 , wherein the prediction horizon, the optimal point and the threshold are derived utilizing an expected value of a predictor in a future and an indication of a range of possible values in the future along with a probability of these values.
5. The computer method of claim 3 , further comprising:
determining an uncertainty for the occurrence of the maintenance events.
6. The computer method of claim 5 , further comprising:
adjusting the threshold based on a cost analysis between an unscheduled occurrence of the event and a scheduled occurrence of the event.
7. The computer method of claim 6 , wherein the cost analysis comprises:
calculating a ratio of the first cost divided by the second cost.
8. The computer method of claim 7 , wherein the first cost is a $NWDT cost associated with nothing wrong down time and the second cost is a difference of a $UDT cost associated with the unscheduled occurrence of the event and a $SDT cost associated with the scheduled occurrence of the event.
9. The computer method of claim 7 , further comprising:
determining a point along the survival function for events in a future where the ratio is substantially equal to a first derivative of the smoothed ROC; and
setting the trigger to said point.
10. The computer method of claim 1 further comprising:
utilizing fault detection to determine a current state in the survival function; and
modifying the trigger based on the current state.
11. A computer-readable storage medium storing a program, which, when executed by a processor performs an operation for determining when to perform maintenance events, the operation comprising:
determining a first cost of false positives and a second cost of missed true positives;
determining a Receive Operating Characteristic (ROC) of a prediction model for an occurrence of one or more maintenance events;
generating a survival function from the prediction model for the occurrence of an event;
determining an operational area on the ROC based on the first costs and second costs;
determining a threshold from the ROC, the operational area and an optimal point and applying the threshold to the survival function and prediction horizon; and
triggering a maintenance event based on the threshold.
12. The computer-readable storage medium of claim 11 , wherein the prediction model is determined for a prediction of a failure associated with the one or more maintenance events.
13. The computer-readable storage medium of claim 12 , further comprising:
determining a smoothed ROC, formed by applying a smoothing function to the ROC, that has a first derivative that is continuous and monotonically decreasing, and wherein the optimal point is determined on the ROC based on the first cost, the second cost, a cost ratio and a derivative of the smoothed ROC.
14. The computer-readable storage medium of claim 13 , wherein the prediction horizon, the optimal point and the threshold are derived utilizing an expected value of a predictor in a future and an indication of a range of possible values in the future along with a probability of these values.
15. The computer-readable storage medium of claim 13 , further comprising:
determining an uncertainty for the occurrence of the maintenance events.
16. The computer-readable storage medium of claim 15 , further comprising:
adjusting the threshold based on a cost analysis between an unscheduled occurrence of the event and a scheduled occurrence of the event.
17. The computer-readable storage medium of claim 16 , wherein the cost analysis comprises:
calculating a ratio of the first cost divided by the second cost.
18. The computer-readable storage medium of claim 17 , wherein the first cost is a $NWDT cost associated with nothing wrong down time and the second cost is a difference of a $UDT cost associated with the unscheduled occurrence of the event and a $SDT cost associated with the scheduled occurrence of the event.
19. The computer-readable storage medium of claim 17 , further comprising:
determining a point along the survival function for events in a future where the ratio is substantially equal to a first derivative of the smoothed ROC; and
setting the trigger to said point.
20. The computer-readable storage medium of claim 11 further comprising:
utilizing fault detection to determine a current state in the survival function; and
modifying the trigger based on the current state.