Anomaly detection for predictive maintenance and deriving outcomes and workflows based on data quality
Systems, methods, and computer readable storage mediums for performing sensor health monitoring are described. The method includes verifying data quality and suppressing alert generation using machine learning techniques to identify whether two anomalies generated by an asset monitoring system are related. The method can include receiving data characterizing measurement data acquired by a sensor coupled to an industrial asset. An anomalous data sample within the received data can be identified and removed from the anomalous data sample. A new sample of the removed data sample can be estimated using interpolation and the new sample can be assessed. Maintenance analysis can be performed based on the assessed, estimated new sample.
1. A method comprising:
receiving data characterizing measurement data values acquired by a sensor coupled to a compressor;
identifying a first data sample within the received data, the first data sample including at least one measurement data value having an anomalous datatype, wherein identifying the first data sample includes applying at least one data validation rule to the received data, the at least one data validation rule configured to verify a rate of change in a mean value of the measurement data values and to generate an alert in response to the first data sample failing the data validation rule;
removing the first data sample;
estimating, using at least one data estimation technique, a second data sample to replace the first data sample, the at least one data estimation technique including performing gradient boosting on the removed first data sample; and
assessing the estimated second data sample by evaluating a fit between the estimated second data sample with respect to a dataset of known measurement data values, the fit determined based on applying an f-test to the second data sample and the dataset of known measurement data values wherein responsive to determining the fit of the estimated second data sample fails the f-test,
generating an alert including at least one tag identifying the failing test; with:
generating the alert including at least one tag identifying the failing test;
executing a maintenance analysis on the assessed, estimated second data sample based on the at least one tag; and
providing the alert and a result of the maintenance analysis for display, wherein the alert is displayed on a single time-series tag on a first axis with a first time stamp and the result of the maintenance analysis is displayed on a second axis, separate from the first axis, with a second time stamp, the first time stamp and the second time stamp being identical.
2. The method of claim 1 , wherein the sensor is affixed to the compressor in an oil and gas industrial environment and the received data further characterizes a state of health of the compressor.
3. The method of claim 2 , wherein the sensor is included in a sensor health monitoring system associated with the oil and gas industrial environment and the received data further characterizes a state of health of the sensor.
4. The method of claim 1 , wherein the at least one data validation rule is included in a plurality of data validation rules configured to verify a datatype of the one or more anomalous datatypes in the measurement data values, an empty data value within the measurement data values, a presence of repeating measurement data values, or an absence of three or more measurement data values.
5. The method of claim 1 , wherein the at least one data estimation technique is selected from a plurality of data estimation techniques including linear, cubic, spline, multiple imputation, k-nearest neighbors, or random forest data estimation.
6. The method of claim 1 , wherein the generation of alerts is suppressed based on evaluating the alert and the fit using one or more machine learning models trained in a machine learning process utilizing historical measurement data values associated with the compressor as training data for the one or more machine learning models.
7. The method of claim 6 , wherein the one or more machine learning models are further configured to generate a combined alert responsive to determining the fit of two or more estimated second data samples fail one or more tests or thresholds identified as failing previously, wherein the two or more estimated second data samples failing the one or more tests or thresholds include identical failure patterns.
8. The method of claim 6 , wherein the one or more machine learning models can be recalibrated and updated based on the fit of two or more estimated second data samples failing one or more tests or thresholds, wherein the training data includes false positive results of a maintenance analysis executed on the assessed, estimated second data.
9. The method of claim 1 , further comprising, modifying an operation of the compressor based on executing a maintenance analysis on the assessed, estimated second data sample.
10. A system comprising:
a data processor, a display, a controller, and a memory storing non-transitory, computer-readable instructions, which when executed cause the data processor to perform operations including
receiving data characterizing measurement data values acquired by a sensor communicatively coupled to a compressor, the sensor generating measurement data values,
identifying a first data sample within the received data, the first data sample including at least one measurement data value having an anomalous datatype, wherein identifying the first data sample includes applying at least one data validation rule to the received data, the at least one data validation rule configured to verify a rate of change in a mean value of the measurement data values and to generate an alert in response to the first data sample failing the data validation rule,
removing the first data sample,
estimating, using at least one data estimation technique, a second data sample to replace the first data sample, the at least one data estimation technique including performing gradient boosting on the removed first data sample, and
assessing the estimated second data sample by evaluating a fit between the estimated second data sample with respect to a dataset of known measurement data values, the fit determined based on applying an f-test to the second data sample and the dataset of known measurement data values wherein responsive to determining the fit of the estimated second data sample fails the f-test,
generating an alert including at least one tag identifying the failing test; with:
generating the alert including at least one tag identifying the failing test;
executing a maintenance analysis on the assessed, estimated second data sample based on the at least one tag, and
providing the alert and a result of the maintenance analysis for display, wherein the alert is displayed on a single time-series tag on a first axis with a first time stamp and the result of the maintenance analysis is displayed on a second axis, separate from the first axis, with a second time stamp, the first time stamp and the second time stamp being identical.
11. The system of claim 10 , wherein the sensor is affixed to the compressor in an oil and gas industrial environment and the received data further characterizes a state of health of the compressor.
12. The system of claim 11 , wherein the sensor is included in a sensor health monitoring system associated with the oil and gas industrial environment and the received data further characterizes a state of health of the sensor.
13. The system of claim 10 , wherein the at least one data validation rule is included in a plurality of data validation rules configured to verify a datatype of the measurement data values, an empty data value within the measurement data values, a presence of repeating measurement data values, or an absence of three or more measurement data values.
14. A non-transitory computer readable storage medium containing program instructions, which when executed by at least one data processor causes the at least one data processor to perform operations comprising:
receiving data characterizing measurement data values acquired by a sensor communicatively coupled to a compressor,
identifying a first data sample within the received data, the first data sample including at least one measurement data value having an anomalous datatype, wherein identifying the first data sample includes applying at least one data validation rule to the received data, the at least one data validation rule configured to verify a rate of change in a mean value of the measurement data values and to generate an alert in response to the first data sample failing the data validation rule,
removing the first data sample,
estimating, using at least one data estimation technique, a second data sample of the removed first data sample, the at least one data estimation technique including performing gradient boosting on the removed first data sample,
assessing the estimated second data sample by evaluating a fit between the estimated second data sample with respect to a dataset of known measurement data values, the fit determined based on applying an f-test to the second data sample and the dataset of known measurement data values wherein responsive to determining the fit of the estimated second data sample fails the f-test,
generating an alert including at least one tag identifying the failing test; with:
generating the alert including at least one tag identifying the failing test;
executing a maintenance analysis on the assessed, estimated second data sample based on the at least one tag,
providing the alert and a result of the maintenance analysis via a display, wherein the alert is displayed on a single time-series tag on a first axis with a first time stamp and the result of the maintenance analysis is displayed on a second axis, separate from the first axis, with a second time stamp, the first time stamp and the second time stamp being identical, and
controlling, via a controller coupled to the at least one data processor and the sensor, an operation of the compressor based on the result of the maintenance analysis.