Technique for validating a prognostic-surveillance mechanism in an enterprise computer system
The disclosed embodiments relate to a system for validating a prognostic-surveillance mechanism, which detects anomalies that arise during operation of a computer system. During operation, the system obtains telemetry data comprising a set of raw signals gathered from sensors in the computer system during operation of the computer system, wherein the telemetry signals are gathered over a monitored time period. Next, for each raw signal in the set of raw signals, the system decomposes the raw signal into deterministic and stochastic components. The system then generates a corresponding set of synthesized signals based on the deterministic and stochastic components of the raw signals, wherein the synthesized signals are generated for a simulated time period, which is longer than the monitored time period. Finally, the system uses the set of synthesized signals to validate one or more performance metrics of the prognostic-surveillance mechanism.
1. A method for validating a prognostic-surveillance mechanism for detecting anomalies that arise during operation of a computer system using synthesized signals, comprising:
obtaining, by a monitoring mechanism of a service processor, telemetry data comprising a set of raw signals gathered from sensors in the computer system during operation of the computer system, wherein the telemetry signals are gathered over a monitored time period;
for each raw signal in the set of raw signals,
decomposing the raw signal into deterministic and stochastic components, and
generating a corresponding synthesized signal in a set of synthesized signals based on the deterministic and stochastic components of the raw signal, wherein the synthesized signal is generated for a simulated time period, which is longer than the monitored time period by at least two orders of magnitude; and
using the set of synthesized signals to validate one or more performance metrics of the prognostic-surveillance mechanism.
2. The method of claim 1 , wherein validating the one or more performance metrics of the prognostic-surveillance mechanism comprises validating whether the prognostic-surveillance mechanism generates an alarm associated with one or more of the following:
a malicious intrusion event; and
an impending failure.
3. The method of claim 1 , wherein validating the one or more performance metrics of the prognostic-surveillance mechanism comprises validating that the prognostic-surveillance mechanism meets one or more of the following:
a desired false alarm probability (FAP); and
a desired missed alarm probability (MAP).
4. The method of claim 1 , wherein each synthesized signal in the set of synthesized signals has the same serial-correlation structure and the same stochastic structure as a corresponding raw signal in the set of raw signals.
5. The method of claim 1 , wherein decomposing the raw signal comprises using an iterative Fourier decomposition technique to decompose the raw signal into deterministic components that comprise an envelope of superimposed sine waves and cosine waves.
6. The method of claim 5 , wherein decomposing the raw signal further comprises subtracting the deterministic components from the raw signal to generate a residual signal that comprises the stochastic components of the raw signal, wherein the stochastic components define a stationary stochastic process.
7. The method of claim 1 , wherein generating the synthesized signal based on the deterministic and stochastic components comprises:
generating a deterministic signal based on the deterministic components;
generating a stochastic signal by simulating a stationary stochastic process defined by the stochastic components; and
superimposing the stochastic signal over the deterministic signal to produce the synthesized signal.
8. The method of claim 1 , wherein a stationary stochastic process that defines the stochastic components comprises one or more of the following:
a Gaussian process;
a Poisson process; and
a uniform random process.
9. The method of claim 1 , wherein a stationary stochastic process that defines the stochastic components generates a probability distribution defined by one or more of the following parameters:
a mean;
a variance;
a skewness; and
a kurtosis.
10. The method of claim 1 , wherein the prognostic-surveillance mechanism uses one or more models that are trained using a nonlinear, nonparametric (NLNP) regression technique to predict signal values.
11. The method of claim 1 , wherein the set of raw signals includes:
signals gathered from physical sensors in the computer system that monitor physical performance parameters, including one or more of temperature, voltage, current, vibration, and acoustic parameters; and
signals that carry software-related telemetry parameters, including one or more of processor load, memory usage, cache usage, system throughput, queue lengths, I/O traffic, and quality of service (QoS).
12. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for validating a prognostic-surveillance mechanism for detecting anomalies that arise during operation of a computer system using synthesized signals, the method comprising:
obtaining telemetry data comprising a set of raw signals gathered from sensors in the computer system during operation of the computer system, wherein the telemetry signals are gathered over a monitored time period;
for each raw signal in the set of raw signals,
decomposing the raw signal into deterministic and stochastic components, and
generating a corresponding synthesized signal in a set of synthesized signals based on the deterministic and stochastic components of the raw signal, wherein the synthesized signal is generated for a simulated time period, which is longer than the monitored time period by at least two orders of magnitude; and
using the set of synthesized signals to validate one or more performance metrics of the prognostic-surveillance mechanism.
13. The non-transitory computer-readable storage medium of claim 12 , wherein validating the one or more performance metrics of the prognostic-surveillance mechanism comprises validating whether the prognostic-surveillance mechanism generates an alarm associated with one or more of the following:
a malicious intrusion event; and
an impending failure.
14. The non-transitory computer-readable storage medium of claim 12 , wherein validating the one or more performance metrics of the prognostic-surveillance mechanism comprises validating that the prognostic-surveillance mechanism meets one or more of the following:
a desired false alarm probability (FAP); and
a desired missed alarm probability (MAP).
15. The non-transitory computer-readable storage medium of claim 12 , wherein each synthesized signal in the set of synthesized signals has the same serial-correlation structure and the same stochastic structure as a corresponding raw signal in the set of raw signals.
16. The non-transitory computer-readable storage medium of claim 12 , wherein decomposing the raw signal comprises using an iterative Fourier decomposition technique to decompose the raw signal into deterministic components that comprise an envelope of superimposed sine waves and cosine waves.
17. The non-transitory computer-readable storage medium of claim 16 , wherein decomposing the raw signal further comprises subtracting the deterministic components from the raw signal to generate a residual signal that comprises the stochastic components of the raw signal, wherein the stochastic components define a stationary stochastic process.
18. The non-transitory computer-readable storage medium of claim 12 , wherein generating the synthesized signal based on the deterministic and stochastic components comprises:
generating a deterministic signal based on the deterministic components;
generating a stochastic signal by simulating a stationary stochastic process defined by the stochastic components; and
superimposing the stochastic signal over the deterministic signal to produce the synthesized signal.
19. The non-transitory computer-readable storage medium of claim 12 , wherein the prognostic-surveillance mechanism uses one or more models that are trained using a nonlinear, nonparametric (NLNP) regression technique to predict signal values.
20. A system that validates a prognostic-surveillance mechanism for detecting anomalies that arise during operation of a computer system using synthesized signals, comprising:
a service processor, comprising a monitoring mechanism and a validation mechanism that,
obtains telemetry data comprising a set of raw signals gathered from sensors in the computer system during operation of the computer system, wherein the telemetry signals are gathered over a monitored time period;
for each raw signal in the set of raw signals,
decomposes the raw signal into deterministic and stochastic components, and
generates a corresponding synthesized signal in the set of synthesized signals based on the deterministic and stochastic components of the raw signal wherein the synthesized signal is generated for a simulated time period, which is longer than the monitored time period by at least two orders of magnitude; and
uses the set of synthesized signals to validate one or more performance metrics of the prognostic-surveillance mechanism.