System and method for detecting and filtering harmful and incorrect data from telemetry streams

1 . A computer-implemented method, executed on a computing device, comprising:

receiving a telemetry data stream associated with a storage system;

identifying an erroneous portion of the telemetry data stream by processing the telemetry data stream using an error detection process, wherein identifying the erroneous portion of the telemetry data stream includes:

processing the telemetry data stream using rule-based error detection;

processing the telemetry data stream using quantitative analysis error detection in response to processing the telemetry data stream using the rule-based error detection; and

processing the telemetry data stream using a plurality of adaptive thresholds generated using a polynomial regression model in response to processing the telemetry data stream using the quantitative analysis error detection; and

filtering the erroneous portion of the telemetry data stream by removing the erroneous portion of the telemetry data stream from the telemetry data stream.

2 . The computer-implemented method of claim 1 , wherein processing the telemetry data stream using quantitative analysis error detection includes flagging a candidate erroneous portion from the telemetry data stream.

3 . The computer-implemented method of claim 1 , further comprising:

performing a remedial action on the storage system in response to identifying the erroneous portion of the telemetry data stream.

4 . The computer-implemented method of claim 1 , further comprising:

training a machine learning model with a filtered portion of the telemetry data stream.

5 . A computer program product comprising a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:

receiving a telemetry data stream associated with a storage system;

identifying an erroneous portion of the telemetry data stream by processing the telemetry data stream using an error detection process, wherein identifying the erroneous portion of the telemetry data stream includes:

processing the telemetry data stream using rule-based error detection;

processing the telemetry data stream using quantitative analysis error detection in response to processing the telemetry data stream using the rule-based error detection; and

processing the telemetry data stream using a plurality of adaptive thresholds generated using a polynomial regression model in response to processing the telemetry data stream using the quantitative analysis error detection; and

filtering the erroneous portion of the telemetry data stream by removing the erroneous portion of the telemetry data stream from the telemetry data stream.

6 . The computer program product of claim 5 , wherein processing the telemetry data stream using quantitative analysis error detection includes flagging a candidate erroneous portion from the telemetry data stream.

7 . The computer program product of claim 5 , wherein the operations further comprise:

performing a remedial action on the storage system in response to identifying the erroneous portion of the telemetry data stream.

8 . The computer program product of claim 5 , wherein the operations further comprise:

training a machine learning model with a filtered portion of the telemetry data stream.

9 . A computing system comprising:

a memory; and

a processor configured to:

receive a telemetry data stream associated with a storage system, identify an erroneous portion of the telemetry data stream by processing the telemetry data stream using an error detection process, wherein identifying the erroneous portion of the telemetry data stream includes:

processing the telemetry data stream using rule-based error detection;

processing the telemetry data stream using quantitative analysis error detection in response to processing the telemetry data stream using the rule-based error detection; and

processing the telemetry data stream using a plurality of adaptive thresholds generated using a polynomial regression model in response to processing the telemetry data stream using the quantitative analysis error detection, and

filter the erroneous portion of the telemetry data stream by removing the erroneous portion of the telemetry data stream from the telemetry data stream.

10 . The computing system of claim 9 , wherein the processor is further configured to:

perform a remedial action on the storage system in response to identifying the erroneous portion of the telemetry data stream.

11 . The computing system of claim 9 , wherein the processor is further configured to:

train a machine learning model with a filtered portion of the telemetry data stream.