System and method for applying deadband filtering to time series data streams to be stored within an industrial process manufacturing/production database
View Patent ↗A control system database server is disclosed that includes a data compression facility that implements a compression test sequence on streams of time stamped data points rendered by components of a control system. A deadband/time override is incorporated into the compression test sequence to ensure that a previously received data point, within the stream of received data points, is saved within a designated time period even if the compression test sequence would have determined that the data point be discarded. The series of compressed/tabled data points, corresponding to the received steams of data points, are thereafter provided by the historian to requesting client applications that, by way of example, retrieve the compressed, time stamped data point streams for the purpose of constructing and displaying trend graphs.
1. A database server compression and storage facility that operates upon a stream of data points received from a data source in a control environment, to reduce the volume of data points that are stored in a database, wherein each data point specifies a value and timestamp, by discarding received data points that do not significantly depart from a signal trend established by a sequence of received data points from the streaming time stamped data source, a database server compression and storage facility comprising:
a database a compression test sequence applied, upon receiving a new data point, to at least the received new data point, a held over data point and a last data point stored in the database; and
a time override that forces storing a candidate data point without regard to whether the candidate point would be stored as a result of applying the compression test sequence, upon receiving the new data points;
wherein the candidate data point is the held over data point; and
wherein the time override further comprises a real-time window timer that measures an elapsed period after the time stamp of the held over point and stores the held over data point in the database in response to a specified period, measured by the real-time window timer, expiring.
2. The data compression and storage facility of claim 1 wherein the time override comprises an operation that determines an elapsed time between the last stored data point and the new data point and stores the held over data point in the database if the elapsed time exceeds a specified override time period.
3. The data compression and storage facility of claim 1 wherein the compression test sequence comprises a value deadband test that is applied to the held over data point and the last stored data point.
4. The data compression and storage facility of claim 1 wherein the compression test sequence comprises a rate deadband test that is applied to at least the last stored data point, the held over data point, and the new data point.
5. The data compression and storage facility of claim 4 wherein the rate deadband test comprises determining a change in slope between a first line segment and a second line segment, wherein the first line segment includes at least the last stored data point, and the second line segment includes the new data point and the held over data point.
6. The data compression and storage facility of claim 5 wherein the first line segment further comprises the held over data point.
7. The data compression and storage facility of claim 1 further comprising a quality change test wherein the held over data point is stored if a quality assigned to the held over data point differs from the quality assigned to the last stored data point.
8. In a control environment, a data compression method for reducing the volume of time stamped data points, received from a streaming data source, that are stored in a database, by discarding received data points that do not significantly depart from a signal trend established by a sequence of received data points from the streaming data source, the method comprising:
providing a compression test sequence that is applied, upon receiving a new data point, to at least the received data point, a held over data point and a last data point stored in the database;
receiving the new data point after the held over data point and the last stored data point, and in response performing the further steps of:
executing the compression test sequence; and
applying a time override to force storage of a candidate data point that would otherwise be discarded as a result of the executing step;
wherein the candidate data point is the held over data point; and
wherein the applying step comprises measuring, by a real-time window timer, an elapsed period after the time stamp of the held over point and storing the held over data point in the database in response to a specified period, measured by the real-time window timer, expiring.
9. The data compression method of claim 8 wherein the applying step comprises determining an elapsed time between the last stored data point and the new data point and storing the held over data point in the database if the elapsed time exceeds a specified override time period.
10. The data compression method of claim 8 wherein the executing the compression test sequence step comprises performing a value deadband test that is applied to the held over data point and the last stored data point.
11. The data compression method of claim 8 wherein the executing the compression test sequence step comprises performing a rate deadband test that is applied to at least the last stored data point, the held over data point, and the new data point.
12. The data compression method of claim 11 wherein the rate deadband test comprises determining a change in slope between a first line segment and a second line segment, wherein the first line segment includes at least the last stored data point, and the second line segment includes the new data point and the held over data point.
13. The data compression method of claim 12 wherein the first line segment further comprises the held over data point.
14. A computer-readable medium containing computer-executable instructions for facilitating performing, in a control environment, a data compression method for reducing the volume of time stamped data points, received from a streaming data source, that are stored in a database, by discarding received data points that do not significantly depart from a signal trend established by a sequence of received data points from the streaming data source, the computer-executable instructions facilitating performing the method comprising:
providing a compression test sequence that is applied, upon receiving a new data point, to at least the received data point, a held over data point and a last data point stored in the database;
receiving the new data point after the held over data point and the last stored data point, and in response performing the further steps of:
executing the compression test sequence; and
applying a time override to force storage of a candidate data point that would otherwise be discarded as a result of the executing step;
wherein the candidate data point is the held over data point; and
wherein the applying step comprises measuring, by a real-time window timer, an elapsed period after the time stamp of the held over point and storing the held over data point in the database in response to a specified period, measured by the real-time window timer, expiring.
15. The computer-readable medium of claim 14 wherein the applying step comprises determining an elapsed time between the last stored data point and the new data point and storing the held over data point in the database if the elapsed time exceeds a specified override time period.
16. The computer-readable medium of claim 14 wherein the executing the compression test sequence step comprises performing a value deadband test that is applied to the held over data point and the last stored data point.
17. The computer-readable medium of claim 14 wherein the executing the compression test sequence step comprises performing a rate deadband test that is applied to at least the last stored data point, the held over data point, and the new data point.
18. The computer-readable medium of claim 17 wherein the rate deadband test comprises determining a change in slope between a first line segment and a second line segment, wherein the first line segment includes at least the last stored data point, and the second line segment includes the new data point and the held over data point.
19. The computer-readable medium of claim 18 wherein the first line segment further comprises the held over data point.