Database management system
A database management system and a method for redundant storage of time-series data in a distributed data storage system with a plurality of storage media is disclosed. The method comprises dividing the time-series data into a plurality of data blocks, storing the plurality of data blocks on ones of the plurality of storage media, and redundantly storing a subset of time series data, which is contiguous in time to one of the plurality of data blocks, on the same one of the plurality of storage media as the contiguous one of the plurality of data blocks.
1. A method for redundant storage of time-series data in a distributed data storage system with a plurality of storage media, the method comprising:
dividing the time-series data into a plurality of data blocks;
storing the plurality of data blocks on ones of the plurality of storage media; and
redundantly storing a subset of time series data, which is contiguous in time to one of the plurality of data blocks, on the same one of the plurality of storage media as the contiguous one of the plurality of data blocks.
2. The method of claim 1 , wherein the data blocks are of the same size.
3. The method of claim 1 , wherein the data blocks are collated into zones of a fixed time period, and wherein all of the data blocks of a single one of the zones are stored on the same one of the plurality of storage media.
4. The method of claim 1 , wherein the time-series data is production data collected from a plurality of sensors.
5. The method of claim 1 , further comprising dynamically changing the size of one or more of the data blocks or the zones based on at least one of executed queries or stored time-series data.
6. The method of claim 5 , further comprising changing storage location of the time-series data after changing the size of at least one of the data blocks, the zones, or the subset of time series data.
7. The method of claim 1 , further comprising adjusting the data partitioning and distribution of data over the cluster in response to parameters to optimize query answer time, wherein the parameters are the size of a block, the size of a zone, and the size of a boundary copy.
8. The method of claim 1 , further comprising putting data blocks in zones, and co-locating data blocks and zones on time for operating on large time periods.
9. A method for accessing required time-series data from a distributed data storage system with a plurality of memory elements, wherein the time-series data is divided into a plurality of data blocks, the method comprising:
executing a query requiring access to the time-series data;
selecting the data blocks with the required time-series data and identifying the ones of the plurality of storage media on which the selected ones of the data blocks are stored; and
accessing at least one of the identified ones of the plurality of storage media to retrieve the selected ones of the data blocks and also retrieve the time series data, which is contiguous in time, to the retrieved data block from the same one of the plurality of memory elements.
10. The method of claim 9 , wherein the executing of the query further comprises dividing the query into a plurality of sub-queries for transmission to different ones of the plurality of storage media for selection of the required time-series data.
11. The method of claim 9 , wherein the data blocks are collated into zones of a fixed time period, and wherein all of the data blocks of a single one of the zones are stored on the same one of the plurality of storage media.
12. The method of claim 9 , further comprising dynamically changing the size of one or more of the data blocks, the subset of time series data, or the zones based on executed queries.
13. A database management system comprising:
a plurality of storage media for storing time-series data in a plurality of data blocks;
a processor connected to the plurality of storage media through a network; and
a local storage medium connected to the processor, wherein the local storage medium stores at least one of the plurality of data blocks for accessing by the processor and redundantly stores a subset of time series data, which is contiguous in time to the stored one of the plurality of data blocks on the local storage medium.
14. The database management system of claim 13 , further comprising a query executor for executing a query and transmitting sub-queries to different ones of the plurality of storage media in the network.
15. The database management system of claim 13 , wherein the plurality of data blocks are collected as zones, the zones having a defined time length, and wherein all of the data blocks from a single one of the zones are stored on a single one of the plurality of storage media.
16. The database management system of claim 13 , further comprising a statistics collector for processing statistics about data accessed by executed queries and adjusting the sizes of at least one of the data blocks, the subset of time series data, or the zones.
17. The database management system of claim 13 , further comprising a statistics collector for processing statistics about newly stored time-series data and adjusting the sizes of at least one of the data blocks, the subset of time series data or the zones.
18. The database management system of claim 13 , further comprising an auto-optimizing data partitioning system for adjusting the data partitioning and distribution of data over the cluster in response to parameters to optimize query answer time.