Linear run length encoding: compressing the index vector
A system and method include storing a table of time series data in a database of a data platform, the table of time series data representing a set of time series blocks. Each time series block of the set of time series blocks has a time series of equally-incremented time intervals and a run length. Each time interval of the time series is associated with one or more values. The run length has a starting position with at least one starting value and an ending position with at least one ending value. The starting position and the at least one starting value is stored for each time series block in a column store of the database. Then, a compressed index is generated in the column store of the database for each time series block, the compressed index comprising the starting position and the at least one starting value.
1. A computer-implemented method comprising:
storing a table of time series data in a database of a data platform, the table of time series data representing a plurality of time series blocks, each time series block of the plurality of time series blocks including a time series of equally-incremented time intervals;
generating a two-column compressed index for a column of the database for each time series block, for each time series block, the two-column compressed index comprising a first column and a second column, the first column comprising a starting position of a corresponding time series block and the second column comprising a corresponding starting value at the starting position of the corresponding time series block;
accessing, in response to a query to the data platform, the two-column compressed index to obtain the starting position and the starting value;
determining a time stamp, the determining based at least on the accessed starting position and the accessed starting value; and
executing the query based at least on the determined timestamp value.
2. The computer-implemented method in accordance with claim 1 , wherein the plurality of time series blocks are each separated by a gap in time that is different from the equally-incremented time intervals, wherein each time series block of the plurality of time series blocks further includes the starting position associated with a starting timestamp value and an ending position associated with an ending timestamp value, and wherein the computer-implemented method further comprises:
calculating, based on the two-column compressed index, each time series block of the plurality of time series blocks.
3. The computer-implemented method in accordance with claim 2 , further comprising calculating a run length associated with each time series block based on the equally-incremented time intervals.
4. The computer-implemented method in accordance with claim 3 , wherein calculation of the run length is further based on one or more values associated with each time interval of the time series.
5. The computer-implemented method in accordance with claim 1 , wherein the database includes an in-memory database having a hybrid store structure, the hybrid store structure comprising the column store and a row store.
6. The computer-implemented method in accordance with claim 1 , further comprising:
storing, in a dictionary, the starting timestamp value, the time interval separating the plurality of timestamp values in each equidistant time series, and a function for determining, based on the starting timestamp and the time interval, one or more of the plurality of timestamp values in each equidistant time series for the column of the table.
7. A computer program product comprising a machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
storing a table of time series data in a database of a data platform, the table of time series data representing a plurality of time series blocks, each time series block of the plurality of time series blocks including a time series of equally-incremented time intervals.
generating a two-column compressed index for a column of the database for each time series block, for each time series block, the two-column compressed index comprising a first column and a second column, the first column comprising a starting position of a corresponding time series block and the second column comprising a corresponding starting value at the starting position of the corresponding time series block;
accessing, in response to a query to the data platform, the two-column compressed index to obtain the starting position and the starting value;
determining a time stamp, the determining based at least on the accessed starting position and the accessed starting value; and
executing the query based at least on the determined timestamp value.
8. The computer program product in accordance with claim 7 , wherein the plurality of time series blocks are each separated by a gap in time that is different from the equally-incremented time intervals, wherein each time series block of the plurality of time series blocks further includes the starting position associated with a starting timestamp value and an ending position associated with an ending timestamp value, and wherein the operations further comprise calculating, based on the two-column compressed index, each time series block of the plurality of time series blocks.
9. The computer program product in accordance with claim 8 , wherein the operations further comprise calculating a run length associated with each time series block based on the equally-incremented time intervals.
10. The computer program product in accordance with claim 9 , wherein calculation of the run length is further based on one or more values associated with each time interval of the time series.
11. The computer program product in accordance with claim 7 , wherein the database includes an in-memory database having a hybrid store structure, the hybrid store structure comprising the column store and a row store.
12. The computer program product in accordance with claim 7 , further comprising:
storing, in a dictionary, the starting timestamp value, the time interval separating the plurality of timestamp values in each equidistant time series, and a function for determining, based on the starting timestamp and the time interval, one or more of the plurality of timestamp values in each equidistant time series for the column of the table.
13. A system comprising:
at least one processor; and
at least one memory including program code which when executed by the at least one processor causes operations comprising:
storing a table of time series data in a database of a data platform, the table of time series data representing a plurality of time series blocks, each time series block of the plurality of time series blocks including a time series of equally-incremented time intervals;
generating a two-column compressed index for a column of the database for each time series block, for each time series block, the two-column compressed index comprising a first column and a second column, the first column comprising a starting position of a corresponding time series block and the second column comprising a corresponding starting value at the starting position of the corresponding time series block;
accessing, in response to a query to the data platform, the two-column compressed index to obtain the starting position and the starting value;
determining a time stamp, the determining based at least on the accessed starting position and the accessed starting value; and
executing the query based at least on the determined timestamp value.
14. The system in accordance with claim 13 , wherein the plurality of time series blocks are each separated by a gap in time that is different from the equally-incremented time intervals, wherein each time series block of the plurality of time series blocks further includes the starting position associated with a starting timestamp value and an ending position associated with an ending timestamp value, and wherein the operations further comprise calculating, based on the two-column compressed index, each time series block of the plurality of time series blocks.
15. The system in accordance with claim 14 , wherein the operations further comprise calculating a run length associated with each time series block based on the equally-incremented time intervals.
16. The system in accordance with claim 15 , wherein calculation of the run length is further based on one or more values associated with each time interval of the time series.
17. The system in accordance with claim 13 , wherein the database includes an in-memory database having a hybrid store structure, the hybrid store structure comprising the column store and a row store.
18. The system in accordance with claim 13 , further comprising:
storing, in a dictionary, the starting timestamp value, the time interval separating the plurality of timestamp values in each equidistant time series, and a function for determining, based on the starting timestamp and the time interval, one or more of the plurality of timestamp values in each equidistant time series for the column of the table.