Multivariate memory vectorization technique to facilitate intelligent caching in time-series databases

1. A method for caching time-series data in a time-series database system, comprising:

receiving the time-series data, wherein the time-series data comprises a series of observations obtained from sensor readings for each signal in a set of signals;

performing a multivariate memory vectorization (MMV) operation on the time-series data, which selects a subset of observations in the time-series data that represents an underlying structure of the time-series data for individual and multivariate signals that comprise the time-series data, wherein performing the MMV operation on the time-series data comprises:

scaling observations for each signal in the time-series data using a mean center, unit variance technique,

dividing observations in the scaled time-series data into a set of windows for consecutive time intervals,

for each window in the set of windows, selecting observations that include minimum and maximum values for each signal in the set of signals,

for all remaining observations in each window, computing a Euclidean norm,

sorting the remaining observations in each window based on the Euclidean norms,

uniformly selecting a subset of the remaining observations based on the sorted Euclidean norms, and

unscaling all selected observations; and

while subsequently processing a query involving the time-series data,

caching the selected subset of the time-series data in an in-memory database cache in the time-series database system, and

accessing the selected subset of the time-series data from the in-memory database cache.

2. The method of claim 1 , wherein the MMV operation selects the subset of observations that represents the underlying structure of the time-series data with respect to one or more of the following: serial correlation, cross correlation, seasonality and stochastic structure.

3. The method of claim 1 ,

wherein after performing the multivariate memory vectorization operation on the time-series data, the method further comprises performing a geometric aging compression (GAC) operation on the time-series data;

wherein the GAC operation compresses observations by averaging pairs of consecutive observations; and

wherein the GAC operation:

does not compress a most-recent window of observations;

compresses a next-most recent window of observations once;

compresses a third-most recent window of observations twice; and

compresses an n th -most recent window of observations n−1 times.

4. The method of claim 1 , wherein receiving the time-series data includes receiving the time-series data directly from sensors, which generated the observations that comprise the time-series data.

5. The method of claim 1 , wherein receiving the time-series data includes retrieving the time-series data from a storage medium in the time-series database system.

6. The method of claim 1 ,

wherein after performing the MMV operation, the selected subset of observations is stored on rotating storage in the time-series database system; and

wherein caching the selected subset of the time-series data involves moving the selected subset of observations from the rotating storage to the in-memory database cache.

7. The method of claim 1 , wherein the time-series data system comprises a multi-dimensional time-series database system.

8. A non-transitory, computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for caching time-series data in a time-series database system, the method comprising:

receiving the time-series data, wherein the time-series data comprises a series of observations obtained from sensor readings for each signal in a set of signals;

performing a multivariate memory vectorization (MMV) operation on the time-series data, which selects a subset of observations in the time-series data that represents an underlying structure of the time-series data for individual and multivariate signals that comprise the time-series data, wherein performing the MMV operation on the time-series data comprises:

scaling observations for each signal in the time-series data using a mean center, unit variance technique,

dividing observations in the scaled time-series data into a set of windrows for consecutive time intervals,

for each window in the set of windows, selecting observations that include minimum and maximum values for each signal in the set of signals,

for all remaining observations in each window, computing a Euclidean norm,

sorting the remaining observations in each window based on the Euclidean norms,

uniformly selecting a subset of the remaining observations based on the sorted Euclidean norms, and

unscaling all selected observations; and

while subsequently processing a query involving the time-series data,

caching the selected subset of the time-series data in an in-memory database cache in the time-series database system, and

accessing the selected subset of the time-series data from the in-memory database cache.

9. The non-transitory, computer-readable storage medium of claim 8 , wherein the MMV operation selects the subset of observations that represents the underlying structure of the time-series data with respect to one or more of the following: serial correlation, cross correlation, seasonality and stochastic structure.

10. The non-transitory, computer-readable storage medium of claim 8 ,

wherein after performing the multivariate memory vectorization operation on the time-series data, the method further comprises performing a geometric aging compression (GAC) operation on the time-series data;

wherein the GAC operation compresses observations by averaging pairs of consecutive observations; and

wherein the GAC operation:

does not compress a most-recent window of observations;

compresses a next-most recent window of observations once;

compresses a third-most recent window of observations twice; and

compresses an n th -most recent window of observations n−1 times.

11. The non-transitory, computer-readable storage medium of claim 8 , wherein receiving the time-series data includes receiving the time-series data directly from sensors, which generated the observations that comprise the time-series data.

12. The non-transitory, computer-readable storage medium of claim 8 , wherein receiving the time-series data includes retrieving the time-series data from a storage medium in the time-series database system.

13. The non-transitory, computer-readable storage medium of claim 8 ,

wherein after performing the MMV operation, the selected subset of observations is stored on rotating storage in the time-series database system; and

wherein caching the selected subset of the time-series data involves moving the selected subset of observations from the rotating storage to the in-memory database cache.

14. The non-transitory, computer-readable storage medium of claim 8 , wherein the time-series data comprises multivariate time-series data.

15. A system that certifies provenance of time-series data in a time-series database, comprising:

at least one processor and at least one associated memory; and

a caching mechanism that executes on the at least one processor, wherein during operation, the caching mechanism:

receives the time-series data, wherein the time-series data comprises a series of observations obtained from sensor readings for each signal in a set of signals;

performs a multivariate memory vectorization (MMV) operation on the time-series data, which selects a subset of observations in the time-series data that represents an underlying structure of the time-series data for individual and multivariate signals that comprise the time-series data, wherein performing the MMV operation on the time-series data comprises:

scaling observation for each signal in the time-series data using a mean center, unit variance technique,

dividing observations in the scaled time-series data into a set of windows for consecutive time intervals,

for each window in the set of windows, selecting observations that include minimum and maximum values for each signal in the set of signals,

for all remaining observations in each window, computing a Euclidean norm,

sorting the remaining observations in each window based on the Euclidean norms,

uniformly selecting a subset of the remaining observations based on the sorted Euclidean norms, and

unscaling all selected observations; and

while subsequently processing a query involving the time-series data,

caches the selected subset of the time-series data in an in-memory database cache in the time-series database system, and

accesses the selected subset of the time-series data from the in-memory database cache.

16. The system of claim 15 , wherein the MMV operation selects the subset of observations that represents the underlying structure of the time-series data with respect to one or more of the following: serial correlation, cross correlation, seasonality and stochastic structure.

17. The system of claim 15 ,

wherein after performing the multivariate memory vectorization operation on the time-series data, the cache mechanism performs a geometric aging compression (GAC) operation on the time-series data;

wherein the GAC operation compresses observations by averaging pairs of consecutive observations; and

wherein the GAC operation:

does not compress a most-recent window of observations;

compresses a next-most recent window of observations once;

compresses a third-most recent window of observations twice; and

compresses an n th -most recent window of observations n−1 times.