Storage of time series data using dynamic schema

1. A method, comprising:

receiving a request to store a data point into a time series, wherein the data point comprises at least one key-value pair;

in response to the received request, (i) evaluating whether a current dynamic schema supports the data point by one or more of: (a) determining whether the at least one key-value pair of the data point comprises a key not having a corresponding data type in the current dynamic schema and (b) determining whether the at least one key-value pair of the data point comprises a key for which the corresponding data type in the current dynamic schema is incompatible with the value of the respective key-value pair, and (ii) updating the current dynamic schema to support the data point responsive to the current dynamic schema not supporting the data point, wherein the current dynamic schema supports a plurality of data types;

encoding the data point using an encoding technique that depends on the data type of at least one key-value pair of the data point; and

storing the encoded data point in at least one time series storage buffer,

wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

2. The method of claim 1 , wherein, responsive to a given key-value pair of the data point comprising the key not having the corresponding data type in the current dynamic schema, the updating the current dynamic schema comprises creating a new dynamic schema version that adds the respective key of the given key-value pair of the data point and the corresponding data type.

3. The method of claim 1 , wherein, responsive to a given key-value pair of the data point comprising the key for which the corresponding data type in the current dynamic schema is incompatible with the value of the given key-value pair, the updating the current dynamic schema comprises creating a new dynamic schema version where the key of the given key-value pair is associated with the data type of the value of the given key-value pair.

4. The method of claim 1 , wherein the at least one time series storage buffer comprises one or more of a first buffer that stores encoded data point values for multiple data types, a second buffer that stores generic values of an object data type, a third buffer that stores a version of the current dynamic schema for each encoded data point, and a fourth buffer that stores an offset position of a beginning of each encoded data point in the first buffer.

5. The method of claim 4 , wherein the encoding comprises: (a) for each key of the current dynamic schema, storing a corresponding value from the data point in one of the first and second buffers, based on the data type of the corresponding key in the current dynamic schema; (b) storing a schema version of the current dynamic schema in the third buffer; and (c) storing an offset position of the beginning of the encoded data point in the fourth buffer.

6. The method of claim 4 , further comprising decoding the encoded data point by (a) determining the version of the dynamic schema for the encoded data point by accessing the third buffer; (b) identifying a beginning of the encoded data in the first buffer by accessing the fourth buffer; (c) obtaining the encoded data point from the first buffer using the identified beginning; and (d) for each key of the current dynamic schema, decoding a corresponding value from the encoded data point from one of the first and second buffers based on the data type of the corresponding key in the current dynamic schema.

7. The method of claim 1 , wherein the plurality of data types comprises an unsigned_integer data type, a signed integer data type, a floating data type and an object data type.

8. The method of claim 7 , wherein one or more of: (a) the encoding for the unsigned integer data type comprises applying a value of a given key-value pair of the data point to a variable length encoder and appending the encoded value of the given key-value pair of the data point to a first buffer that stores encoded data point values for multiple data types; (b) the encoding for the signed integer data type comprises applying a value of a given key-value pair of the data point to a zigzag encoder and applying an output of the zigzag encoder to the variable length encoder and appending the encoded value of the given key-value pair of the data point to the first buffer; (c) the encoding for the floating data type comprises encoding a value of a given key-value pair of the data point into one or more long values, applying the one or more long values to the variable length encoder and appending the encoded one or more long values to the first buffer; and (d) the encoding for the object data type comprises appending a value of a given key-value pair of the data point to a second buffer that stores generic values of the object data type and applying an index of the data point in the second buffer to the variable length encoder and appending the encoded index of the data point in the second buffer to the first buffer.

9. The method of claim 7 , wherein the object data type is compatible with substantially all values of any key-value pair.

10. An apparatus comprising:

at least one processing device comprising a processor coupled to a memory;

the at least one processing device being configured to implement the following steps:

receiving a request to store a data point into a time series, wherein the data point comprises at least one key-value pair;

in response to the received request, (i) evaluating whether a current dynamic schema supports the data point by one or more of: (a) determining whether the at least one key-value pair of the data point comprises a key not having a corresponding data type in the current dynamic schema and (b) determining whether the at least one key-value pair of the data point comprises a key for which the corresponding data type in the current dynamic schema is incompatible with the value of the respective key-value pair, and (ii) updating the current dynamic schema to support the data point responsive to the current dynamic schema not supporting the data point, wherein the current dynamic schema supports a plurality of data types;

encoding the data point using an encoding technique that depends on the data type of at least one key-value pair of the data point; and

storing the data point in at least one time series storage buffer.

11. The apparatus of claim 10 , wherein (i) responsive to a given key-value pair of the data point comprising the key not having the corresponding data type in the current dynamic schema, the updating the current dynamic schema comprises creating a new dynamic schema version that adds the respective key of the given key-value pair of the data point and the corresponding data type; and (ii) responsive to a given key-value pair of the data point comprising the key for which the corresponding data type in the current dynamic schema is incompatible with the value of the given key-value pair, the updating the current dynamic schema comprises creating a new dynamic schema version where the key of the given key-value pair is associated with the data type of the value of the given key-value pair.

12. The apparatus of claim 10 , wherein the at least one time series storage buffer comprises one or more of a first buffer that stores encoded data point values for multiple data types, a second buffer that stores generic values of an object data type, a third buffer that stores a version of the current dynamic schema for each encoded data point, and a fourth buffer that stores an offset position of a beginning of each encoded data point in the first buffer.

13. The apparatus of claim 12 , wherein the encoding comprises: (a) for each key of the current dynamic schema, storing a corresponding value from the data point in one of the first and second buffers, based on the data type of the corresponding key in the current dynamic schema; (b) storing a schema version of the current dynamic schema in the third buffer; and (c) storing an offset position of the beginning of the encoded data point in the fourth buffer.

14. The apparatus of claim 12 , further comprising decoding the encoded data point by (a) determining the version of the dynamic schema for the encoded data point by accessing the third buffer; (b) identifying a beginning of the encoded data in the first buffer by accessing the fourth buffer; (c) obtaining the encoded data point from the first buffer using the identified beginning; and (d) for each key of the current dynamic schema, decoding a corresponding value from the encoded data point from one of the first and second buffers based on the data type of the corresponding key in the current dynamic schema.

15. The apparatus of claim 10 , wherein the plurality of data types comprises an unsigned integer data type, a signed integer data type, a floating data type and an object data type.

16. The apparatus of claim 15 , wherein one or more of: (a) the encoding for the unsigned integer data type comprises applying a value of a given key-value pair of the data point to a variable length encoder and appending the encoded value of the given key-value pair of the data point to a first buffer that stores encoded data point values for multiple data types; (b) the encoding for the signed integer data type comprises applying a value of a given key-value pair of the data point to a zigzag encoder and applying an output of the zigzag encoder to the variable length encoder and appending the encoded value of the given key-value pair of the data point to the first buffer; (c) the encoding for the floating data type comprises encoding a value of a given key-value pair of the data point into one or more long values, applying the one or more long values to the variable length encoder and appending the encoded one or more long values to the first buffer; and (d) the encoding for the object data type comprises appending a value of a given key-value pair of the data point to a second buffer that stores generic values of the object data type and applying an index of the data point in the second buffer to the variable length encoder and appending the encoded index of the data point in the second buffer to the first buffer.

17. A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform the following steps:

receiving a request to store a data point into a time series, wherein the data point comprises at least one key-value pair;

in response to the received request, (i) evaluating whether a current dynamic schema supports the data point by one or more of: (a) determining whether the at least one key-value pair of the data point comprises a key not having a corresponding data type in the current dynamic schema and (b) determining whether the at least one key-value pair of the data point comprises a key for which the corresponding data type in the current dynamic schema is incompatible with the value of the respective key-value pair, and (ii) updating the current dynamic schema to support the data point responsive to the current dynamic schema not supporting the data point, wherein the current dynamic schema supports a plurality of data types;

encoding the data point using an encoding technique that depends on the data type of at least one key-value pair of the data point; and

storing the data point in at least one time series storage buffer.

18. The non-transitory processor-readable storage medium of claim 17 , wherein the at least one time series storage buffer comprises one or more of a first buffer that stores encoded data point values for multiple data types, a second buffer that stores generic values of an object data type, a third buffer that stores a version of the current dynamic schema for each encoded data point, and a fourth buffer that stores an offset position of a beginning of each encoded data point in the first buffer.

19. The non-transitory processor-readable storage medium of claim 18 , further comprising decoding the encoded data point by (a) determining the version of the dynamic schema for the encoded data point by accessing the third buffer; (b) identifying a beginning of the encoded data in the first buffer by accessing the fourth buffer; (c) obtaining the encoded data point from the first buffer using the identified beginning; and (d) for each key of the current dynamic schema, decoding a corresponding value from the encoded data point from one of the first and second buffers based on the data type of the corresponding key in the current dynamic schema.

20. The non-transitory processor-readable storage medium of claim 17 , wherein the plurality of data types comprises an unsigned integer data type, a signed integer data type, a floating data type and an object data type.