Enhanced processing of time series data via parallelization of instructions
Systems and methods are provided for enhanced processing of time series data via parallelization of instructions. An example method includes receiving a query indicating time series datasets and operations to be performed on the time series datasets. Nodes associated with the query are identified, with each node associated with a time series dataset. Nodes associated with operations to be performed are generated. The nodes are assembled into query tree, with parent nodes of the query tree indicating operations that are to be applied to children nodes. Instructions for processing the query tree are generated. At least a subset of the instructions is provided to one or more compute systems for processing in parallel. Results are received, and presented in a user interface.
1. A computer-implemented method comprising:
by a system of one or more computers,
receiving a query indicating a plurality of time series datasets and operations to be performed on the time series datasets;
identifying nodes associated with the query, each node being associated with an individual time series dataset of the plurality of time series datasets;
generating, based at least in part on the query, one or more nodes associated with respective operations performed on the time series datasets, wherein a first node of the one or more nodes causes interpolation of two or more time series datasets of the plurality of time series datasets, wherein the two or more time series datasets store information at different sampling rates and wherein new values for at least one of the two or more time series datasets are determined based on the different sampling rates;
assembling the nodes into at least one query tree, wherein parent nodes of the query tree indicate operations that are to be applied to children nodes;
generating instructions associated with processing the query tree, and providing at least a subset of the instructions to one or more compute systems for processing in parallel; and
in response to receiving results associated with the generated instructions, presenting, in a user interface, the results.
2. The method of claim 1 , wherein each node points to a location at which a respective time series dataset is stored.
3. The method of claim 1 , wherein the query indicates metadata and wherein identifying comprises:
accessing metadata information associated with nodes; and
searching for nodes which satisfy the indicated metadata.
4. The method of claim 1 , wherein searching for nodes comprises utilizing a metadata service, the metadata service extracting metadata from time series datasets.
5. The method of claim 1 , wherein assembling the nodes into at least one query tree comprises:
determining, based on a generated node, that an operation is to be applied to at least two sets of nodes associated with time series datasets;
identifying, based on the determination, the nodes to be included in each of the two sets; and
assembling the respective sets into a respective query tree, wherein each query tree includes same nodes associated with operations to be performed.
6. The method of claim 5 , wherein the nodes included in each set are associated with same metadata, the metadata being specified in the received query.
7. The method of claim 1 , further comprising:
accessing information indicating one or more user-defined functions to be applied to the results; and
applying the one or more user-defined functions to the results, wherein the functions are applied sequentially to a stream of the results.
8. The method of claim 1 , wherein the results are received as a data frame, and wherein the user interface presents the results as an interactive table responsive to user input.
9. A system comprising one or more computers and computer storage media storing instructions that, when executed by the system, cause the system to perform operations comprising:
receiving a query indicating a plurality of time series datasets and operations to be performed on the time series datasets;
identifying nodes associated with the query, each node being associated with an individual time series dataset of the plurality of time series datasets;
generating, based at least in part on the query, one or more nodes associated with respective operations performed on the time series datasets, wherein a first node of the one or more nodes causes interpolation of two or more time series datasets of the plurality of time series datasets, wherein the two or more time series datasets store information at different sampling rates and wherein new values for at least one of the two or more time series datasets are determined based on the different sampling rates;
assembling the nodes into at least one query tree, wherein parent nodes of the query tree indicate operations that are to be applied to children nodes;
generating instructions associated with processing the query tree, and providing at least a subset of the instructions to one or more compute systems for processing in parallel; and
in response to receiving results associated with the generated instructions, presenting, in a user interface, the results.
10. The system of claim 9 , wherein each node points to a location at which a respective time series dataset is stored.
11. The system of claim 9 , wherein assembling the nodes into at least one query tree comprises:
determining, based on a generated node, that an operation is to be applied to at least two sets of nodes associated with time series datasets;
identifying, based on the determination, the nodes to be included in each of the two sets; and
assembling the respective sets into a respective query tree, wherein each query tree includes same nodes associated with operations to be performed.
12. The system of claim 11 , wherein the nodes included in each set are associated with same metadata, the metadata being specified in the received query.
13. The system of claim 9 , wherein the operations further comprise:
accessing information indicating one or more user-defined functions to be applied to the results; and
applying the one or more user-defined functions to the results, wherein the functions are applied sequentially to a stream of the results.
14. The system of claim 9 , wherein the results are received as a data frame, and wherein the user interface presents the results as an interactive table responsive to user input.
15. Non-transitory computer storage media storing instructions that when executed by a system of one or more computers, cause the system to perform operations comprising:
receiving a query indicating a plurality of time series datasets and operations to be performed on the time series datasets;
identifying nodes associated with the query, each node being associated with an individual time series dataset of the plurality of time series datasets;
generating, based at least in part on the query, one or more nodes associated with respective operations performed on the time series datasets, wherein a first node of the one or more nodes causes interpolation of two or more time series datasets of the plurality of time series datasets, wherein the two or more time series datasets store information at different sampling rates and wherein new values for at least one of the two or more time series datasets are determined based on the different sampling rates;
assembling the nodes into at least one query tree, wherein parent nodes of the query tree indicate operations that are to be applied to children nodes;
generating instructions associated with processing the query tree, and providing at least a subset of the instructions to one or more compute systems for processing in parallel; and
in response to receiving results associated with the generated instructions, presenting, in a user interface, the results.
16. The computer storage media of claim 15 , wherein each node points to a location at which a respective time series dataset is stored.
17. The computer storage media of claim 15 , wherein assembling the nodes into at least one query tree comprises:
determining, based on a generated node, that an operation is to be applied to at least two sets of nodes associated with time series datasets;
identifying, based on the determination, the nodes to be included in each of the two sets; and
assembling the respective sets into a respective query tree, wherein each query tree includes same nodes associated with operations to be performed.
18. The computer storage media of claim 17 , wherein the nodes included in each set are associated with same metadata, the metadata being specified in the received query.
19. The computer storage media of claim 15 , wherein the operations further comprise:
accessing information indicating one or more user-defined functions to be applied to the results; and
applying the one or more user-defined functions to the results, wherein the functions are applied sequentially to a stream of the results.
20. The computer storage media of claim 15 , wherein the results are received as a data frame, and wherein the user interface presents the results as an interactive table responsive to user input.