Anomaly database system for processing telemetry data
In some examples, an anomaly database system is provided for processing metrics in telemetry data. An example anomaly database system comprises a continuous data management (CDM) node, the CDM node including a metrics library for sending out system metrics in a sparse manner and a statistics relay for receiving streaming metrics from nodes in a node cluster, the node cluster including the CDM node, the statistics relay pushing the received metrics to a metrics collector. A sparse consumers module pulls metrics, from the metrics collector, pushed to the metrics collector by the statistics relay.
1. An apparatus, comprising:
at least one processor;
memory coupled to the at least one processor; and
instructions stored in the memory and executable by the at least one processor to:
store telemetry data at a plurality of nodes in a data management system, wherein the plurality of nodes are connected to each other via a network, and wherein the plurality of nodes store one or more snapshots of a computing object, the telemetry data comprising a plurality of data points that indicate performance information related to the data management system;
receive a query at a first node of the plurality of nodes in the data management system, the query comprising a request to display at least a portion of the telemetry data via a user interface associated with the data management system, wherein some or all of the at least the portion of the telemetry data is stored at a second node of the plurality of nodes in the data management system and the second node is different from the first node;
obtain, based at least in part on the query, a subset of the at least the portion of the telemetry data from the plurality of nodes by running a sparse algorithm on the at least the portion of the telemetry data, wherein the sparse algorithm includes, in the subset of the at least the portion of the telemetry data, data points that:
are outside a standard deviation band of the at least the portion of the telemetry data,
are different from one or more previous data points, or
satisfy respective difference thresholds with respect to the one or more previous data points or one or more subsequent data points; and
output, in response to the query, an indication of the subset of the at least the portion of the telemetry data for display via the user interface.
2. The apparatus of claim 1 , wherein, to obtain the subset of the at least the portion of the telemetry data, the instructions are executable by the at least one processor to cause the apparatus to:
obtain a first portion of the subset of the at least the portion of the telemetry data from the first node of the plurality of nodes in the data management system; and
obtain a second portion of the subset of the at least the portion of the telemetry data from the second node of the plurality of nodes in the data management system.
3. The apparatus of claim 1 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
transmit the some or all of the at least the portion of the telemetry data from the second node to the first node in response to the query.
4. The apparatus of claim 1 , wherein, to receive the query at the first node of the plurality of nodes in the data management system, the instructions are executable by the at least one processor to cause the apparatus to:
receive, via the user interface, the request indicating to display telemetry data associated with a designated time range, wherein the subset of the at least the portion of the telemetry data comprises a subset of the plurality of data points that correspond to the designated time range.
5. The apparatus of claim 1 , wherein, the instructions are executable by the at least one processor to cause the apparatus to:
determine that a variance of a data point with respect to a previous data point or a subsequent data point is above a threshold, wherein the at least the portion of the telemetry data comprises the previous data point, the data point, and the subsequent data point; and
select the data point to include in the subset of the at least the portion of the telemetry data based at least in part on the determination.
6. The apparatus of claim 1 , wherein the performance information comprises system failure information, resource utilization information, system activity information, or a combination thereof.
7. The apparatus of claim 1 , wherein the instructions are executable by the at least one processor to cause the apparatus to:
receive the query at the first node of the plurality of nodes in the data management system, wherein a first portion of the subset is obtained from the first node, and wherein a second portion of the subset is obtained from the second node of the plurality of nodes.
8. A method, comprising:
storing telemetry data at a plurality of nodes in a data management system, wherein the plurality of nodes are connected to each other via a network, and wherein the plurality of nodes store one or more snapshots of a computing object, the telemetry data comprising a plurality of data points that indicate performance information related to the data management system;
receiving a query at a first node of the plurality of nodes in the data management system, the query comprising a request to display at least a portion of the telemetry data via a user interface associated with the data management system, wherein some or all of the at least the portion of the telemetry data is stored at a second node of the plurality of nodes in the data management system and the second node is different from the first node;
obtaining, based at least in part on the query, a subset of the at least the portion of the telemetry data from the plurality of nodes by running a sparse algorithm on the at least the portion of the telemetry data, wherein the sparse algorithm includes, in the subset of the at least the portion of the telemetry data, data points that:
are outside a standard deviation band of the at least the portion of the telemetry data,
are different from one or more previous data points, or
satisfy respective difference thresholds with respect to the one or more previous data points; and
outputting, in response to the query, an indication of the subset of the at least the portion of the telemetry data for display via the user interface.
9. The method of claim 8 , wherein obtaining the subset of the at least the portion of the telemetry data comprises:
obtaining a first portion of the subset of the at least the portion of the telemetry data from the first node of the plurality of nodes in the data management system; and
obtaining a second portion of the subset of the at least the portion of the telemetry data from the second node of the plurality of nodes in the data management system.
10. The method of claim 8 , further comprising:
transmitting the some or all of the at least the portion of the telemetry data from the second node to the first node in response to the query.
11. The method of claim 8 , wherein receiving the query at the first node of the plurality of nodes in the data management system comprises:
receiving, via the user interface, the request indicating to display telemetry data associated with a designated time range, wherein the subset of the at least the portion of the telemetry data comprises a subset of the plurality of data points that correspond to the designated time range.
12. The method of claim 8 , further comprising:
determining that a variance of a data point with respect to a previous data point or a subsequent data point is above a threshold, wherein the at least the portion of the telemetry data comprises the previous data point, the data point, and the subsequent data point; and
selecting the data point to include in the subset of the at least the portion of the telemetry data based at least in part on the determination.
13. The method of claim 8 , wherein the performance information comprises system failure information, resource utilization information, system activity information, or a combination thereof.
14. A non-transitory computer-readable medium storing code, the code comprising instructions executable by at least one processor to:
store telemetry data at a plurality of nodes in a data management system, wherein the plurality of nodes are connected to each other via a network, and wherein the plurality of nodes store one or more snapshots of a computing object, the telemetry data comprising a plurality of data points that indicate performance information related to the data management system;
receive a query at a first node of the plurality of nodes in the data management system, the query comprising a request to display at least a portion of the telemetry data via a user interface associated with the data management system, wherein some or all of the at least the portion of the telemetry data is stored at a second node of the plurality of nodes in the data management system and the second node is different from the first node;
obtain, based at least in part on the query, a subset of the at least the portion of the telemetry data from the plurality of nodes by running a sparse algorithm on the at least the portion of the telemetry data, wherein the sparse algorithm includes, in the subset of the at least the portion of the telemetry data, data points that:
are outside a standard deviation band of the at least the portion of the telemetry data,
are different from one or more previous data points, or
satisfy respective difference thresholds with respect to the one or more previous data points; and
output, in response to the query, an indication of the subset of the at least the portion of the telemetry data for display via the user interface.
15. The non-transitory computer-readable medium of claim 14 , wherein, to obtain the subset of the at least the portion of the telemetry data, the instructions are executable by the at least one processor to:
obtain a first portion of the subset of the at least the portion of the telemetry data from the first node of the plurality of nodes in the data management system; and
obtain a second portion of the subset of the at least the portion of the telemetry data from the second node of the plurality of nodes in the data management system.
16. The non-transitory computer-readable medium of claim 14 , wherein the instructions are further executable by the at least one processor to:
transmit the some or all of the at least the portion of the telemetry data from the second node to the first node in response to the query.
17. The non-transitory computer-readable medium of claim 14 , wherein, to receive the query at the first node of the plurality of nodes in the data management system, the instructions are executable by the at least one processor to:
receive, via the user interface, the request indicating to display telemetry data associated with a designated time range, wherein the subset of the at least the portion of the telemetry data comprises a subset of the plurality of data points that correspond to the designated time range.