Compressing digital metrics for transmission across a network utilizing a graph-based compression dictionary and time slice delta compression
The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate compressed metric data for digital metrics utilizing a graph-based compression dictionary and time slice compression. For instance, the disclosed systems can utilize a dynamically modifiable graph-based compression dictionary to generate compressed metric label identifiers for metric labels of digital metrics. The graph-based compression dictionary can include nodes and edges corresponding to metric label segments and metric label identifier values, respectively. The disclosed systems can traverse the graph-based compression dictionary using a metric label to determine the corresponding compressed metric label identifier. The disclosed systems can further generate delta compression values for the metric values of the digital metrics. For instance, the disclosed systems can compare metric values within a single time slice (e.g., a time stamp) to generate corresponding delta compression values. In some cases, the disclosed systems further compare the metric values across a time window.
1. A computer-implemented method comprising:
generating a set of compressed digital data from a set of key-value pairs comprising key labels and values by:
generating, utilizing a graph-based compression dictionary shared between a first computing device and a second computing device, compressed label identifiers based on the key labels;
determining a sorted value sequence from the key-value pairs by sorting the key-value pairs according to the values; and
generating delta compression values from the values utilizing the sorted value sequence; and
transmitting the set of compressed digital data comprising the compressed label identifiers and the delta compression values from the first computing device to the second computing device across a network.
2. The computer-implemented method of claim 1 , wherein transmitting the set of compressed digital data further comprises:
identifying a key label, from the key labels, unrepresented within the graph-based compression dictionary; and
transmitting a dictionary update to the graph-based compression dictionary between from the first computing device to the second computing device with the set of compressed digital data.
3. The computer-implemented method of claim 2 , wherein the graph-based compression dictionary comprises a plurality of nodes corresponding to key label segments and a plurality of edges corresponding to label identifier values.
4. The computer-implemented method of claim 3 , wherein transmitting the dictionary update comprises transmitting at least one of a node or an edge corresponding to the key label to modify within the graph-based compression dictionary.
5. The computer-implemented method of claim 1 , wherein generating the compressed label identifiers based on the key labels and utilizing the graph-based compression dictionary comprises, for a particular key label:
traversing a plurality of nodes of the graph-based compression dictionary using segments of the particular key label; and
generating a compressed label identifier for the particular key label using one or more label identifier values corresponding to one or more edges between the plurality of nodes.
6. The computer-implemented method of claim 1 , wherein generating the compressed label identifiers based on the key labels and utilizing the graph-based compression dictionary comprises generating the compressed label identifiers based on the key labels utilizing a finite-state transducer.
7. The computer-implemented method of claim 1 , wherein generating the delta compression values from the values utilizing the sorted value sequence comprises:
determining that a first key value and a first preceding key value from the sorted value sequence include a same value;
generating a first delta compression value indicating that the first key value and the first preceding key value include the same value;
determining a value difference between a second key value and a second preceding key value from the sorted value sequence; and
generating a second delta compression value representing the value difference.
8. The computer-implemented method of claim 1 , further comprising:
generating, at the first computing device, a metric blocklist indicating key-value pairs to be withheld from transmission across the network;
generating, at the first computing device, a compressed metric blocklist; and
transmitting the compressed metric blocklist from the first computing device to another computing device across the network.
9. A non-transitory computer-readable medium storing program instructions thereon that, when executed by at least one processor, cause a computing device system to:
generate a set of compressed digital data from a set of key-value pairs comprising key labels and values by:
generating, utilizing a graph-based compression dictionary shared between a first computing device and a second computing device, compressed label identifiers based on the key labels;
determining a sorted value sequence from the key-value pairs by sorting the key-value pairs according to the values; and
generating delta compression values from the values utilizing the sorted value sequence; and
cause the set of compressed digital data comprising the compressed label identifiers and the delta compression values to be transmitted from the first computing device to the second computing device across a network.
10. The non-transitory computer-readable medium of claim 9 , wherein the program instructions that cause the computing system to cause the set of compressed digital data to be transmitted from the first computing device to the second device comprise program instructions that, when executed by the at least one processor, cause the computing system to:
identify a key label, from the key labels, unrepresented within the graph-based compression dictionary; and
cause a dictionary update to the graph-based compression dictionary to be transmitted from the first computing device to the second computing device with the set of compressed digital data.
11. The non-transitory computer-readable medium of claim 10 , wherein the graph-based compression dictionary comprises a plurality of nodes corresponding to key label segments and a plurality of edges corresponding to label identifier values, and wherein the program instructions that cause the computing system to cause the dictionary update to be transmitted from the first computing device to the second computing device comprise program instructions that, when executed by the at least one processor, cause the computing system to cause at least one of a node or an edge corresponding to the key label to be transmitted from the first computing device to the second computing device to modify within the graph-based compression dictionary.
12. The non-transitory computer-readable medium of claim 9 , wherein the program instructions that cause the computing system to generate the compressed label identifiers based on the key labels and utilizing the graph-based compression dictionary comprise program instructions that, when executed by the at least one processor, cause the computing system to, for a particular key label:
traverse a plurality of nodes of the graph-based compression dictionary using segments of the particular key label; and
generate a compressed label identifier for the particular key label using one or more label identifier values corresponding to one or more edges between the plurality of nodes.
13. The non-transitory computer-readable medium of claim 9 , wherein the program instructions that cause the computing system to generate the compressed label identifiers based on the key labels utilizing the graph-based compression dictionary comprise program instructions that, when executed by the at least one processor, cause the computing system to generate the compressed label identifiers based on the key labels utilizing a finite-state transducer.
14. The non-transitory computer-readable medium of claim 9 , wherein the program instructions that cause the computing system to generate the delta compression values from the values utilizing the sorted value sequence comprise program instructions that, when executed by the at least one processor, cause the computing system to:
determine that a first key value and a first preceding key value from the sorted value sequence include a same value;
generate a first delta compression value indicating that the first key value and the first preceding key value include the same value;
determine a value difference between a second key value and a second preceding key value from the sorted value sequence; and
generate a second delta compression value representing the value difference.
15. A computing system comprising:
at least one processor;
at least one non-transitory computer-readable medium; and
program instructions stored on the at least one non-transitory computer-readable medium that, when executed by the at least one processor, cause the computing system to:
generate a set of compressed digital data from a set of key-value pairs comprising key labels and values by:
generating, utilizing a graph-based compression dictionary shared between a first computing device and a second computing device, compressed label identifiers based on the key labels;
determining a sorted value sequence from the key-value pairs by sorting the key-value pairs according to the values; and
generating delta compression values from the values utilizing the sorted value sequence; and
cause the set of compressed digital data comprising the compressed label identifiers and the delta compression values to be transmitted from the first computing device to the second computing device across a network.
16. The computing system of claim 15 , wherein the program instructions that cause the computing system to cause the set of compressed digital data to be transmitted from the first computing device to the second computing device comprise program instruction that, when executed by the at least one processor, cause the computing system to:
identify a key label, from the key labels, unrepresented within the graph-based compression dictionary; and
cause a dictionary update to the graph-based compression dictionary to be transmitted from the first computing device to the second computing device with the set of compressed digital data.
17. The computing system of claim 16 , wherein the graph-based compression dictionary comprises a plurality of nodes corresponding to key label segments and a plurality of edges corresponding to label identifier values, and wherein the program instructions that cause the computing system to cause the dictionary update to be transmitted from the first computing device to the second computing device comprise program instructions that, when executed by the at least one processor, cause the computing system to cause at least one of a node or an edge corresponding to the key label to be transmitted from the first computing device to the second computing device to modify within the graph-based compression dictionary.
18. The computing system of claim 15 , wherein the program instructions that cause the computing system to generate the compressed label identifiers based on the key labels and utilizing the graph-based compression dictionary comprise program instructions that, when executed by the at least one processor, cause the computing system to, for a particular key label:
traverse a plurality of nodes of the graph-based compression dictionary using segments of the particular key label; and
generate a compressed label identifier for the particular key label using one or more label identifier values corresponding to one or more edges between the plurality of nodes.
19. The computing system of claim 15 , wherein the program instructions that cause the computing system to generate the compressed label identifiers based on the key labels utilizing the graph-based compression dictionary comprise program instructions that, when executed by the at least one processor, cause the computing system to generate the compressed label identifiers based on the key labels utilizing a finite-state transducer.
20. The computing system of claim 15 , wherein the program instructions that cause the computing system to generate the delta compression values from the values utilizing the sorted value sequence comprise program instructions that, when executed by the at least one processor, cause the computing system to:
determine that a first key value and a first preceding key value from the sorted value sequence include a same value;
generate a first delta compression value indicating that the first key value and the first preceding key value include the same value;
determine a value difference between a second key value and a second preceding key value from the sorted value sequence; and
generate a second delta compression value representing the value difference.