Graph-based compression of data records
In general, embodiments of the present invention provide systems, methods and computer readable media for data record compression using graph-based techniques.
1. A computer-implemented method for generating a compressed list of impression data records, the method comprising:
receiving, by a processor, a set of impression data records describing a sequence of consumer behavior instances collected during a time window, wherein the set of impression data records is associated with a particular consumer;
generating, by the processor, a directed link graph representing the consumer behavior instances; and
generating, by the processor, the compressed list of the impression data records based at least in part on properties of the directed link graph.
2. The method of claim 1 , wherein each impression data record of the set of impression data records comprises a plurality of data components.
3. The method of claim 2 , wherein the plurality of data components includes one or more of an identifier of the promotion, a date on which the interaction occurred, and a position of the promotion within the impression display layout.
4. The method of claim 1 , wherein the graph nodes respectively represent the consumer behavior instances and each of the graph edges connecting a pair of the nodes represents a hyperlink between the nodes.
5. The method of claim 1 , further comprising:
determining a first component of the set of impression data records to be an index component;
generating a sorted list of the set of impression data records by ordering the set of impression data records using a respective value of the index component within each impression data record;
identifying a set of unique index component values within the sorted list of the set of impression data records; and
generating an ordered list of the set of unique index component values using the sorted list of the set of impression data records.
6. The method of claim 5 , further comprising generating an index value list by replacing each impression data record of the set of impression data records with its respective position in the ordered list of the set of unique index component values.
7. The method of claim 6 , further comprising generating an encoded index value list by calculating an encoded index value for each element of the index value list.
8. The method of claim 7 , wherein calculating the encoded index value for each element of the index value list comprises subtracting a value from a previous value in the index value list to produce a difference value; and
in instances where the difference value is positive, multiplying the difference by an integer;
in instances where the difference value is negative, multiplying a mod of the difference by the integer and subtracting the integer.
9. The method of claim 8 , wherein the integer is 2.
10. The method of claim 8 , further comprising compressing the encoded index value list.
11. The method of claim 10 , wherein the encoded index value list is compressed using Elias delta encoding.
12. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations implementing generating a compressed list of impression data records, the operations comprising:
receiving a set of impression data records describing a sequence of consumer behavior instances collected during a time window, wherein the set of impression data records is associated with a particular consumer;
generating a directed link graph representing the consumer behavior instances; and
generating the compressed list of the impression data records based at least in part on properties of the directed link graph.
13. The system of claim 12 , wherein each impression data record of the set of impression data records comprises a plurality of data components.
14. The system of claim 13 , further comprising:
determining a first component of the set of impression data records to be an index component;
generating a sorted list of the set of impression data records by ordering the set of impression data records using a respective value of the index component within each impression data record;
identifying a set of unique index component values within the sorted list of the set of impression data records; and
generating an ordered list of the set of unique index component values using the sorted list of the set of impression data records.
15. The system of claim 14 , further comprising generating an index value list by replacing each impression data record of the set of impression data records with its respective position in the ordered list of the set of unique index component values.
16. The system of claim 15 , further comprising generating an encoded index value list by calculating an encoded index value for each element of the index value list.
17. The system of claim 16 , wherein calculating the encoded index value for each element of the index value list comprises subtracting a value from a previous value in the index value list to produce a difference value; and
in instances where the difference value is positive, multiplying the difference by an integer;
in instances where the difference value is negative, multiplying a mod of the difference by the integer and subtracting the integer.
18. The system of claim 17 , wherein the integer is 2.
19. The system of claim 17 , further comprising compressing the encoded index value list using Elias delta encoding.
20. The system of claim 12 , wherein the graph nodes respectively represent the consumer behavior instances and each of the graph edges connecting a pair of the nodes represents a hyperlink between the nodes.