Method of efficient compression for measurement data
A method for compressing measurement data that includes grouping common data elements in common data messages. Sequential data elements in the groups are subtracted from each other to provide difference values. The difference values are compressed using a suitable compression algorithm to provide compressed blocks of data. Time stamp periods associated with each common data message are also subtracted from each other to provide time stamp difference values. The time stamp difference values are also compressed to provide compressed time stamp blocks. The compressed blocks of data and time stamp blocks are then stored.
1. A method for compressing data, said method comprising:
providing a time sequenced block of messages, each message including a time stamp, a message ID and data, where the data is provided in a plurality of bytes;
identifying messages in the block of messages with the same message ID;
grouping the bytes of data in the messages where bytes of data at the same location in the message for messages with the same message ID are grouped together;
sequentially subtracting the bytes of data in each group to provide a series of subtracted values for each group; and
compressing each series of subtracted values for each group into compressed data.
2. The method according to claim 1 wherein sequentially subtracting the bytes of data includes subtracting the first byte of data in the group from zero.
3. The method according to claim 1 further comprising subtracting the time stamps of sequential messages having the same message ID to provide time stamp series values, sequentially subtracting the time stamp series values to provide time stamp difference values and then compressing the time stamp difference values to provide compressed time stamp information.
4. The method according to claim 3 wherein the first time stamp series value is subtracted from zero to get the time stamp difference value.
5. The method according to claim 1 wherein compressing each series of subtracted values includes using a Burrows-Wheeler compression algorithm.
6. The method according to claim 1 wherein the data is measurement data.
7. The method according to claim 1 wherein providing a time sequenced block of messages includes providing a block of messages stored in at least one buffer.
8. The method according to claim 7 wherein providing a time sequence block of messages includes providing blocks of messages stored in two buffers where one buffer is being loaded with uncompressed messages while the messages in the other buffer is being compressed.
9. The method according to claim 1 wherein providing a time sequenced block of messages includes providing a block of data messages from a CAN data bus on a vehicle.
10. A method for compressing data, said method comprising:
grouping common data elements in common data messages;
sequentially subtracting the common data elements in each group to provide a series of subtracted values for each group; and
compressing each series of subtracted values for each group into compressed data.
11. The method according to claim 10 wherein sequentially subtracting the common data elements includes subtracting a first data element in the group from zero.
12. The method according to claim 10 wherein the data is measurement data.
13. The method according to claim 10 further comprising storing the uncompressed data messages in at least one buffer.
14. The method according to claim 13 wherein the uncompressed messages are stored in two buffers where one buffer is being loaded with uncompressed data messages while data messages in the other buffer are being compressed.
15. The method according to claim 10 wherein the data messages are from a CAN data bus on a vehicle.
16. A method for compressing measurement data from a CAN data bus on a vehicle, said method comprising:
providing a time sequence block of messages from the CAN data bus, each message including a time stamp, a message ID and a plurality of data bytes;
identifying messages in the block of messages with the same message ID;
grouping the bytes of data in the messages where bytes of data at the same location in the message for messages with the same message ID are grouped together;
sequentially subtracting the bytes of data in each group to provide a series of subtracted values for each group, wherein sequentially subtracting the bytes of data includes subtracting the first byte of data in the group from zero;
compressing each series of subtracted values for each group into compressed data;
subtracting the time stamps of sequential messages having the same message ID to provide time stamp series values;
sequentially subtracting the time stamp series values to provide time stamp difference values, where the first time stamp series value is subtracted from zero to get the time stamp difference value;
compressing the time stamp difference values to provide compressed time stamp information; and
storing the compressed data and the compressed time stamp information.
17. The method according to claim 16 wherein compressing the each series of subtracted values and the time stamp difference values includes using a Burrows-Wheeler compression algorithm.
18. The method according to claim 16 wherein providing a time sequenced block of messages includes providing a block of messages stored in at least one buffer.
19. The method according to claim 18 wherein providing a time sequence block of messages includes providing blocks of messages stored in two buffers where one buffer is being loaded with uncompressed data while the data in the other buffer is being compressed.