Dynamic data compression selection
Aspects of dynamic data compression selection are presented. In an example method, as uncompressed data chunks of a data stream are compressed, at least one performance factor affecting selection of one of multiple compression algorithms for the uncompressed data chunks of the data stream may be determined. Each of the multiple compression algorithms may facilitate a different expected compression ratio. One of the multiple compression algorithms may be selected separately for each uncompressed data chunk of the data stream based on the at least one performance factor. Each uncompressed data chunk may be compressed using the selected one of the multiple compression algorithms for the uncompressed data chunk.
1. A method for data compression, the method comprising:
receiving uncompressed source data at a data compression system from a data storage device, the uncompressed source data comprising data chunks of a data stream;
monitoring, utilizing a hardware processor of the data compression system, at least one performance factor of the data compression system;
selecting a first compression algorithm from a plurality of compression algorithms based on the monitored at least one performance factor of the data compression system; and
compressing a first uncompressed data chunk of the data stream using the selected first compression algorithm.
2. The method of claim 1 wherein each of the plurality of compression algorithms facilitates a different expected compression ratio.
3. The method of claim 2 wherein the plurality of compression algorithms comprise the first compression algorithm and a second compression algorithm, the second compression algorithm facilitating a lower expected compression ratio than the first compression algorithm, the method further comprising:
selecting the second compression algorithm to compress a second uncompressed data chunk of the data stream in response to a resulting compression ratio being lower than a particular threshold.
4. The method of claim 1 wherein the at least one performance factor comprises a network throughput for at least one compressed data chunk of the data stream transmitted over a network for storing at a target location.
5. The method of claim 4 further comprising:
comparing the network throughput relative to a storage throughput for uncompressed data chunks of the data stream retrieved from the data storage device prior to compression.
6. The method of claim 5 further comprising:
determining a depth of a queue of compressed data chunks awaiting transmission over the network for storing at the target location.
7. The method of claim 6 wherein the plurality of compression algorithms comprise the first compression algorithm and a second compression algorithm, the second compression algorithm facilitating a lower expected compression ratio than the first compression algorithm, the method further comprising:
selecting the second compression algorithm in response to the network throughput being higher than the storage throughput for uncompressed data chunks of the data stream retrieved from the data storage device prior to compression.
8. The method of claim 1 wherein the at least one performance factor comprises a current processor utilization of the data compression system.
9. The method of claim 8 further comprising:
evaluating the current processor utilization by determining a number of currently active execution threads in the data compression system.
10. A system for compression of data, the system comprising:
a dynamic compression system to compress received data;
a data storage in communication with the dynamic compression system, the data storage providing an uncompressed source of data comprising uncompressed data chunks of a data stream to the dynamic compression system; and
a network interface transmitting compressed data chunks of the data stream to a target device over a network;
wherein the dynamic compression system monitors at least one performance factor of the data compression system, selects a first compression algorithm from a plurality of compression algorithms based on the monitored at least one performance factor of the dynamic compression system, and compresses a first uncompressed data chunk of the data stream using the selected first compression algorithm.
11. The system of claim 10 wherein each of the plurality of compression algorithms facilitates a different expected compression ratio.
12. The system of claim 11 wherein the plurality of compression algorithms comprise the first compression algorithm and a second compression algorithm, the second compression algorithm facilitating a lower expected compression ratio than the first compression algorithm, the dynamic compression system further selecting the second compression algorithm to compress a second uncompressed data chunk of the data stream in response to a resulting compression ratio being lower than a particular threshold.
13. The system of claim 10 wherein the at least one performance factor comprises a network throughput at the network interface for at least one compressed data chunk of the data stream transmitted for storing at the target device.
14. The system of claim 13 wherein the dynamic compression system further compares the network throughput relative to a storage throughput for uncompressed data chunks of the data stream prior to compression and determines a depth of a queue of compressed data chunks awaiting transmission over the network for storing at a target location.
15. The system of claim 14 wherein the plurality of compression algorithms comprise the first compression algorithm and a second compression algorithm, the second compression algorithm facilitating a lower expected compression ratio than the first compression algorithm, the dynamic compression system further to select the second compression algorithm in response to the network throughput being higher than the storage throughput for uncompressed data chunks of the data stream prior to compression.
16. The system of claim 10 wherein the at least one performance factor comprises a current processor utilization of the dynamic compression system.
17. A computer-implemented method executed by a processing device from instructions stored on a computer-readable storage medium, the computer-implemented method comprising:
receiving uncompressed source data at a data compression system from a local data storage device, the uncompressed source data comprising data chunks of a data stream;
monitoring, utilizing the processing device, at least one performance factor of the data compression system;
selecting a first compression algorithm from a plurality of compression algorithms based on the monitored at least one performance factor of the data compression system; and
compressing a first uncompressed data chunk of the data stream using the selected first compression algorithm.
18. The computer-implemented method of claim 17 wherein each of the plurality of compression algorithms facilitates a different expected compression ratio.
19. The computer-implemented method of claim 17 wherein the at least one performance factor comprises a network throughput for at least one compressed data chunk of the data stream transmitted over a network for storing at a target location.
20. The computer-implemented method of claim 17 wherein the at least one performance factor comprises a current processor utilization of the data compression system.