Determining whether to compress a data segment in a dispersed storage network
A method for execution by a computing device of a dispersed storage network (DSN) begins by receiving a data segment for dispersed storage error encoding. Prior to encoding, the method continues by determining whether to compress the data segment by predicting a first estimated processing cost (EPC) based on EPCs to dispersed storage error decode a compressed set of encoded data slices to recover a compressed data segment and EPCs to decompress the compressed data segment to recover the data segment and by predicting a second EPC based on EPCs to dispersed storage error decode the set of encoded data slices to recover the data segment. When the first EPC compares favorably to the second EPC, the method continues by compressing the data segment to produce the compressed data segment and dispersed storage error encoding the compressed data segment to produce the compressed set of encoded data slices.
1. A method for execution by a computing device of a dispersed storage network (DSN), the method comprises:
receiving a data segment of a data object for dispersed storage error encoding;
prior to the dispersed storage error encoding, determining whether to compress the data segment by:
predicting a first estimated processing cost based on estimated processing costs to dispersed storage error decode a compressed set of encoded data slices to recover a compressed data segment and estimated processing costs to decompress the compressed data segment to recover the data segment, wherein the data segment is compressed to produce the compressed data segment, and wherein the compressed data segment is dispersed storage error encoded to produce the compressed set of encoded data slices; predicting a second estimated processing cost based on estimated processing costs to dispersed storage error decode a set of encoded data slices to recover the data segment, wherein the data segment is dispersed storage error encoded to produce the set of encoded data slices; and
when the first estimated processing cost compares favorably to the second estimated processing cost:
compressing the data segment to produce the compressed data segment; and
dispersed storage error encoding the compressed data segment to produce the compressed set of encoded data slices.
2. The method of claim 1 , wherein the determining whether to compress the data segment further comprises:
determining whether the data segment has previously undergone a compression function; and
when the data segment has previously undergone a compression function, determining that the first estimated processing cost compares unfavorably to the second estimated processing cost; and
when the first estimated processing cost compares unfavorably to the second estimated processing cost, foregoing the compressing of the data segment.
3. The method of claim 1 , wherein the compressing the data segment comprises:
determining a compression codec; and
compressing the data segment in accordance with the compression codec to produce the compressed data segment.
4. The method of claim 1 further comprises:
prior to compressing the data segment:
determining whether to encrypt the data segment; and
when determined to encrypt the data segment, determining an encryption codec; and
performing the encryption codec and a compression codec on the data segment to produce the compressed data segment.
5. The method of claim 1 further comprises:
prior to compressing the data segment:
determining whether to produce an integrity check for the data segment; and
when determined to produce the integrity check for the data segment, determining an integrity check codec; and
performing the integrity check codec and a compression codec on the data segment to produce the compressed data segment.
6. The method of claim 1 further comprises:
prior to compressing the data segment:
determining whether to produce an integrity check for the data segment;
when determined to produce the integrity check for the data segment, determining an integrity check codec;
determining whether to encrypt the data segment; and
when determined to encrypt the data segment, determining an encryption codec; and
performing the integrity check codec, the encryption codec, and a compression codec on the data segment to produce the compressed data segment.
7. A computing device comprises:
memory;
an interface; and
a processing module operably coupled to the memory and the interface, wherein the processing module is operable to:
receive a data segment of a data object for dispersed storage error encoding;
prior to the dispersed storage error encoding, determine whether to compress the data segment by:
predicting a first estimated processing cost based on estimated processing costs to dispersed storage error decode a compressed set of encoded data slices to recover a compressed data segment and estimated processing costs to decompress the compressed data segment to recover the data segment, wherein the data segment is compressed to produce the compressed data segment, and wherein the compressed data segment is dispersed storage error encoded to produce the compressed set of encoded data slices; predicting a second estimated processing cost based on estimated processing costs to dispersed storage error decode a set of encoded data slices to recover the data segment, wherein the data segment is dispersed storage error encoded to produce the set of encoded data slices; and
when the first estimated processing cost compares favorably to the second estimated processing cost:
compressing the data segment to produce the compressed data segment; and
dispersed storage error encoding the compressed data segment to produce the compressed set of encoded data slices.
8. The computing device of claim 7 , wherein the processing module is further operable to determine whether to compress the data segment by:
determining whether the data segment has previously undergone a compression function; and
when the data segment has previously undergone a compression function, determining that the first estimated processing cost compares unfavorably to the second estimated processing cost; and
when the first estimated processing cost compares unfavorably to the second estimated processing cost, foregoing the compressing of the data segment.
9. The computing device of claim 7 , wherein the processing module is further operable to compress the data segment by:
determining a compression codec; and
compressing the data segment in accordance with the compression codec to produce the compressed data segment.
10. The computing device of claim 7 , wherein the processing module is further operable to:
prior to compressing the data segment:
determine whether to encrypt the data segment; and
when determined to encrypt the data segment, determine an encryption codec; and
perform the encryption codec and a compression codec on the data segment to produce the compressed data segment.
11. The computing device of claim 7 , wherein the processing module is further operable to:
prior to compressing the data segment:
determine whether to produce an integrity check for the data segment; and
when determined to produce the integrity check for the data segment, determine an integrity check codec; and
perform the integrity check codec and a compression codec on the data segment to produce the compressed data segment.
12. The computing device of claim 7 , wherein the processing module is further operable to:
prior to compressing the data segment:
determine whether to produce an integrity check for the data segment;
when determined to produce the integrity check for the data segment, determine an integrity check codec;
determine whether to encrypt the data segment; and
when determined to encrypt the data segment, determine an encryption codec; and
perform the integrity check codec, the encryption codec, and a compression codec on the data segment to produce the compressed data segment.