Determining whether to compress a data segment in a dispersed storage network
View Patent ↗A method for execution by a computing device of a dispersed storage network (DSN). The method begins by receiving a data segment of a data object for dispersed storage error encoding. Prior to the dispersed storage error encoding, the method continues by determining whether to compress the data segment by predicting a first estimated processing cost based on estimated processing costs to compress the data segment to produce a compressed data segment and estimated processing costs to dispersed storage error encode the compressed data segment and predicting a second estimated processing cost based on estimated processing costs to dispersed storage error encode the data segment. When the first estimated processing cost compares favorably to the second estimated processing cost, the method continues by compressing the data segment to produce the compressed data segment and dispersed storage error encoding the compressed data segment to produce a set of encoded data slices.
1. A method for execution by a computing device of a dispersed storage network (DSN), the method comprises:
receiving a data segment of a data object for dispersed storage error encoding;
prior to the dispersed storage error encoding, determining whether to compress the data segment by:
predicting a first estimated processing cost based on estimated processing costs to compress the data segment to produce a compressed data segment and estimated processing costs to dispersed storage error encode the compressed data segment;
predicting a second estimated processing cost based on estimated processing costs to dispersed storage error encode the data segment; and
when the first estimated processing cost compares favorably to the second estimated processing cost:
compressing the data segment to produce the compressed data segment; and
dispersed storage error encoding the compressed data segment to produce a set of encoded data slices.
2. The method of claim 1 , wherein the determining whether to compress the data segment further comprises:
determining whether the data segment has previously undergone a compression function; and
when the data segment has previously undergone a compression function, determining that the first estimated processing cost compares unfavorably to the second estimated processing cost; and
when the first estimated processing cost compares unfavorably to the second estimated processing cost, foregoing the compressing of the data segment.
3. The method of claim 1 , wherein the compressing the data segment comprises:
determining a compression codec; and
compressing the data segment in accordance with the compression codec to produce the compressed data segment.
4. The method of claim 1 further comprises:
prior to compressing the data segment:
determining whether to encrypt the data segment; and
when determined to encrypt the data segment, determining an encryption codec; and
performing the encryption codec and a compression codec on the data segment to produce the compressed data segment.
5. The method of claim 1 further comprises:
prior to compressing the data segment:
determining whether to produce an integrity check for the data segment; and
when determined to produce the integrity check for the data segment, determining an integrity check codec; and
performing the integrity check codec and a compression codec on the data segment to produce the compressed data segment.
6. The method of claim 1 further comprises:
prior to compressing the data segment:
determining whether to produce an integrity check for the data segment;
when determined to produce the integrity check for the data segment, determining an integrity check codec;
determining whether to encrypt the data segment; and
when determined to encrypt the data segment, determining an encryption codec; and
performing the integrity check codec, the encryption codec, and a compression codec on the data segment to produce the compressed data segment.
7. A computing device comprises:
memory;
an interface; and
a processing module operably coupled to the memory and the interface, wherein the processing module is operable to:
receive a data segment of a data object for dispersed storage error encoding;
prior to the dispersed storage error encoding, determine whether to compress the data segment by:
predicting a first estimated processing cost based on estimated processing costs to compress the data segment to produce a compressed data segment and estimated processing costs to dispersed storage error encode the compressed data segment;
predicting a second estimated processing cost based on estimated processing costs to dispersed storage error encode the data segment; and
when the first estimated processing cost compares favorably to the second estimated processing cost:
compressing the data segment to produce the compressed data segment; and
dispersed storage error encoding the compressed data segment to produce a set of encoded data slices.
8. The computing device of claim 7 , wherein the processing module is further operable to determine whether to compress the data segment by:
determining whether the data segment has previously undergone a compression function; and
when the data segment has previously undergone a compression function, determining that the first estimated processing cost compares unfavorably to the second estimated processing cost; and
when the first estimated processing cost compares unfavorably to the second estimated processing cost, foregoing the compressing of the data segment.
9. The computing device of claim 7 , wherein the processing module is further operable to compress the data segment by:
determining a compression codec; and
compressing the data segment in accordance with the compression codec to produce the compressed data segment.
10. The computing device of claim 7 , wherein the processing module is further operable to:
prior to compressing the data segment:
determine whether to encrypt the data segment; and
when determined to encrypt the data segment, determine an encryption codec; and
perform the encryption codec and a compression codec on the data segment to produce the compressed data segment.
11. The computing device of claim 7 , wherein the processing module is further operable to:
prior to compressing the data segment:
determine whether to produce an integrity check for the data segment; and
when determined to produce the integrity check for the data segment, determine an integrity check codec; and
perform the integrity check codec and a compression codec on the data segment to produce the compressed data segment.
12. The computing device of claim 7 , wherein the processing module is further operable to:
prior to compressing the data segment:
determine whether to produce an integrity check for the data segment;
when determined to produce the integrity check for the data segment, determine an integrity check codec;
determine whether to encrypt the data segment; and
when determined to encrypt the data segment, determine an encryption codec; and
perform the integrity check codec, the encryption codec, and a compression codec on the data segment to produce the compressed data segment.