Efficiently storing data in a dispersed storage network
View Patent ↗A method includes determining that one or more data blocks of a permanently stored data blocks are to be deleted. In response, the method further includes obtaining a group of partial redundancy data for the permanently stored data blocks. The method further includes identifying a temporarily stored plurality of data blocks for which partial redundancy data does not yet exist. The method further includes creating a new plurality of data blocks from data blocks of the permanently stored plurality of data blocks that are to remain permanently stored and data blocks from the temporarily stored plurality of data blocks that are to be permanently stored. The method further includes permanently storing the new plurality of data blocks. The method further includes generating a new group of partial redundancy data. The method further includes sending the new group of partial redundancy data and the group of partial redundancy data.
1. A method for execution by a dispersed storage (DS) unit of a dispersed storage network (DSN), the method comprises:
determining that one or more data blocks of a permanently stored plurality of data blocks are to be deleted; and
in response to the determining that the one or more data blocks are to be deleted:
obtaining a group of partial redundancy data for the permanently stored plurality of data blocks;
identifying a temporarily stored plurality of data blocks for which partial redundancy data does not yet exist;
creating a new plurality of data blocks from data blocks of the permanently stored plurality of data blocks that are to remain permanently stored and data blocks from the temporarily stored plurality of data blocks that are to be permanently stored;
permanently storing the new plurality of data blocks;
generating a new group of partial redundancy data based on the new plurality of data blocks and in accordance with a dispersed storage error coding function; and
sending the new group of partial redundancy data and the group of partial redundancy data such that redundancy data DS units are able to generate redundancy data regarding the new plurality of data blocks.
2. The method of claim 1 , wherein the determining that the one or more data blocks of the permanently stored plurality of data blocks are to be deleted comprises:
determining that at least one other DS unit of a set of DS units is storing data blocks that are substantially similar to the one or more data blocks; and
in response to the determining that the at least one other DS unit is storing data blocks that are substantially similar to the one or more data blocks:
determining whether a de-duplication function is to be applied to the one or more data blocks;
when the de-duplication function is to be applied, determining that the one or more data blocks are to be deleted; and
when the de-duplication function is not to be applied, determining that the one or more data blocks are not to be deleted.
3. The method of claim 1 , wherein the determining that the one or more data blocks of the permanently stored plurality of data blocks are to be deleted comprises:
analyzing the permanently stored plurality of data blocks in accordance with data analysis criteria; and
when the analysis of the permanently stored plurality of data blocks is unfavorable, determining that the one or more data blocks of the permanently stored plurality of data blocks are to be deleted.
4. The method of claim 1 , wherein the generating the new group of partial redundancy data based on the new plurality of data blocks comprises:
generating a first partial redundancy data for a first redundancy DS unit based on at least some of the new plurality of data blocks and a first encoding parameter of the dispersed storage error coding function; and
generating a second partial redundancy data for a second redundancy DS unit based on at least another some of the new plurality of data blocks and a second encoding parameter of the dispersed storage error coding function.
5. A dispersed storage (DS) module of a dispersed storage (DS) unit of a dispersed storage network (DSN), the DS module comprises:
a first module, when operable within a computing device, causes the computing device to:
determine that one or more data blocks of a permanently stored plurality of data blocks are to be deleted;
a second module, when operable within the computing device, causes the computing device to:
in response to the determining that the one or more data blocks are to be deleted:
obtain a group of partial redundancy data for the permanently stored plurality of data blocks;
identify a temporarily stored plurality of data blocks for which partial redundancy data does not yet exist;
create a new plurality of data blocks from data blocks of the permanently stored plurality of data blocks that are to remain permanently stored and data blocks from the temporarily stored plurality of data blocks that are to be permanently stored; and
permanently store the new plurality of data blocks; and
a third module, when operable within the computing device, causes the computing device to:
generate a new group of partial redundancy data based on the new plurality of data blocks and in accordance with a dispersed storage error coding function; and
send the new group of partial redundancy data and the group of partial redundancy data such that redundancy data DS units are able to generate redundancy data regarding the new plurality of data blocks.
6. The DS module of claim 5 , wherein the first module functions to determine that the one or more data blocks of the permanently stored plurality of data blocks are to be deleted by:
determining that at least one other DS unit of a set of DS units is storing data blocks that are substantially similar to the one or more data blocks; and
in response to the determining that the at least one other DS unit is storing data blocks that are substantially similar to the one or more data blocks:
determining whether a de-duplication function is to be applied to the one or more data blocks;
when the de-duplication function is to be applied, determining that the one or more data blocks are to be deleted; and
when the de-duplication function is not to be applied, determining that the one or more data blocks are not to be deleted.
7. The DS module of claim 5 , wherein the first module functions to determine that the one or more data blocks of the permanently stored plurality of data blocks are to be deleted by:
analyzing the permanently stored plurality of data blocks in accordance with data analysis criteria; and
when the analysis of the permanently stored plurality of data blocks is unfavorable, determining that the one or more data blocks of the permanently stored plurality of data blocks are to be deleted.
8. The DS module of claim 5 , wherein the third module functions to generate the new group of partial redundancy data based on the new plurality of data blocks by:
generating a first partial redundancy data for a first redundancy DS unit based on at least some of the new plurality of data blocks and a first encoding parameter of the dispersed storage error coding function; and
generating a second partial redundancy data for a second redundancy DS unit based on at least another some of the new plurality of data blocks and a second encoding parameter of the dispersed storage error coding function.