Systems and methods for optimized re-striping in an erasure encoded storage
Aspects of the present disclosure describe methods and systems for optimized re-striping in an erasure encoded storage. In one exemplary aspect, a method may receive a request to re-stripe a plurality of data blocks arranged as a tile in the erasure encoded storage, wherein the request comprises a desired tile width. The method may identify (1) a number of data blocks in the tile and (2) a width of the tile. The method may determine a maximum number of data blocks that do not need to be rearranged when reconfiguring the tile to the desired tile width. Furthermore, the method may determine a tile reconfiguration with the desired tile width that does not rearrange the maximum number of the data blocks of the tile, and may re-stripe the tile in accordance with the tile reconfiguration.
1. A computer-implemented method for optimized re-striping in an erasure encoded storage, the method comprising:
receiving a request to re-stripe a plurality of data blocks arranged as a tile in the erasure encoded storage, wherein the request comprises a desired tile width;
identifying (1) a number of data blocks in the tile and (2) a width of the tile;
determining a maximum number of data blocks that do not need to be rearranged when reconfiguring the tile to the desired tile width;
determining a tile reconfiguration with the desired tile width that does not rearrange the maximum number of the data blocks of the tile by calculating tile configurations for all possible tile widths that the number of data blocks is divisible by; and
re-striping the tile in accordance with the tile reconfiguration.
2. The method of claim 1 , wherein applying CP transformation comprises:
determining a respective maximum number of data blocks that do not need to be rearranged when reconfiguring a given tile to a given tile width and determining a respective tile reconfiguration with the given tile width that does not rearrange the respective maximum number of the data blocks of the given tile.
3. The method of claim 2 , wherein calculating the tile configurations further comprises successively applying the CP transformation to all possible tiles with the number of data blocks in order of increasing heights starting from the tile specified by an initial width or height.
4. The method of claim 2 , wherein calculating the tile configurations further comprises successively applying the CP transformation to all possible tiles with the number of data blocks in order of increasing width starting from the tile specified by an initial width or height.
5. The method of claim 4 , wherein the initial height of the tile is equal to 1.
6. The method of claim 4 , wherein the initial tile width is equal to a width of a stripe where the data were originally stored.
7. The method of claim 1 , further comprising:
prior to determining the maximum number of the data blocks:
determining that at least one intermediate tile width exists between the width of the tile and the desired tile width such that the number of data blocks is divisible by the at least one intermediate tile width;
determining at least a second maximum number of data blocks that do not need to be rearranged; and
determining a second tile reconfiguration with the at least one intermediate tile width that does not rearrange the second maximum number of the data blocks of the tile,
wherein determining the maximum number of the data blocks that do not need to rearranged when reconfiguring the tile to the desired tile width is based on the second tile reconfiguration.
8. The method of claim 7 , wherein all determined tile reconfigurations of the plurality of data blocks are retained in the erasure encoded storage until the tile has been re-striped to the desired tile width, wherein the determined tile reconfigurations comprise a tile size, a current width or height and an initial width or height.
9. The method of claim 8 , wherein a coordinator node performs re-striping and wherein a centralized metadata service (MDS) determines configurations and delivers node assignments to the coordinator node in special storage mapping messages.
10. The method of claim 9 , further comprising:
subsequent to re-striping the tile to the desired tile width, sending a completion message to the MDS, wherein the MDS is configured to:
store (1) an initial tile configuration of the re-striped tile and the width and (2) the second tile reconfiguration of the re-striped tile and the desired tile width; and
deliver new mapping messages to all the storage nodes to confirm completion of the re-striping, wherein the storage nodes drop all data and metadata associated with the initial tile configuration of the re-striped tile.
11. The method of claim 1 , wherein the maximum number of data blocks is determined based on the number of data blocks in the tile and a maximum possible fraction S max , wherein
S max ( n,m )=min( n,m )/max( n,m )
and wherein n is the width of the tile and m is the desired tile width.
12. A system for optimized re-striping in an erasure encoded storage, the system comprising:
a hardware processor configured to:
receive a request to re-stripe a plurality of data blocks arranged as a tile in the erasure encoded storage, wherein the request comprises a desired tile width;
identify (1) a number of data blocks in the tile and (2) a width of the tile;
determine a maximum number of data blocks that do not need to be rearranged when reconfiguring the tile to the desired tile width;
determine a tile reconfiguration with the desired tile width that does not rearrange the maximum number of the data blocks of the tile by calculating tile configurations for all possible tile widths that the number of data blocks is divisible by; and
re-stripe the tile in accordance with the tile reconfiguration.
13. The system of claim 12 , wherein the hardware processor is further configured to apply CP transformation by:
determining a respective maximum number of data blocks that do not need to be rearranged when reconfiguring a given tile to a given tile width and determining a respective tile reconfiguration with the given tile width that does not rearrange the respective maximum number of the data blocks of the given tile.
14. The system of claim 13 , wherein the hardware processor is further configured to calculate the tile configurations by successively applying the CP transformation to all possible tiles with the number of data blocks in order of increasing heights starting from the tile specified by an initial width or height.
15. The system of claim 13 , wherein the hardware processor is further configured to calculate the tile configurations by successively applying the CP transformation to all possible tiles with the number of data blocks in order of increasing width starting from the tile specified by an initial width or height.
16. The system of claim 15 , wherein the initial height of the tile is equal to 1 .
17. The system of claim 15 , wherein the initial tile width is equal to a width of a stripe where the data were originally stored.
18. A non-transitory computer readable medium storing thereon computer executable instructions for optimized re-striping in an erasure encoded storage, including instructions for:
receiving a request to re-stripe a plurality of data blocks arranged as a tile in the erasure encoded storage, wherein the request comprises a desired tile width;
identifying (1) a number of data blocks in the tile and (2) a width of the tile;
determining a maximum number of data blocks that do not need to be rearranged when reconfiguring the tile to the desired tile width;
determining a tile reconfiguration with the desired tile width that does not rearrange the maximum number of the data blocks of the tile by calculating tile configurations for all possible tile widths that the number of data blocks is divisible by; and
re-striping the tile in accordance with the tile reconfiguration.