Efficient data write approach for distributed multi-mirror erasure coding system
Certain embodiments described herein are generally directed to generating an updated version of an existing code block in response to a data update to an existing data block of a plurality of data blocks in a distributed multi-mirror erasure coding system. For example, in some embodiments, a first operation cost to generate the updated version of the existing code block by a first method based on the plurality of data blocks and the data update is determined. In certain embodiments, a second operation cost to generate the updated version of the existing code block by a second method based on the existing code block, the existing data block, and the data update is determined. In some embodiments, if the first operation cost is less than or equal to the second operation cost, the first method is used. Otherwise, the second method is used.
1. A method for generating an updated version of an existing code block in response to a data update to an existing data block of a plurality of data blocks in a distributed multi-mirror erasure coding system, comprising:
determining a first operation cost comprising a number of read operations required to generate the updated version of the existing code block based on the plurality of data blocks and the data update;
determining a second operation cost comprising a number of read operations required to generate the updated version of the existing code block based on the existing code block, the existing data block, and the data update;
upon determining that the first operation cost is less than or equal to the second operation cost, generating the updated version of the existing code block based on the plurality of data blocks and the data update;
upon determining that the second operation cost is less than the first operation cost, generating the updated version of the existing code block based on the one or more code blocks, the existing data block, and the data update.
2. The method of claim 1 , wherein the first operation cost and the second operation cost are determined by traversing an object layout tree which comprises a representation of topological locations of the plurality of data blocks and the existing code block.
3. The method of claim 2 , wherein the first operation cost and the second operation cost are determined based, at least in part, on an availability of one or more data blocks of the plurality of data blocks.
4. The method of claim 3 , wherein the first operation cost and the second operation cost are determined based, at least in part, on the topological locations of the plurality of data blocks and the existing code block in the object layout tree.
5. The method of claim 3 , wherein the number of read operations is determined based, at least in part, on the number of read operations required to reconstruct one or more unavailable data blocks.
6. The method of claim 3 , wherein an unavailable data block results in a higher operation cost than an available data block.
7. The method of claim 1 , wherein the existing code blocks comprises a Reed-Solomon code.
8. A non-transitory computer readable medium comprising instructions to be executed in a computer system, wherein the instructions when executed in the computer system perform a method for generating an updated version of an existing code block in response to a data update to an existing data block of a plurality of data blocks in a distributed multi-mirror erasure coding system, the method comprising:
determining a first operation cost comprising a number of read operations required to generate the updated version of the existing code block based on the plurality of data blocks and the data update;
determining a second operation cost comprising a number of read operations required to generate the updated version of the existing code block based on the existing code block, the existing data block, and the data update;
upon determining that the first operation cost is less than or equal to the second operation cost, generating the updated version of the existing code block based on the plurality of data blocks and the data update;
upon determining that the second operation cost is less than the first operation cost, generating the updated version of the existing code block based on the one or more code blocks, the existing data block, and the data update.
9. The non-transitory computer readable medium of claim 8 , wherein the first operation cost and the second operation cost are determined by traversing an object layout tree which comprises a representation of topological locations of the plurality of data blocks and the existing code block.
10. The non-transitory computer readable medium of claim 9 , wherein the first operation cost and the second operation cost are determined based, at least in part, on an availability of one or more data blocks of the plurality of data blocks.
11. The non-transitory computer readable medium of claim 10 , wherein the first operation cost and the second operation cost are determined based, at least in part, on the topological locations of the plurality of data blocks and the existing code block in the object layout tree.
12. The non-transitory computer readable medium of claim 10 , wherein the number of read operations is determined based, at least in part, on the number of read operations required to reconstruct one or more unavailable data blocks.
13. The non-transitory computer readable medium of claim 10 , wherein an unavailable data block results in a higher operation cost than an available data block.
14. The non-transitory computer readable medium of claim 8 , wherein the existing code blocks comprises a Reed-Solomon code.
15. A computer system, wherein system software for the computer system is programmed to execute a method for generating an updated version of an existing code block in response to a data update to an existing data block of a plurality of data blocks in a distributed multi-mirror erasure coding system, the method comprising:
determining a first operation cost comprising a number of read operations required to generate the updated version of the existing code block based on the plurality of data blocks and the data update;
determining a second operation cost comprising a number of read operations required to generate the updated version of the existing code block based on the existing code block, the existing data block, and the data update;
upon determining that the first operation cost is less than or equal to the second operation cost, generating the updated version of the existing code block based on the plurality of data blocks and the data update;
upon determining that the second operation cost is less than the first operation cost, generating the updated version of the existing code block based on the one or more code blocks, the existing data block, and the data update.
16. The computer system of claim 15 , wherein the first operation cost and the second operation cost are determined by traversing an object layout tree which comprises a representation of topological locations of the plurality of data blocks and the existing code block.
17. The computer system of claim 16 , wherein the first operation cost and the second operation cost are determined based, at least in part, on an availability of one or more data blocks of the plurality of data blocks.
18. The computer system of claim 17 , wherein the first operation cost and the second operation cost are determined based, at least in part, on the topological locations of the plurality of data blocks and the existing code block in the object layout tree.
19. The computer system medium of claim 17 , wherein the number of read operations is determined based, at least in part, on the number of read operations required to reconstruct one or more unavailable data blocks.
20. The computer system of claim 17 , wherein an unavailable data block results in a higher operation cost than an available data block.
21. The computer system of claim 15 , wherein the existing code blocks comprises a Reed-Solomon code.