Verifiable intra-cluster migration for a chunk storage system
Verifiable intra-cluster migration (VICM) for a chunk storage system is disclosed. VICM can migrate data from a first portion of a cluster to a second portion of a cluster. VICM can comprise locking a first portion of a cluster and locking a corresponding first cluster table during a preparation phase. Chunks of the first portion can then be migrated, during a migration phase, to the second portion and a second cluster table, corresponding to the second portion, can be updated accordingly. Garbage management operations, including recovery operations, can be performed via the second cluster table and the second portion during the migration phase. Upon completion of the migration phase, a reconciliation phase can comprise verifying chunk relationships of the second cluster table and the second portion based on the first cluster table. Exceptions to the verification can be reported via an exception report.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
in response to determining a condition of a cluster storage system satisfies a rule related to performing an intra-cluster migration, locking a first chunk storage space of a cluster storage system and locking a first cluster table corresponding to the first chunk storage space;
performing an intra-cluster migration of a chunk from the first chunk storage space to a second chunk storage space of the cluster storage system;
updating a second cluster table corresponding to the second chunk storage space based on the intra-cluster migration of the chunk; and
verifying the second cluster table and the second chunk storage space based on the first cluster table.
2. The system of claim 1 , wherein the condition corresponds to addition of a node to the cluster storage system.
3. The system of claim 1 , wherein the operations further comprise performing a garbage management process during the intra-cluster migration.
4. The system of claim 3 , wherein the garbage management process comprises marking a chunk of the second cluster table with an indication corresponding to a chunk status.
5. The system of claim 4 , wherein the chunk status is selected from a group of chunk statuses comprising a deleted indication, a recovered indication, and a recovered and deleted indication.
6. The system of claim 1 , wherein the locking the first chunk storage space and the locking the first cluster table permit reading of data from chunks stored in the first chunk storage space.
7. The system of claim 1 , wherein the locking the first chunk storage space and the locking the first cluster table restrict writing a chunk into the first chunk storage space.
8. The system of claim 1 , wherein the locking the first chunk storage space and the locking the first cluster table restrict modifying a chunk stored in the first chunk storage space.
9. The system of claim 1 , wherein the operations further comprise generating an exception report based on the verifying the second cluster table and the second chunk storage space.
10. The system of claim 9 , wherein the operations further comprise correction of an exception of the exception report via human interaction.
11. The system of claim 1 , wherein the operations further comprise, subsequent to the verifying, deletion or recovery of storage space of the first chunk storage space by the cluster storage system.
12. The system of claim 11 , wherein the deletion of storage space comprises removal of a node of the cluster storage system.
13. The system of claim 1 , wherein the first cluster table is in at least one tree chunk and wherein the at least one tree chunk is stored in the first chunk storage space.
14. A method, comprising:
prohibiting, by a processor of a real cluster storage system executing a first chunk storage space, storage of a first chunk to the first chunk storage space, wherein the first chunk storage space corresponds to a first cluster table;
updating, by the processor, a second cluster table in response to causing an intra-cluster migration of a second chunk stored by the first chunk storage space to a second chunk storage space of the real cluster storage system, wherein the second chunk storage space corresponds to the second cluster table; and
generating, by the processor, an exception report based on verifying the second cluster table based on the first cluster table.
15. The method of claim 14 , further comprising performing, by the processor, a garbage management process during the intra-cluster migration of the second chunk from the first chunk storage space to the second chunk storage space.
16. The method of claim 14 , further comprising receiving, by the processor, an instruction related to manual healing of an exception of the exception report.
17. The method of claim 14 , further comprising, deleting, by the processor, storage space of the first chunk storage space, wherein the deleting is subsequent to the verifying the second cluster table.
18. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
marking a first chunk storage space of a cluster storage system as read-only;
marking a first cluster table as read-only, wherein the first cluster table corresponds to the first chunk storage space;
migrating a chunk stored by the first chunk storage space to a second chunk storage space of the cluster storage system;
updating a second cluster table in response to the migrating, wherein the second chunk storage space corresponds to the second cluster table;
verifying the second cluster table based on the first cluster table; and
communicating an exception report based on the verifying.
19. The non-transitory machine-readable storage medium of claim 18 , wherein the operations further comprise performing a garbage management operation contemporaneously with intra-cluster migrating of the chunk between the first chunk storage space and the second chunk storage space.
20. The non-transitory machine-readable storage medium of claim 18 , wherein the operations further comprise deleting storage space of the first chunk storage space subsequent to the verifying the second cluster table.