Intra-cluster migration for elastic cloud storage devices
View Patent ↗Facilitating intra-cluster migration of data in an elastic cloud storage environment is provided herein. A system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise scheduling a migration of a data chunk from a source storage node to a target storage node. Further, the operations can comprise facilitating the migration of the data chunk from a first location in the source storage node to a second location in the target storage node. Data indicative of an identity of the data chunk can be migrated with the data chunk. The operations can also comprise replacing the first location with the second location in a storage mapping table.
1. A method, comprising:
ascertaining, by a system comprising a processor, that a first storage device is to be removed from service;
determining, by the system, that a data chunk of the first storage device is scheduled to be moved to a second storage device;
moving, by the system, the data chunk, and a chunk identifier associated with the data chunk from the first storage device to the second storage device; and
updating, by the system, a storage reference data structure with the chunk identifier and an indication of a location of the data chunk in the second storage device, wherein the moving the data chunk and the chunk identifier comprises eliminating repeated replication of the data chunk.
2. The method of claim 1 , wherein the location is a first location, wherein the indication of the location is a first indication of the first location, and wherein the method further comprises:
removing, by the system, a second indication of a second location of the data chunk in the first storage device from the storage reference data structure.
3. The method of claim 1 , further comprising:
prior to the moving the data chunk, dividing, by the system, a capacity of the second storage device into a set of free blocks, wherein the set of free blocks comprises the location.
4. The method of claim 1 , further comprising:
prior to the moving the data chunk, determining, by the system, that the data chunk comprises immutable data.
5. The method of claim 1 , further comprising:
prior to the moving the data chunk,
determining, by the system, the data chunk is open for writes at the first storage device,
adding, by the system, a reference of the data chunk to a backlog data structure, and
forcing, by the system, a closure of the data chunk at the first storage device based on another determination, after a defined interval, that the data chunk is still open for writes at the first storage device.
6. The method of claim 1 , wherein the moving the data chunk and the chunk identifier comprises preserving locations of existing objects at the first storage device.
7. The method of claim 1 , further comprising:
prior to the moving the data chunk,
calculating, by the system, a first amount of used capacity in the first storage device, and
reserving, by the system, a second amount of capacity in the second storage device, wherein the first amount of used capacity and the second amount of capacity are a similar amount of capacity as determined by a similarity criterion.
8. The method of claim 1 , further comprising:
prior to the moving the data chunk,
recovering, by the system, a consistency of the data chunk based on a first determination that the data chunk is not consistent,
allocating, by the system, an amount of available space in the second storage device, and
approving, by the system, a movement of the data chunk based on a second determination that the amount of available space allocated in the second storage device is at a sufficient level to accommodate the data chunk.
9. The method of claim 1 , wherein the first storage device and the second storage device are geographically distributed devices.
10. The method of claim 1 , wherein the moving the data chunk and the chunk identifier comprises avoiding a creation of a quantity of garbage for collection by a garbage collector.
11. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
scheduling a migration of a data chunk from a source storage node to a target storage node based on a determination that the source storage node is scheduled to be removed from service; and
mitigating repeated data replication of the data chunk comprising:
facilitating the migration of the data chunk from a first location in the source storage node to a second location in the target storage node, wherein data indicative of an identity of the data chunk is migrated with the data chunk; and
replacing the first location with the second location in a storage mapping table.
12. The system of claim 11 , wherein the replacing the first location with the second location comprises:
removing a first reference to the first location for the identity of the data chunk from the storage mapping table; and
adding a second reference to the second location for the identity of the data chunk to the storage mapping table.
13. The system of claim 11 , wherein the operations further comprise:
prior to the facilitating the migration of the data chunk, dividing a storage capacity of the target storage node into a set of free blocks, wherein the first location is included in a block of the set of free blocks.
14. The system of claim 11 , wherein the operations further comprise:
prior to the facilitating the migration of the data chunk, determining the data chunk is sealed and comprises immutable data.
15. The system of claim 11 , wherein the operations further comprise:
prior to the facilitating the migration of the data chunk,
performing a first determination that the data chunk is open for writes at the source storage node;
performing a second determination that the data chunk is open for writes at the source storage node after a defined interval measured from the first determination; and
causing, at the source storage node, a forced closure of the data chunk based on the second determination.
16. The system of claim 11 , wherein the operations further comprise facilitating a processing efficiency based on the migration of the data chunk and data indicative of the identity of the data chunk at a chunk level as compared to movement of data at an object level.
17. The system of claim 11 , wherein the source storage node and the target storage node are storage devices of an elastic cloud storage system.
18. A computer-readable storage medium comprising instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising:
determining a data chunk is to be migrated from a first storage node device to a second storage node device based on an elimination of the first storage node device; and
eliminating repeated replication of the data chunk based on a movement of the data chunk and information indicative of an identity of the data chunk from a first location to a second location, wherein the eliminating comprises:
moving the data chunk and information indicative of the identity of the data chunk from the first storage node device to the second storage node device; and
updating a location of the data chunk in a chunk manager data structure from the first location in the first storage node device to the second location in the second storage node device.
19. The computer-readable storage medium of claim 18 , wherein the operations further comprise preserving locations of existing objects at the first storage node device after the moving the data chunk.
20. The computer-readable storage medium of claim 18 , wherein the moving the data chunk and the information indicative of the identity of the data chunk comprises mitigating an amount of garbage collected during a garbage collection process.