Partition extension method and apparatus
In a distributed storage system, a method for extending a number N of data node devices comprises a server receives an instruction of adding X new data nodes into the distributed storage system. Then, the server obtains a number M of the all partitions included in a hash ring and determines that M/(N+X) is lower than a preset threshold. The hash ring is organized according to a distributed hash table (DHT), and includes a plurality of partitions. Each partition is mapping to a data node. Based upon the determination, the server generates new partitions by multiplying partitions mapping to each data node, based upon the determination. After storing mapping relationship between the new partitions and the X new data nodes, the server adds the X new data nodes into the distributed storage system.
1. A method comprising:
maintaining a distributed hash table (DHT) that maps partitions to physical nodes on which the partitions are stored;
determining that M/N is lower than a threshold wherein M is a count of all partitions identified in the DHT and wherein N is a count of physical nodes in which the M partitions are stored;
adding a respective plurality of new partitions to each of the N nodes, based upon the determination; and
storing a mapping relationship between the new partitions and the nodes on which they are stored in the DHT.
2. The method according to claim 1 , further comprising:
obtaining data stored in a first data node of the N data nodes;
performing a hashing operation on the data to identify a matching partition; and
migrating the data from the first data node to a target data node that stores the matching partition.
3. The method according to claim 2 , further comprising:
identifying the target data node using the mapping relationship.
4. A device comprising:
a memory storage comprising instructions; and
one or more processors in communication with the memory, wherein the one or more processors execute the instructions to:
maintain a distributed hash table (DHT) that maps partitions to physical nodes on which the partitions are stored;
determine that M/N is lower than a threshold wherein M is a count of all partitions identified in the DHT and wherein N is a count of physical nodes in which the M partitions are stored;
add a respective plurality of new partitions to each of the N nodes, based upon the determination; and
store a mapping relationship between the new partitions and the data nodes on which they are stored in the DHT.
5. The device according to claim 4 , wherein the processor is further configured to:
obtain data stored in an first data node of the N data nodes;
perform a hashing operation on the data to identify a matching partition; and
migrate the data from the first data node to a target data node that stores the matching partition.
6. The device according to claim 5 , wherein the processor is further configured to:
identify the target data node using the mapping relationship.
7. A non-transitory computer-readable medium storing computer instructions, that when executed by one or more processors, cause the one or more processors to perform the steps of:
maintain a distributed hash table (DHT) that maps partitions to physical nodes on which the partitions are stored;
determine that M/N is lower than a threshold wherein M is a count of all partitions identified in the DHT and wherein N is a count of physical nodes in which the M partitions are stored;
add a respective plurality of new partitions to each of the N nodes, based upon the determination; and
store a mapping relationship between the new partitions and the data nodes on which they are stored in the DHT.
8. The computer-readable medium according to claim 7 , wherein the steps further comprise:
obtain data stored in an first data node of the N data nodes;
perform a hashing operation on the data to identify a matching partition; and
migrate the data from the first data node to a target data node that stores the matching partition.
9. The computer-readable medium according to claim 8 , wherein the steps further comprise:
identify the target data node using the mapping relationship.