Method, device, and computer program product for extending partition of storage system
A partition of a storage system is extended in a more effective manner, thereby improving the overall operating efficiency of the storage system. For instance, objects are divided into a first group of objects and a second group of objects, based upon which a first partition including the first group of objects and a second partition including the second group of objects are created. A partition index of the partition is obtained, wherein a group of nodes included in the partition index respectively represents metadata of the group of objects. A first partition index and a second partition index are respectively generated based on the partition index, wherein a first group of nodes included in the first partition index respectively represents metadata of the first group of objects, and a second group of nodes included in the second partition index respectively represents metadata of the first group of objects.
1. A method for partition extending for a storage system, wherein the storage system comprises at least one partition, a partition of the at least one partition comprises a group of objects, and the method comprises:
dividing, by a system comprising a processor, the group of objects into a first group of objects and a second group of objects, wherein the dividing the group of objects comprises applying a hash function that identifies a number of bits in a hash value, wherein the number of bits in the hash value determines how to divide the group of objects according to different processes usable to divide the group of objects;
creating a first partition and a second partition based on the first group of objects and the second group of objects, wherein the first partition comprises the first group of objects, and the second partition comprises the second group of objects;
obtaining a partition index of the partition, wherein a group of nodes comprised in the partition index respectively represents metadata of the group of objects; and
generating a first partition index and a second partition index based on the partition index, wherein a first group of nodes comprised in the first partition index respectively represents metadata of the first group of objects, and a second group of nodes comprised in the second partition index respectively represents metadata of the first group of objects.
2. The method according to claim 1 , wherein obtaining the partition index of the partition comprises:
flushing data in a memory table of the partition to a storage chunk corresponding to the partition in a storage chunk layer of the storage system; and
reading the partition index from the storage chunk.
3. The method according to claim 1 , wherein generating the second partition index based on the partition index comprises: determining the second partition index based on nodes corresponding to the second group of objects in the partition index.
4. The method according to claim 3 , wherein determining the second partition index comprises:
based on a copy on write process, reading the nodes corresponding to the second group of objects from the partition index;
updating the read nodes based on the second group of objects to write the updated nodes into the memory table of the second partition index; and
flushing the memory table of the second partition index to a storage chunk corresponding to the second partition in the storage chunk layer of the storage system to generate the second partition index.
5. The method according to claim 4 , wherein determining the second partition index further comprises:
before reading the nodes corresponding to the second group of objects, stopping processing of an access request for the partition.
6. The method according to claim 5 , wherein the partition index is an additional index, and generating the first partition index based on the partition index comprises:
after the partition index has been read, setting nodes representing metadata of objects in the second group of objects in the partition index to be invalid to generate the first partition index.
7. The method according to claim 6 , further comprising: resuming processing of an access request for the storage system based on the first partition index and the second partition index.
8. The method according to claim 7 , wherein resuming processing of the access request for the partition comprises:
determining an extended hash value of a target object involved in the access request based on the hash function associated with the partition;
determining a target partition where the target object is located from the first partition and the second partition based on the extended hash value; and
processing the access request based on an index of the target partition.
9. The method according to claim 1 , wherein dividing the group of objects into the first group of objects and the second group of objects comprises: for an object in the group of objects,
determining an extended hash value of the object based on the hash function associated with the partition;
in response to a determination that the extended hash value is the same as the hash value of the object, classifying the object into the first group of objects, wherein a length of the hash value is less than that of the extended hash value; and
in response to a determination that the extended hash value is different from the hash value, classifying the object into the second group of objects.
10. The method according to claim 1 , wherein the method is executed in a management process at a partition management node for managing the partition, and wherein the method is executed in response to at least any one of the following:
extension of a storage capacity of the storage system; and
extension of a parallel access capability of the storage system.
11. An electronic device, comprising:
at least one processor;
a volatile memory; and
a memory coupled to the at least one processor, wherein the memory has instructions stored therein, the instructions, when executed by the at least one processor, cause the device to execute actions relating to partition extension for a storage system, the storage system comprises at least one partition, a partition of the at least one partition comprises a group of objects, and the actions comprise:
dividing the group of objects into a first group of objects and a second group of objects, wherein the dividing the group of objects comprises applying a hash function that identifies a number of bits in a hash value, wherein the number of bits in the hash value determines how to divide the group of objects according to different defined processes to divide the group of objects;
creating a first partition and a second partition based on the first group of objects and the second group of objects, wherein the first partition comprises the first group of objects, and the second partition comprises the second group of objects;
obtaining a partition index of the partition, wherein a group of nodes comprised in the partition index respectively represents metadata of the group of objects; and
generating a first partition index and a second partition index based on the partition index, wherein a first group of nodes comprised in the first partition index respectively represents metadata of the first group of objects, and a second group of nodes comprised in the second partition index respectively represents metadata of the first group of objects.
12. The device according to claim 11 , wherein obtaining the partition index of the partition comprises:
flushing data in a memory table of the partition to a storage chunk corresponding to the partition in a storage chunk layer of the storage system; and
reading the partition index from the storage chunk.
13. The device according to claim 11 , wherein generating the second partition index based on the partition index comprises: determining the second partition index based on nodes corresponding to the second group of objects in the partition index.
14. The device according to claim 13 , wherein determining the second partition index comprises:
based on a copy on write process, reading the nodes corresponding to the second group of objects from the partition index;
updating the read nodes based on the second group of objects to write the updated nodes into the memory table of the second partition index; and
flushing the memory table of the second partition index to a storage chunk corresponding to the second partition in the storage chunk layer of the storage system to generate the second partition index.
15. The device according to claim 14 , wherein determining the second partition index further comprises:
before reading the nodes corresponding to the second group of objects, stopping processing of an access request for the partition.
16. The device according to claim 15 , wherein the partition index is an additional index, and wherein generating the first partition index based on the partition index comprises:
after the partition index has been read, setting nodes representing metadata of objects in the second group of objects in the partition index to be invalid to generate the first partition index.
17. The device according to claim 16 , further comprising: resuming processing of an access request for the storage system based on the first partition index and the second partition index.
18. The device according to claim 17 , wherein resuming processing of the access request for the partition comprises:
determining an extended hash value of a target object involved in the access request based on a hash function associated with the partition;
determining a target partition where the target object is located from the first partition and the second partition based on the extended hash value; and
processing the access request based on an index of the target partition.
19. The device according to claim 11 , wherein dividing the group of objects into the first group of objects and the second group of objects comprises: for an object in the group of objects,
determining an extended hash value of the object based on a hash function associated with the partition;
in response to a determination that the extended hash value is the same as a hash value of the object, classifying the object into the first group of objects, wherein a length of the hash value is less than that of the extended hash value; and
in response to a determination that the extended hash value is different from the hash value, classifying the object into the second group of objects.
20. A computer program product tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions for usage to perform partition extension in a storage system, wherein the storage system comprises partitions, a partition of the partitions comprises a group of objects, and wherein the machine-executable instructions are used to implement operations, comprising:
dividing the group of objects into a first group of objects and a second group of objects, wherein the dividing the group of objects comprises applying a hash function that identifies a number of bits in a hash value, wherein the number of bits in the hash value determines how to divide the group of objects according to different ways to divide the group of objects;
creating a first partition and a second partition based on the first group of objects and the second group of objects, wherein the first partition comprises the first group of objects, and the second partition comprises the second group of objects;
obtaining a partition index of the partition, wherein a group of nodes comprised in the partition index respectively represents metadata of the group of objects; and
generating a first partition index and a second partition index based on the partition index, wherein a first group of nodes comprised in the first partition index respectively represents metadata of the first group of objects, and a second group of nodes comprised in the second partition index respectively represents metadata of the first group of objects.