Data storage method and apparatus in storage system
A storage method includes: calculating M check blocks of N first data blocks based on a first erasure coding (EC) technology; respectively storing the N first data blocks and the M check blocks of the N first data blocks into (N+M) storage nodes in a storage system; updating the first EC technology to a second EC technology; calculating R check blocks of S second data blocks based on the second EC technology, where S is greater than N, a ratio of S to R is greater than a ratio of N to M, S, R, N, and M are all positive integers, and the second data block is a data block received after the first EC technology is updated to the second EC technology.
1 . A method in a storage system, wherein the method comprises:
calculating M check blocks of N first data blocks based on a first erasure coding (EC) technology;
respectively storing the N first data blocks and the M check blocks into first corresponding storage nodes in (N+M) storage nodes in the storage system;
migrating, after adding first storage nodes to the storage system and without updating the first EC technology, one or more of the N first data blocks and the M check blocks to the first storage nodes;
updating the first EC technology to a second EC technology, wherein prior to the updating, the storage system is configured with the first EC technology and not the second EC technology;
receiving S second data blocks after updating the first EC technology to the second EC technology;
calculating R first check blocks of the S second data blocks based on the second EC technology, wherein S is greater than N, wherein a first ratio of S to R is greater than a second ratio of N to M, and wherein S, R, N, and M are positive integers; and
respectively storing the S second data blocks and the R first check blocks into second corresponding storage nodes in (S+R) storage nodes in the storage system.
2 . The method of claim 1 , further comprising:
selecting S target data blocks from third data blocks stored in the storage system based on the first EC technology;
calculating R second check blocks of the S target data blocks based on the second EC technology; and
respectively storing the S target data blocks and the R second check blocks into third corresponding storage nodes in the storage system.
3 . The method of claim 1 , wherein R is greater than or equal to M.
4 . The method of claim 1 , further comprising:
receiving a read request;
reading first data in the N first data blocks based on the first EC technology when the read request requests to read the first data; and
reading second data in the S second data blocks based on the second EC technology when the read request requests to read the second data.
5 . The method of claim 1 , wherein each of and the (S+R) storage nodes is any one of a storage drive, a disk enclosure, or a storage server.
6 . The method of claim 1 , further comprising:
receiving a data access request for one or more data blocks;
reading metadata corresponding to the one or more data blocks, wherein the metadata indicates which of the first EC technology and the second EC technology is associated with the one or more data blocks; and
accessing the one or more data blocks based on the one of the first EC technology and the second EC technology indicated by the metadata.
7 . The method of claim 2 , wherein the S target data blocks are selected after updating the first EC technology to the second EC technology and during an idle time period of the storage system, and wherein the idle time period comprises at least one of a time period in which a running load of the storage system is lower than a load threshold, current to-be-written data of the storage system being less than a first preset threshold, current to-be-read data of the storage system being less than a second preset threshold, or hardware resource utilization of the storage system being less than a third preset threshold.
8 . A storage apparatus comprising:
a memory configured to store programming instructions; and
at least one processor coupled to the memory and configured to execute the programming instructions to cause the storage apparatus to:
calculate M check blocks of N first data blocks based on a first erasure coding (EC) technology;
respectively store the N first data blocks and the M check blocks into first corresponding storage nodes in (N+M) storage nodes in a storage system;
migrate, after adding first storage nodes to the storage system and without updating the first EC technology, one or more of the N first data blocks and the M check blocks to the first storage nodes;
update the first EC technology to a second EC technology, wherein prior to the updating, the storage system is configured with the first EC technology and not the second EC technology;
receive S second data blocks after updating the first EC technology to the second EC technology;
calculate R first check blocks of the S second data blocks based on the second EC technology, wherein S is greater than N, wherein a first ratio of S to R is greater than a second ratio of N to M, and wherein S, R, N, and M are positive integers; and
respectively store the S second data blocks and the R first check blocks into second corresponding storage nodes in (S+R) storage nodes in the storage system.
9 . The storage apparatus of claim 8 , wherein the at least one processor is further configured to execute the programming instructions to cause the storage apparatus to:
select S target data blocks from third data blocks stored in the storage system based on the first EC technology;
calculate R second check blocks of the S target data blocks based on the second EC technology; and
respectively store the S target data blocks and the R second check blocks into third corresponding storage nodes in the storage system.
10 . The storage apparatus of claim 8 , wherein R is greater than or equal to M.
11 . The storage apparatus of claim 8 , wherein the at least one processor is further configured to execute the programming instructions to cause the storage apparatus to:
receive a read request;
read first data in the N first data blocks based on the first EC technology when the read request requests to read the first data; and
read second data in the S second data blocks based on the second EC technology when the read request requests to read the second data.
12 . The storage apparatus of claim 8 , wherein each of the (N+M) storage nodes and the (S+R) storage nodes is any one of a storage drive, a disk enclosure, or a storage server.
13 . The storage apparatus of claim 8 , wherein the at least one processor is further configured to execute the programming instructions to cause the storage apparatus to:
receive a data access request for one or more data blocks;
read metadata corresponding to the one or more data blocks, wherein the metadata indicates which of the first EC technology and the second EC technology is associated with the one or more data blocks; and
access the one or more data blocks based on the one of the first EC technology and the second EC technology indicated by the metadata.
14 . The storage apparatus of claim 9 , wherein the at least one processor is further configured to further select the S target data blocks after updating the first EC technology to the second EC technology and during an idle time period of the storage system, and wherein the idle time period comprises at least one of a time period in which a running load of the storage system is lower than a load threshold, current to-be-written data of the storage system being less than a first preset threshold, current to-be-read data of the storage system being less than a second preset threshold, or hardware resource utilization of the storage system being less than a third preset threshold.
15 . A computer program product comprising computer-executable instructions that are stored on a non-transitory computer-readable medium and that, when executed by at least one processor, cause a storage apparatus to:
calculate M check blocks of N first data blocks based on a first erasure coding (EC) technology;
respectively store the N first data blocks and the M check blocks into first corresponding storage nodes in (N+M) storage nodes in a storage system;
migrate, after adding first storage nodes to the storage system and without updating the first EC technology, one or more of the N first data blocks and the M check blocks to the first storage nodes;
update the first EC technology to a second EC technology, wherein prior to the updating, the storage system is configured with the first EC technology and not the second EC technology;
receive S second data blocks after updating the first EC technology to the second EC technology;
calculate R first check blocks of the S second data blocks based on the second EC technology, wherein S is greater than N, wherein a first ratio of S to R is greater than a second ratio of N to M, and wherein S, R, N, and M are positive integers; and
respectively store the S second data blocks and the R first check blocks into second corresponding storage nodes in (S+R) storage nodes in the storage system.
16 . The computer program product of claim 15 , wherein the computer-executable instructions further cause the storage apparatus to:
select S target data blocks from third data blocks stored in the storage system based on the first EC technology;
calculate R second check blocks of the S target data blocks based on the second EC technology; and
respectively store the S target data blocks and the R second check blocks into third corresponding storage nodes in the storage system.
17 . The computer program product of claim 15 , wherein R is greater than or equal to M.
18 . The computer program product of claim 15 , wherein the computer-executable instructions further cause the storage apparatus to:
receive a read request;
read first data in the N first data blocks based on the first EC technology when the read request requests to read the first data; and
read second data in the S second data blocks based on the second EC technology when the read request requests to read the second data.
19 . The computer program product of claim 15 , wherein when executed by the at least one processor, the computer-executable instructions further cause the storage system to:
receive a data access request for one or more data blocks;
read metadata corresponding to the one or more data blocks, wherein the metadata indicates which of the first EC technology and the second EC technology is associated with the one or more data blocks; and
access the one or more data blocks based on the one of the first EC technology and the second EC technology indicated by the metadata.
20 . The computer program product of claim 16 , wherein when executed by the at least one processor, the computer-executable instructions further cause the storage system to further select the S target data blocks after updating the first EC technology to the second EC technology and during an idle time period of the storage system, and wherein the idle time period comprises at least one of a time period in which a running load of the storage system is lower than a load threshold, current to-be-written data of the storage system being less than a first preset threshold, current to-be-read data of the storage system being less than a second preset threshold, or hardware resource utilization of the storage system being less than a third preset threshold.