Data storage method, apparatus, and system
A storage client needs to store to-be-written data into a distributed storage system, and storage nodes corresponding to a first data unit assigned for the to-be-written data by a management server are only some nodes in a storage node group. When receiving a status of the first data unit returned by the management server, the storage client may determine quantities of data blocks and parity blocks needing to be generated during EC coding on the to-be-written data. The storage client stores the generated data blocks and parity blocks into some storage nodes designated by the management server in a partition where the first data unit is located. Accordingly, dynamic adjustment of an EC redundancy ratio is implemented, and the management server may exclude some nodes in the partition from a storage range of the to-be-written data based on a requirement, thereby reducing a data storage IO amount.
1. A data storage system, wherein the data storage system comprises:
a storage client;
a management server; and
a storage node group includes N+M storage nodes for storing data according to a erasure coding (EC) technology, wherein
when the N+M storage nodes are in a normal state, the N storage nodes are configured to store data blocks;
the M storage nodes are configured to store parity blocks generated based on the data blocks with the EC technology; and
N and M are positive integers greater than 2;
the storage client is configured to:
obtain a quantity of the storage nodes that the storage client is allowed to use in the storage group from the management server, wherein the quantity of the storage nodes is less than N+M;
generate a plurality of data blocks by dividing data and one or more parity blocks for the plurality of data blocks, wherein a quantity of the plurality of data blocks and the one or more parity blocks are the same as the quantity of the storage nodes; and
store the plurality of data blocks and the one or more parity blocks to the storage nodes that the storage client is allowed to use in the storage group; and
store the quantity of storage nodes in a first data unit in a first partition of the storage node group.
2. The system according to claim 1 , wherein the management server is further configured to record a status of the N+M storage nodes in the storage node group.
3. The system according to claim 2 , wherein the storage client is further configured to obtain the status from the management server.
4. The system according to claim 1 , wherein the quantity of the one or more parity blocks is equal to M, and the quantity of the plurality of data blocks is equal to a value obtained by subtracting M from the quantity of the storage nodes the storage client is allowed to use.
5. A data storage method in a data storage system, wherein the data storage system comprises:
a storage client;
a management server; and
a storage node group includes N+M storage nodes for storing data according to an erasure coding (EC) technology, wherein
when the N+M storage nodes are in a normal state, the N storage nodes are configured to store data blocks;
the M storage nodes are configured to store parity blocks generated based on the data blocks with the EC technology; and
N and M are positive integers greater than 2; and, wherein the method comprising:
obtaining, by the storage client, a quantity of the storage nodes that the storage client is allowed to use in the storage group from the management server, wherein the quantity of the storage nodes is less than N+M;
generating, by the storage client, a plurality of data blocks by dividing data and one or more parity blocks for the plurality of data blocks;
a quantity of the plurality of data blocks and the one or more parity blocks are the same as the quantity of the storage nodes;
storing, by the storage client, the plurality of data blocks and the one or more parity blocks to the storage nodes that the storage client is allowed to use in the storage group; and
storing the quantity of storage nodes in a first data unit in a first partition of the storage node group.
6. The method according to claim 5 , wherein the method further comprises:
recording, by the management server, a status of the N+M storage nodes in the storage node group.
7. The method according to claim 6 , wherein the method further comprises:
obtaining, by the storage client, the status from the management server.
8. The method according to claim 5 , wherein the quantity of the one or more parity blocks is equal to M, and the quantity of the plurality of data blocks is equal to a value obtained by subtracting M from the quantity of the storage nodes the storage client is allowed to use.
9. A non-volatile and non-transitory computer readable storage medium, wherein the computer readable storage medium stores a computer instruction applied to a data storage system, wherein the data storage system comprises:
a storage client;
a management server; and
a storage node group includes N+M storage nodes for storing data according to an erasure coding (EC) technology, wherein
when the N+M storage nodes are in a normal state, the N storage nodes are configured to store data blocks;
the M storage nodes are configured to store parity blocks generated based on the data blocks with the EC technology; and
N and M are positive integers greater than 2; and, wherein
when the computer instruction executed by a computer as the storage client to cause the computer perform:
obtaining a quantity of the storage nodes that the storage client is allowed to use in the storage group from the management server, wherein the quantity of the storage nodes is less than N+M;
generating a plurality of data blocks by dividing data and one or more parity blocks for the plurality of data blocks, wherein a quantity of the plurality of data blocks and the one or more parity blocks is the same as the quantity of the storage nodes;
storing the plurality of data blocks and the one or more parity blocks to the storage nodes that the storage client is allowed to use in the storage group; and
storing the quantity of storage nodes in a first data unit in a first partition of the storage node group.
10. The non-volatile and non-transitory computer readable storage medium according to claim 9 , wherein the quantity of the one or more parity blocks is equal to M, and the quantity of the plurality of data blocks is equal to a value obtained by subtracting M from the quantity of the storage nodes the storage client is allowed to use.