Mapped cluster stretching for increasing workload in a data storage system
The described technology is generally directed towards stretching a mapped storage clusters by adding nodes to a mapped cluster of mapped nodes and storage devices mapped to a real cluster of nodes and storage devices. Stretching the mapped cluster can provide additional computing resources to a set of storage devices. In one implementation, one or more newly mapped nodes are added to increase the node count of an existing mapped cluster to form a stretched cluster, with the storage devices distributed among the increased number of nodes; a mapping table is updated to relate the stretched cluster nodes and storage devices to the real cluster nodes and storage devices. Also described is un-stretching a stretched cluster, or further stretching a stretched cluster.
1. A method, comprising,
modifying, by a system comprising a processor, a first mapped cluster comprising a first group of first mapped nodes and mapped storage devices of a real cluster into a second mapped cluster comprising a second group of second mapped nodes and the mapped storage devices, in which the second group of second mapped nodes comprises an increased number of mapped nodes relative to the first group of first mapped nodes;
distributing the mapped storage devices between the second mapped nodes of the second group of second mapped nodes;
based on a cluster stretching logic, processing a sample data set to sense a first request for additional mapped nodes; and
based on the first request for the additional mapped nodes being sensed, generating a second request to increase the first group of first mapped nodes from a first number of mapped nodes to a second number of mapped nodes.
2. The method of claim 1 , wherein the distributing the mapped storage devices comprises updating a mapping data structure that relates the first mapped cluster to the real cluster into an updated mapping data structure that relates the second mapped cluster to the real cluster.
3. The method of claim 2 , further comprising reading data from a real node of the real cluster based on the updated mapping data structure.
4. The method of claim 1 , wherein the second group of second mapped nodes has an increased number of the mapped nodes relative to the first group of first mapped nodes by way of adding a new mapped node, and wherein the distributing the mapped storage devices comprises reassigning a mapped storage device from a mapped node of the first group of first mapped nodes to the new mapped node in the second group of second mapped nodes.
5. The method of claim 1 , wherein the distributing the mapped storage devices comprises distributing the mapped storage devices evenly among the second group of second mapped nodes of the second mapped cluster.
6. The method of claim 1 , wherein the second group of second mapped nodes has an increased number of the mapped nodes relative to the first group of first mapped nodes by way of adding a new mapped node, and wherein the distributing the mapped storage devices comprises reassigning two or more mapped storage devices from a mapped node of the first group of first mapped nodes to the new mapped node in the second group of second mapped nodes.
7. The method of claim 1 , wherein the distributing the mapped storage devices comprises distributing the mapped storage devices based on relative computing capabilities of the second mapped nodes of the second mapped cluster.
8. The method of claim 1 , further comprising:
remodifying the second mapped cluster into a third group of third mapped nodes and the mapped storage devices, in which the third group of third mapped nodes has a decreased number of mapped nodes relative to the second group of second mapped nodes, and
distributing the mapped storage devices between third mapped nodes of the third group of mapped nodes.
9. The method of claim 1 , further comprising:
remodifying the second mapped cluster into the first group of first mapped nodes and the mapped storage devices, and
distributing the mapped storage devices between the first mapped nodes of the first group of first mapped nodes.
10. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:
maintaining a mapping data structure that relates a mapped cluster comprising mapped storage devices and a first group of first mapped nodes to real storage devices and real nodes, respectively, of a real cluster;
based on a cluster stretching logic, processing a sample data set to sense a first request for additional mapped nodes for the real cluster;
increasing a first number of the first mapped nodes of the first group of first mapped nodes to provide a stretched mapped cluster comprising a second number of second mapped nodes of a second group of second mapped nodes;
distributing the mapped storage devices between the second group of second mapped nodes;
based on detecting a second request to decrease the stretched mapped cluster to a third number of third mapped nodes of a third group of third mapped nodes less than the second number, prompting an administrator device associated with an administrator identity to decrease the stretched mapped cluster to the third number, resulting in a modified mapped cluster comprising the third group of third mapped nodes;
updating the mapping data structure into an updated mapping data structure that relates the modified mapped cluster, comprising the mapped storage devices and the third group of third mapped nodes, to the real cluster; and
accessing the updated mapping data structure to relate first input-output requests directed to the modified mapped cluster to data locations in the real cluster.
11. The system of claim 10 , wherein the distributing the mapped storage devices comprises distributing the mapped storage devices evenly among the second group of second mapped nodes of the stretched mapped cluster.
12. The system of claim 10 , wherein the operations further comprise a modifying the third number of the third mapped nodes of the third group to revert to the mapped cluster comprising the first group of first mapped nodes, distributing the mapped storage devices between the first group of first mapped nodes, updating the updated mapping data structure to return to the mapping data structure that relates the mapped storage devices and the first group of first mapped nodes to the real cluster, and accessing the mapping data structure to relate second input-output requests directed to the mapped cluster to the data locations in the real cluster.
13. The system of claim 10 , wherein the increasing the number of the first mapped nodes comprises adding a new mapped node, and wherein the distributing the mapped storage devices comprises reassigning two or more mapped storage devices from a mapped node of the first group of first mapped nodes to the new mapped node in the second group of second mapped nodes.
14. The system of claim 10 , wherein the distributing the mapped storage devices comprises distributing the mapped storage devices based on relative computing capabilities of the second mapped nodes of the second group.
15. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, the operations comprising:
based on a cluster stretching logic, processing a sample data set to sense a first request for additional mapped nodes that map to a real cluster;
increasing a first number of first mapped nodes of a first group of first mapped nodes that map to the real cluster to provide a stretched mapped cluster comprising a second group of second mapped nodes that mapped to the real cluster, the second group of second mapped nodes having a second number of second mapped nodes that is greater than the first number;
distributing mapped storage devices that map to storage devices of the real cluster among the second group of second mapped nodes;
sending a second request to an administrator device associated with an administrator identity to decrease the second group of second mapped nodes from the second number of second mapped nodes to a third number of second mapped nodes, wherein the second group being decreased results in a modified mapped cluster comprising a modified second group of second mapped nodes with the third number of second mapped nodes; and
updating a mapping data structure that relates the mapped storage devices and the first group of first mapped nodes to the real cluster into an updated mapping data structure that relates the modified mapped cluster, comprising the mapped storage devices and the modified second group of second mapped nodes, to the real cluster.
16. The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise accessing the updated mapping data structure to relate input-output requests directed to the modified mapped cluster to data locations in the real cluster.
17. The non-transitory machine-readable medium of claim 15 , wherein the distributing the mapped storage devices comprises distributing the mapped storage devices evenly or substantially evenly among the second group of the second mapped nodes of the second mapped cluster.
18. The non-transitory machine-readable medium of claim 15 , wherein the second group of second mapped nodes has an increased number of mapped nodes relative to the first group of first mapped nodes by way of adding a new mapped node, and wherein the distributing the mapped storage devices comprises reassigning two or more mapped storage devices from a mapped node of the first group of first mapped nodes to the new mapped node in the second group of second mapped nodes.
19. The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise modifying the modified mapped cluster into a third group of third mapped nodes and the mapped storage devices, in which the third group of third mapped nodes has an increased number of mapped nodes relative to the third number of the modified second group of second mapped nodes, and distributing the mapped storage devices between the third mapped nodes of the third group of mapped nodes.
20. The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise modifying the modified mapped cluster into a third group of third mapped nodes and the mapped storage devices, in which the third group of third mapped nodes has a decreased number of mapped nodes relative to the third number of the modified second group of second mapped nodes, and distributing the mapped storage devices between the third mapped nodes of the third group of third mapped nodes.