Automatic space sharing of disaggregated storage of a storage pod by multiple nodes of a distributed storage system
Systems and methods are provided for implementation and use of disaggregated storage by a distributed storage system. In various examples described herein, the association of blocks of storage space within a storage pod with a given dynamically extensible file system (DEFS) may be in large chunks of one or more gigabytes (GB), which may be referred to herein as “allocation areas” (AAs) that each include multiple RAID stripes. The use of large, multi-GB chunks, as the unit of space allocation/assignment to DEFSs of a distributed storage system facilitates ease of management and independence of write allocation. The use of AAs described herein further allows disk space to be used more fluidly across individual (nodes) storage systems of a distributed storage system (cluster of nodes working together), thereby eliminating silos of storage; and processing resource (e.g., CPU) load may be distributed across the cluster.
1 . A method comprising:
providing a storage pod having a group of disks containing multiple Redundant Array of Independent Disks (RAID) groups, wherein storage space of the storage pod is accessible concurrently to all nodes of a plurality of nodes of a cluster, wherein the cluster represents a distributed storage system, and wherein the storage space is partitioned into a plurality of allocation areas (AAs), and wherein each AA of the plurality of AAs includes multiple RAID stripes of a given RAID group of the multiple RAID groups;
monitoring, by a node of the cluster, storage space availability or usage by one or more dynamically extensible file systems (DEFSs) of the node, wherein ownership of a given set of one or more AAs of the plurality of AAs is assigned to respective DEFSs of the plurality of nodes;
based on the storage space availability or usage meeting a predetermined or configurable threshold in relation to storage space availability or usage of one or more DEFSs of one or more other nodes of the plurality of nodes of the cluster, requesting ownership of one or more of the plurality of AAs currently owned by the one or more DEFSs of the one or more other nodes be transferred to a DEFS of the one or more DEFSs of the node.
2 . The method of claim 1 , wherein said requesting ownership includes sending a space request message from the node on behalf of a donee DEFS of the one or more DEFSs of the node to a donor DEFS of the one or more DEFSs of a second node of the one or more other nodes.
3 . The method of claim 2 , wherein the space request message is sent via an on-wire internode communication mechanism to the second node.
4 . The method of claim 2 , wherein the space request message is posted to a persistent message queue of the donor DEFS or the second node.
5 . The method of claim 2 , further comprising after receiving the space request message, selecting, by the donor DEFS, an AA to donate to the donee DEFS from among a plurality of AAs owned by the donor DEFS by prioritizing a free AA of the plurality of AAs owned by the donor DEFS over a partial AA of the plurality of AAs owned by the donor DEFS, wherein the free AA includes no blocks marked as used, and wherein the partial AA includes one or more blocks that are marked as used.
6 . The method of claim 5 , further comprising sending an AA ownership change message from the second node to the node, wherein the AA ownership change message includes an AA identifier of the selected AA.
7 . A non-transitory machine readable medium storing instructions, which when executed by one or more processing resources of a distributed storage system, cause the distributed storage system to:
provide a storage pod having a group of disks containing multiple Redundant Array of Independent Disks (RAID) groups, wherein storage space of the storage pod is accessible concurrently to all nodes of a plurality of nodes of a cluster, wherein the cluster represents a distributed storage system, and wherein the storage space is partitioned into a plurality of allocation areas (AAs), and wherein each AA of the plurality of AAs includes multiple RAID stripes of a given RAID group of the multiple RAID groups;
monitor storage space availability or usage by one or more dynamically extensible file systems (DEFSs) of a node of the plurality of nodes, wherein ownership of a given set of one or more AAs of the plurality of AAs is assigned to respective DEFSs of the plurality of nodes;
based on the storage space availability or usage meeting a predetermined or configurable threshold in relation to storage space availability or usage of one or more DEFSs of one or more other nodes of the plurality of nodes of the cluster, requesting ownership of one or more of the plurality of AAs currently owned by the one or more DEFSs of the one or more other nodes be transferred to a DEFS of the one or more DEFSs of the node.
8 . The non-transitory machine readable medium of claim 7 , wherein requesting transfer of ownership includes sending a space request message from the node on behalf of a donee DEFS of the one or more DEFSs of the node to a donor DEFS of the one or more DEFSs of a second node of the one or more other nodes.
9 . The non-transitory machine readable medium of claim 8 , wherein the space request message is sent via an on-wire internode communication mechanism to the second node.
10 . The non-transitory machine readable medium of claim 8 , wherein the space request message is posted to a persistent message queue of the donor DEFS or the second node.
11 . The non-transitory machine readable medium of claim 10 , wherein the instructions further cause the distributed storage system to, after receipt of the space request message, select, by the donor DEFS, an AA to donate to the donee DEFS from among a plurality of AAs owned by the donor DEFS by prioritizing a free AA of the plurality of AAs owned by the donor DEFS over a partial AA of the plurality of AAs owned by the donor DEFS, wherein the free AA includes no blocks marked as used, and wherein the partial AA includes one or more blocks that are marked as used.
12 . The non-transitory machine readable medium of claim 11 , wherein the instructions further cause the distributed storage system to send an AA ownership change message from the second node to the node, wherein the AA ownership change message includes an AA identifier of the selected AA.
13 . A distributed storage system comprising:
one or more processing resources; and
instructions that when executed by the one or more processing resources cause the distributed storage system to:
provide a storage pod having a group of disks containing multiple Redundant Array of Independent Disks (RAID) groups, wherein storage space of the storage pod is accessible concurrently to all nodes of a plurality of nodes of a cluster, wherein the cluster represents a distributed storage system, and wherein the storage space is partitioned into a plurality of allocation areas (AAs), and wherein each AA of the plurality of AAs includes multiple RAID stripes of a given RAID group of the multiple RAID groups;
monitor storage space availability or usage by one or more dynamically extensible file systems (DEFSs) of a node of the plurality of nodes, wherein ownership of a given set of one or more AAs of the plurality of AAs is assigned to respective DEFSs of the plurality of nodes;
based on the storage space availability or usage meeting a predetermined or configurable threshold in relation to storage space availability or usage of one or more DEFSs of one or more other nodes of the plurality of nodes of the cluster, request ownership of one or more of the plurality of AAs currently owned by the one or more DEFSs of the one or more other nodes be transferred to a DEFS of the one or more DEFSs of the node.
14 . The distributed storage system of claim 13 , wherein requesting transfer of ownership includes sending a space request message from the node on behalf of a donee DEFS of the one or more DEFSs of the node to a donor DEFS of the one or more DEFSs of a second node of the one or more other nodes.
15 . The distributed storage system of claim 14 , wherein the space request message is sent via an on-wire internode communication mechanism to the second node.
16 . The distributed storage system of claim 14 , wherein the space request message is posted to a persistent message queue of the donor DEFS or the second node.
17 . The distributed storage system of claim 14 , wherein the instructions further cause the distributed storage system to, after receipt of the space request message, select, by the donor DEFS, an AA to donate to the donor DEFS from among a plurality of AAs owned by the donee DEFS by prioritizing a free AA of the plurality of AAs owned by the donor DEFS over a partial AA of the plurality of AAs owned by the donor DEFS, wherein the free AA includes no blocks marked as used, and wherein the partial AA includes one or more blocks that are marked as used.
18 . The distributed storage system of claim 17 , wherein the instructions further cause the distributed storage system to send an AA ownership change message from the second node to the node, wherein the AA ownership change message includes an AA identifier of the selected AA.