Group based QoS policies for volumes
A technique provides efficient management of policies for objects of a distributed storage architecture configured to service storage requests issued by one or more clients of a storage cluster. The objects may include volumes for storing data served by storage nodes of the cluster and the policies may include quality of service (QoS) policies. The technique enables dynamic grouping of the volumes as management domains and applying attributes, such as performance settings of the QoS policies, to the management domains. A group of volumes may be organized as a management domain and a QoS policy may be applied to the domain. If membership of the management domain is modified, the QoS policy is automatically applied to the added volume or stripped from the removed volume. If a performance setting of the policy is modified, the modification is atomically applied and propagated to each volume of the management domain.
1 . A method comprising:
executing a service on a database of a storage cluster to implement a request issued by a client of the storage cluster to create a management domain as a data structure in a namespace of the database;
organizing, by the service, a plurality of volumes as members of the management domain, wherein the management domain is maintained as a persistent indirection object separate from metadata of the plurality of volumes and maintains attributes corresponding to the attributes of the plurality volumes including performance settings of one or more quality of service (QoS) policies;
modifying attributes of the management domain;
invoking, in response to modification of the attributes of the management domain, one or more callback functions registered with the management domain to propagate corresponding attribute changes to each volume of the plurality of volumes;
providing automatic and atomic updates to the attributes of each volume of the plurality of volumes via the invoked callback functions; and
distributing enforcement of the one or more QoS policies at a level of service for the plurality of volumes independently of direct modification of per-volume metadata.
2 . The method of claim 1 , wherein providing automatic and atomic updates to the attributes of each volume of the management domain further comprises invoking a callback function to propagate the corresponding attribute changes to each volume of the plurality of volumes.
3 . The method of claim 2 , wherein invoking the callback function provides the automatic and atomic updates to the plurality of volumes.
4 . The method of claim 2 , further comprising executing the service on the database to invoke the callback function.
5 . The method of claim 2 , further comprising atomically modifying the plurality of volumes in tandem via one or more callback functions.
6 . The method of claim 1 , wherein each volume of the plurality of volumes is associated with a management domain identifier stored in a data structure associated with a respective volume of the plurality of volumes.
7 . The method of claim 1 , wherein the management domain is associated with a snapshot data structure configured to group snapshots of the plurality of volumes across the storage cluster.
8 . The method of claim 1 , wherein the database comprises a distributed share-nothing database across a plurality of nodes of the storage cluster.
9 . The method of claim 1 further comprising in response to removing a volume from the plurality of volumes, invoking a callback function to remove the level of indirection provided by the management domain.
10 . The method of claim 1 , further comprising responsive to a change in membership of the management domain, atomically applying the corresponding attributes to any volumes added to the plurality of volumes as part of the change in membership and eliminating application of the corresponding attributes to any volumes removed from the plurality of volumes as part of the change in membership.
11 . A system comprising:
a plurality of nodes of a storage cluster;
each node having a processor coupled to one or more storage devices, the processor executing instructions that cause the system to,
execute a service on a database of the storage cluster to implement a request issued by a client of the storage cluster to create a management domain as a data structure in a namespace of the database;
organize a plurality of volumes as members of the management domain, wherein the management domain is maintained as a persistent indirection object separate from metadata of the plurality of volumes and maintains attributes corresponding to the attributes of the plurality volumes including performance settings of one or more quality of service (QoS) policies;
modify attributes of the management domain;
invoke, in response to modification of the attributes of the management domain, one or more callback functions registered with the management domain to propagate corresponding attribute changes to each volume of the plurality of volumes;
provide automatic and atomic updates to the attributes of each volume of the plurality of volumes via the invoked callback functions; and
distribute enforcement of the one or more QoS policies at a level of service for the plurality of volumes independently of direct modification of per-volume metadata.
12 . The system of claim 11 , wherein the instructions further cause the system to invoke a callback function to propagate the corresponding attribute changes to each volume of the plurality of volumes.
13 . The system of claim 12 , wherein invoking the callback function provides the automatic and atomic updates to the plurality of volumes.
14 . The system of claim 12 , wherein the instructions further cause the system to execute the service on the database to invoke the callback function.
15 . The system of claim 12 , wherein the instructions further cause the system to atomically modify the plurality of volumes in tandem via one or more callback functions.
16 . The system of claim 11 , wherein each volume of the plurality of volumes is associated with a management domain identifier stored in a data structure associated with a respective volume of the plurality of volumes.
17 . The system of claim 11 , wherein the management domain is associated with a snapshot data structure configured to group snapshots of the plurality of volumes across the storage cluster.
18 . The system of claim 11 , wherein the instructions further cause the system to in response to removing a volume from the plurality of volumes, invoke a callback function to remove the level of indirection provided by the management domain.
19 . The system of claim 11 , wherein the instructions further cause the system to responsive to a change in membership of the management domain, atomically apply the corresponding attributes to any volumes added to the plurality of volumes as part of the change in membership and eliminating application of the corresponding attributes to any volumes removed from the plurality of volumes as part of the change in membership.
20 . A non-transitory computer readable medium having program instructions that when executed by a processor of a storage cluster cause the storage cluster to:
execute a service on a database of the storage cluster to implement a request issued by a client of the storage cluster to create a management domain as a data structure in a namespace of the database;
organize a plurality of volumes as members of the management domain, wherein the management domain is maintained as a persistent indirection object separate from metadata of the plurality of volumes and maintains attributes corresponding to the attributes of the plurality volumes including performance settings of one or more quality of service (QoS) policies;
modify attributes of the management domain;
invoke, in response to modification of the attributes of the management domain, one or more callback functions registered with the management domain to propagate corresponding attribute changes to each volume of the plurality of volumes;
provide automatic and atomic updates to the attributes of each volume of the plurality of volumes via the invoked callback functions; and
distribute enforcement of the one or more QoS policies at a level of service for the plurality of volumes independently of direct modification of per-volume metadata.
21 . The non-transitory computer readable medium of claim 20 , wherein the instructions further cause the storage cluster to responsive to a change in membership of the management domain, atomically apply the corresponding attributes to any volumes added to the plurality of volumes as part of the change in membership and eliminating application of the corresponding attributes to any volumes removed from the plurality of volumes as part of the change in membership.