Goal-oriented storage management for a distributed data storage network
View Patent ↗A system and method for performing goal-oriented storage management for a distributed data storage network. Storage management software may execute on each node to cause the distributed data storage network to converge toward a system-wide storage goal. In various embodiments, the system-wide storage goal may be defined to accomplish any of various kinds of optimizations or performance improvements for the system, such as balancing storage utilization, achieving appropriate data replication, increasing performance, increasing fault tolerance, etc.
1. A system for distributed data storage, the system comprising:
a plurality of computing nodes, wherein
each node of the plurality of computing nodes includes memory and one or more processors,
the each node of the plurality of computing nodes is configured to execute storage management software that causes the system to converge toward a system-wide storage goal, wherein
the storage management software is configured to implement role-based addressing such that the plurality of nodes do not need to maintain information regarding a global state of the system,
the memory of each respective node of the plurality of computing nodes stores program instructions executable by the one or more processors of the respective node to determine a local state of the respective node, and
at least one node of the plurality of computing nodes is associated with a corresponding exclusive role of a plurality of exclusive roles and at least two nodes of the plurality of computing nodes are associated with a shared role; and
a goal-oriented storage manager configured to
select a first node group from a plurality of node groups, wherein
the first node group corresponds to the local state of the respective node, and
the first node group is associated with a resource role,
add the respective node to the first node group,
receive information regarding each node group of the plurality of node groups,
determine states of other nodes of the plurality of computing nodes based on the information regarding each node group, and
depending upon the local state of the respective node and the states of the other nodes relative to the system-wide storage goal, perform an action to modify the local state of the respective node to conform the local state to the system-wide storage goal such that an optimal state for the distributed data storage system is identified, wherein
the optimal state is defined by at least one goal metric,
the plurality of exclusive roles are defined to represent a range of utilization of a plurality of node resources, and
the distributed data storage system converges towards the optimal state based on the system-wide storage goal.
2. The system of claim 1 ,
wherein each respective node group of the plurality of node groups corresponds to a respective set of node states.
3. The system of claim 1 ,
wherein the program instructions are further executable by the one or more processors of the respective node to:
determine a change in the local state of the respective node;
perform the following in response to determining the change in the local state of the respective node:
select a second node group from the plurality of node groups, wherein the second node group corresponds to the changed local state of the respective node;
remove the respective node from the first node group; and
add the respective node to the second node group.
4. The system of claim 1 ,
wherein the plurality of node groups includes a second node group;
wherein receiving the information regarding each node group of the plurality of node groups comprises receiving information indicating one or more nodes in the second node group;
wherein the program instructions are further executable by the one or more processors of the respective node to select a particular node in the second node group;
wherein performing the action to modify the local state of the respective node comprises the respective node communicating with the particular node in the second node group to perform the action.
5. The system of claim 1 ,
wherein the program instructions are further executable by the one or more processors of the respective node to:
determine an average utilization amount of resources for the plurality of computing nodes based on the local state of the respective node and the states of the other nodes; and
determine that a utilization amount of a resource on the respective node is greater than the average utilization amount for the plurality of computing nodes;
wherein performing the action to modify the local state of the respective node comprises performing an action to decrease the utilization amount of the resource on the respective node.
6. The system of claim 1 ,
wherein the program instructions are further executable by the one or more processors of the respective node to:
determine an average utilization amount of resources for the plurality of nodes based on the local state of the respective node and the states of the other nodes; and
determine that a utilization amount of a resource on the respective node is less than the average utilization amount for the plurality of computing nodes;
wherein performing the action to modify the local state of the respective node comprises performing an action to increase the utilization amount of the resource on the respective node.
7. The system of claims 1 ,
wherein performing the action to modify the local state of the respective node comprises one or more of:
initiating a transfer operation to transfer data to the respective node from another node;
initiating a transfer operation to transfer data from the respective node to another node; and
deleting data from the respective node.
8. The system of claim 1 ,
wherein the plurality of computing nodes operate together a peer-to-peer manner to converge upon the system-wide storage goal.
9. The system of claim 1 ,
wherein each node group has a corresponding network address;
wherein receiving the information regarding each node group comprises receiving the information in response to sending one or more messages to each of the network addresses corresponding to the node groups.
10. The system of claim 1 ,
wherein the system-wide storage goal comprises a goal of balancing a percentage of storage utilized by the nodes;
wherein determining the local state of the respective node comprises determining a percentage of storage utilized by the respective node;
wherein determining the states of the other nodes comprises determining information indicative of percentages of storage utilized by the other nodes;
wherein performing the action to modify the local state of the respective node comprises performing an action to modify an amount of data stored by the respective node.
11. The system of claims 10 ,
wherein performing the action to modify the amount of data stored by the respective node comprises transferring data to the respective node from a node with a higher percentage of utilized storage.
12. The system of claim 1 ,
wherein each respective node repeatedly performs said operations to cause the system to converge upon the system-wide storage goal.
13. The system of claim 1 wherein the plurality of computing nodes comprise a plurality of role instances associable with each respective node of the plurality of computing nodes such that each respective node maintains a list of role instances associated with the plurality of computing nodes.
14. The system of claim 1 wherein the goal-oriented storage manager is configured to be executed by one of the plurality of computing nodes and is further configured to be managed by, among other things, the plurality of node groups.
15. The system of claim 1 , wherein,
the respective node adds or removes itself from the first node group based on the range of availability or utilization of the plurality of node resources.
16. A computer-readable memory medium comprising program instructions executable to implement a method comprising:
each respective node of a plurality of nodes performing the following:
executing storage management software that causes a distributed data storage system to converge towards a system-wide storage goal, wherein
the storage management software is configured to implement role-based addressing such that the plurality of nodes do not need to maintain information regarding a global state of the system,
determining a local state of the respective node wherein at least one node of the plurality of computing nodes is associated with a corresponding exclusive role of a plurality of exclusive roles and at least two nodes of the plurality of computing nodes are associated with a shared role;
selecting a first node group from a plurality of node groups, wherein
the first node group corresponds to the local state of the respective node, and
the first node group in associated with a resource role;
adding the respective node to the first node group;
receiving information regarding each node group of the plurality of node groups;
determining states of other nodes of the plurality of nodes based on the information regarding each node group; and
depending upon the local state of the respective node and the states of the other nodes relative to the system-wide storage goal, performing an action to modify the local state of the respective node to conform the local state to the system-wide storage goal such that an optimal state for the distributed data storage system is identified, wherein
the optimal state is defined by at least one goal metric,
the plurality of exclusive roles are defined to represent a range of utilization of a plurality of node resources, and
the distributed data storage system converges towards the optimal state based on the system-wide storage goal.
17. The computer-readable memory medium of claim 16 ,
wherein each respective node group of the plurality of node groups corresponds to a respective set of node states.
18. The computer-readable memory medium of claim 16 ,
wherein the method implemented by the program instructions further comprises:
each respective node of the plurality of nodes performing the following:
determining a change in the local state of the respective node;
performing the following in response to determining the change in the local state of the respective node:
selecting a second node group from the plurality of node groups, wherein the second node group corresponds to the changed local state of the respective node;
removing the respective node from the first node group; and
adding the respective node to the second node group.
19. The computer-readable memory medium of claim 16 ,
wherein the plurality of node groups includes a second node group;
wherein receiving the information regarding each node group of the plurality of node groups comprises receiving information indicating one or more nodes in the second node group;
wherein the method implemented by the program instructions further comprises selecting a particular node in the second node group;
wherein performing the action to modify the local state of the respective node comprises the respective node communicating with the particular node in the second node group to perform the action.
20. The computer-readable memory medium of claim 16 ,
wherein the method implemented by the program instructions further comprises:
determining an average utilization amount of resources for the plurality of nodes based on the local state of the respective node and the states of the other nodes; and
determining that a utilization amount of a resource on the respective node is greater than the average utilization amount for the plurality of nodes;
wherein performing the action to modify the local state of the respective node comprises performing an action to decrease the utilization amount of the resource on the respective node.
21. The computer-readable memory medium of claim 16 ,
wherein the method implemented by the program instructions further comprises:
determining an average utilization amount of resources for the plurality of nodes based on the local state of the respective node and the states of the other nodes; and
determining that a utilization amount of a resource on the respective node is less than the average utilization amount for the plurality of nodes;
wherein performing the action to modify the local state of the respective node comprises performing an action to increase the utilization amount of the resource on the respective node.
22. The computer-readable memory medium of claim 16 ,
wherein each node group has a corresponding network address;
wherein receiving the information regarding each node group comprises receiving the information in response to sending one or more messages to each of the network addresses corresponding to the node groups.
23. A method comprising:
each respective node of a plurality of nodes performing the following:
executing storage management software that causes a distributed data storage system to converge towards a system-wide storage goal, wherein
the storage management software is configured to implement role-based addressing such that the plurality of nodes do not need to maintain information regarding a global state of the system,
determining a local state of the respective node wherein at least one node of the plurality of computing nodes is associated with a corresponding exclusive role of a plurality of exclusive roles and at least two nodes of the plurality of computing nodes is are associated with a shared role;
selecting a first node group from a plurality of node groups, wherein
the first node group corresponds to the local state of the respective node, and
the first node group is associated with a resource role;
adding the respective node to the first node group;
receiving information regarding each node group of the plurality of node groups;
determining states of other nodes of the plurality of nodes based on the information regarding the node groups; and
depending upon the local state of the respective node and the states of the other nodes relative to the system-wide storage goal, performing an action to modify the local state of the respective node to conform the local state to the system-wide storage goal such that an optimal state for the distributed data storage system is identified, wherein
the optimal state is defined by at least one goal metric,
the plurality of exclusive roles are defined to represent a range of utilization of a plurality of node resources, and
the distributed data storage system converges towards the optimal state based on the system-wide storage goal.