Policy-based storage structure distribution
View Patent ↗Policy-based storage and retrieval combined with a distribution algorithm results in automatic and even distribution of policy-based storage structures across a set of nodes and dynamic, automated homing or ownership of policy-based storage structures. Large numbers of policy-based storage structures may be distributed without manual administration, allowing for rapid creation and destruction of storage structures. The overall load may be distributed and balanced across the server pool. Multiple entries having the same key value in a database- or table-like structure allow for distribution of policy-based storage and retrieval by key value and for queue semantics to be utilized for microqueues in the large database- or table-like structure.
1. A method of adding a node to a collection of computing nodes, the method comprising:
joining a new node to the collection of computing nodes as a leech node;
receiving, at the leech node, information from other active nodes in the collection of computing nodes;
synchronizing the leech node based on the received information, the leech node becoming substantially synchronized before promoting the leech node to an active node;
triggering, by the leech node, a synchronized re-join process; and
in response to triggering the synchronized re-join process, promoting, by the leech node, the leech node to an active node in the collection of computing nodes,
wherein the joining the new node as a leech node comprises the leech node obtaining a key space from the other active nodes in the collection of computing nodes,
wherein the leech node is not an active node, and
wherein the receiving information comprises receiving copies of key values from other active nodes, the key values being key values whose ownership will be transferred from the other active nodes to the leech node once the leech node becomes an active node, wherein the receiving information further comprises the leech node monitoring the key values.
2. The method of claim 1 , wherein the joining the new node as a leech node comprises the leech node interacting with the other active nodes in the collection of computing nodes.
3. The method of claim 1 , wherein the receiving information further comprises reconciling data between the leech node and the other active nodes in the collection of computing nodes.
4. The method of claim 1 , wherein receiving information further comprises calculating, using a consistent hashing algorithm, a distribution of key values over the collection of computing nodes including the new node.
5. The method of claim 1 , wherein the leech node takes ownership of the key space in response to triggering the synchronized re-join process.
6. The method of claim 1 , further comprising temporarily suspending the key space during promoting the leech node to the active node.
7. A system of adding a node to a collection of computing nodes, the system configured for performing:
joining a new node to the collection of computing nodes as a leech node;
receiving, at the leech node, information from other active nodes in the collection of computing nodes;
synchronizing the leech node based on the received information, the leech node becoming substantially synchronized before promoting the leech node to an active node;
triggering, by the leech node, a synchronized re-join process; and
in response to triggering the synchronized re-join process, promoting, by the leech node, the leech node to an active node in the collection of computing nodes,
wherein the joining the new node as a leech node comprises the leech node obtaining a key space from the other active nodes in the collection of computing nodes,
wherein the leech node is not an active node, and
wherein the receiving information comprises receiving copies of key values from other active nodes, the key values being key values whose ownership will be transferred from the other active nodes to the leech node once the leech node becomes an active node, wherein the receiving information further comprises the leech node monitoring the key values.