Method and system for tree management of trees under multi-version concurrency control
Method for tree management of trees under multi-version concurrency control is described herein. Method starts by detecting change in a size of a cluster. The number of nodes in the cluster is counted to obtain the changed size of the cluster. The number of trees needed for the number of nodes in the cluster is determined. The number of trees may be based on the number of nodes in the cluster and predetermined system balancing coefficient. When the number of trees needed is greater than existing number of trees, existing number of trees is doubled as a single-step operation which includes modifying and scaling hash function used to derive each tree to generate new hash function and using new hash function to partition each tree instantly into two new trees. Scaling happens on demand without service disruption. Hash function scales automatically when number of trees increases. Other embodiments are described.
1. A method for tree management of trees under multi-version concurrency control comprising:
detecting a change in a size of a cluster;
counting a number of nodes in the cluster to obtain the changed size of the cluster;
determining a number of trees needed for the number of nodes in the cluster, wherein the number of trees is based on the number of nodes in the cluster and predetermined system balancing coefficient
determining whether the number of trees needed is greater than an existing number of trees; and
increasing the number of trees by doubling the existing number of trees as a single-step operation to obtain the number of trees needed when the number of trees needed is greater than the existing number of trees.
2. The method in claim 1 , wherein the number of trees is determined using a formula:
t=x* 2 ┌√{square root over (n)}┐
wherein t is the number of trees, x is the predetermined system balancing coefficient and n is the number of nodes in the cluster.
3. The method of claim 1 , wherein doubling the existing number of trees as a single-step operation to obtain the number of trees needed includes:
modifying and scaling a hash function used to derive each one of the trees to generate a new hash function and using the new hash function to partition each one of the trees instantly into two new trees, wherein the two new trees include a first new tree and a second new tree.
4. The method of claim 3 , wherein the scaling happens on demand without a service disruption, wherein the hash function scales automatically when the number of trees increases.
5. The method of claim 4 , further comprising performing a garbage collection of keys included in the two new trees, wherein performing the garbage collection includes:
scanning each of the two new trees;
assessing keys stored in tree leaves included in the two new trees to determine whether the keys belong to the first new tree or to the second new tree, wherein the new hash function is used to determine whether the keys belong to the first new tree or to the second new tree; and
deleting the keys belonging to the first new tree from the second new tree and deleting the keys belonging to the second new tree from the first new tree.
6. The method of claim 5 , wherein the new hash function is:
i=k mod t
wherein i is a tree index, k is a key value and t is the number of trees.
7. The method of claim 6 , wherein the number of trees is increased without a data unavailability window.
8. A computer-readable storage medium having stored thereon instructions, when executed by a processor, causes the processor to perform a method for tree management of trees under multi-version concurrency control comprising:
detecting a change in a size of a cluster;
counting a number of nodes in the cluster to obtain the changed size of the cluster;
determining a number of trees needed for the number of nodes in the cluster, wherein the number of trees is based on the number of nodes in the cluster and predetermined system balancing coefficient;
determining whether the number of trees needed is greater than an existing number of trees; and
increasing the number of trees by doubling the existing number of trees as a single-step operation to obtain the number of trees needed when the number of trees needed is greater than the existing number of trees.
9. The computer-readable storage medium in claim 8 , wherein the number of trees is determined using a formula:
t=x* 2 ┌√{square root over (n)}┐
wherein t is the number of trees, x is the predetermined system balancing coefficient and n is the number of nodes in the cluster.
10. The computer-readable storage medium of claim 8 , wherein doubling the existing number of trees as a single-step operation to obtain the number of trees needed includes:
modifying and scaling a hash function used to derive each one of the trees to generate a new hash function and using the new hash function to partition each one of the trees instantly into two new trees, wherein the two new trees include a first new tree and a second new tree.
11. The computer-readable storage medium of claim 10 , having stored thereon instructions, when executed by the processor, causes the processor to perform the method further comprising:
performing a garbage collection of keys included in the two new trees, wherein performing the garbage collection includes:
scanning each of the two new trees;
assessing keys stored in tree leaves included in the two new trees to determine whether the keys belong to the first new tree or to the second new tree, wherein the new hash function is used to determine whether the keys belong to the first new tree or to the second new tree; and
deleting the keys belonging to the first new tree from the second new tree and deleting the keys belonging to the second new tree from the first new tree.
12. The computer-readable storage medium of claim 11 , wherein the new hash function is:
i=k mod t
wherein i is a tree index, k is a key value and t is the number of trees.
13. A system for tree management of trees under multi-version concurrency control comprising:
a storage storing content of a plurality of trees in a system, each tree is associated with a cluster node, wherein a cluster includes a plurality of nodes;
a processor coupled to the storage; and
a tree manager coupled to the processor,
when the processor executes instructions stored in the storage, the processor signals to the tree manager to:
detect a change in a size of a cluster,
count a number of nodes in the cluster to obtain the changed size of the cluster;
determine a number of trees needed for the number of nodes in the cluster, wherein the number of trees is based on the number of nodes in the cluster and predetermined system balancing coefficient;
determine whether the number of trees needed is greater than an existing number of trees; and
increase the number of trees by doubling the existing number of trees as a single-step operation to obtain the number of trees needed when the number of trees needed is greater than the existing number of trees.
14. The system in claim 13 , wherein the number of trees is determined using a formula:
t=x* 2 ┌√{square root over (n)}┐
wherein t is the number of trees, x is the predetermined system balancing coefficient and n is the number of nodes in the cluster.
15. The system of claim 13 , wherein doubling the existing number of trees as a single-step operation to obtain the number of trees needed includes:
modifying and scaling a hash function used to derive each one of the trees to generate a new hash function and using the new hash function to partition each one of the trees instantly into two new trees, wherein the two new trees include a first new tree and a second new tree.
16. The system of claim 15 , wherein the tree manager further to:
perform a garbage collection of keys included in the two new trees including:
scanning each of the two new trees;
assessing keys stored in tree leaves included in the two new trees to determine whether the keys belong to the first new tree or to the second new tree, wherein the new hash function is used to determine whether the keys belong to the first new tree or to the second new tree; and
deleting the keys belonging to the first new tree from the second new tree and deleting the keys belonging to the second new tree from the first new tree.
17. The system of claim 16 , wherein the new hash function is:
i=k mod t
wherein i is a tree index, k is a key value and t is the number of trees.