Cascading payload replication
Cascading payload replication to target compute nodes is disclosed. Cascading payload replication can be accomplished using a two-stage operation for a replication operation. In the first stage, a plan is generated and distributed for the replication operation. The plan includes an assignment of compute nodes to tree nodes in a tree hierarchy. In the second phase, the payload is distributed according to the plan. The plan is different for at least two replication operations. Thus, the cascading payload replication is adaptable to changing target compute nodes and provides for load balancing.
1. A computer implemented method comprising:
determining a plurality of payload groups, each payload group of the plurality of payload groups comprising a corresponding payload;
partitioning a plurality of target compute nodes into sets of target compute nodes based on the plurality of payload groups; and
performing a first replication operation for each payload group of the plurality of payload groups, wherein performing the first replication operation for a payload group comprises:
generating, for the first replication operation, a plan for replicating the corresponding payload on a set of target compute nodes corresponding to the payload group, the plan comprising an assignment of the set of target compute nodes to a plurality of tree nodes in a tree hierarchy,
distributing the plan to the set of target compute nodes, and
distributing, according to the plan, the corresponding payload to the set of target compute nodes in an order defined by the plan.
2. The computer implemented method of claim 1 , further comprising:
generating the assignment by, for at least a subset of the plurality of tree nodes, randomly selecting a target compute node from at least a subset of the set of target compute nodes and assigning the target compute node to a tree node in the subset of the plurality of tree nodes.
3. The computer implemented method of claim 1 , further comprising:
generating the assignment by partitioning the plurality of target compute nodes into a plurality of partitions based on an attribute of the plurality of target compute nodes and assigning each of the plurality of partitions to a corresponding subtree of the tree hierarchy.
4. The computer implemented method of claim 1 , wherein the plurality of payload groups is based on a last update of each of the plurality of target compute nodes.
5. The computer implemented method of claim 1 , wherein the corresponding payload for a first payload group of the plurality of payload groups is a delta payload, and the corresponding payload for a first payload group of the plurality of payload groups is a full payload.
6. The computer implemented method of claim 1 , further comprising:
generating a table mapping a tree height to a number of replication threads and number of compute nodes in the set of compute nodes, wherein the table defines the tree height such that a resulting tree has an initial fanout equal to the number of replication threads, and the resulting tree is a complete tree before a leaf level of the tree hierarchy.
7. The computer implemented method of claim 1 , further comprising:
generating the tree hierarchy by:
determining a tree height of the tree hierarchy from a table mapping the tree height to a number of replication threads and a number of compute nodes, wherein the number of compute nodes is a total of the number of the set of target compute nodes and a source compute node, and
setting a fanout of the tree hierarchy as a floor of the number of compute nodes to a power of (1/tree height−1), wherein the tree height is greater than one, and the floor is a function that rounds down to a nearest whole number.
8. The computer implemented method of claim 1 , further comprising:
performing a second replication operation for replicating a payload across the plurality of target compute nodes.
9. The computer implemented method of claim 1 , further comprising:
performing a second replication operation for replicating a payload across the plurality of target compute nodes, wherein the sets of target compute nodes are different between the first replication operation and the second replication operation.
10. A computing device, comprising:
a processor; and
a non-transitory computer-readable medium having stored thereon instructions that, when executed by the processor, cause the processor to perform operations including:
determining a plurality of payload groups, each payload group of the plurality of payload groups comprising a corresponding payload,
partitioning a plurality of target compute nodes into sets of target compute nodes based on the plurality of payload groups, and
performing a first replication operation for each payload group of the plurality of payload groups, wherein performing the first replication operation for a payload group comprises:
generating, for the first replication operation, a plan for replicating the corresponding payload on a set of target compute nodes corresponding to the payload group, the plan comprising an assignment of the set of target compute nodes to a plurality of tree nodes in a tree hierarchy,
distributing the plan to the set of target compute nodes, and
distributing, according to the plan, the corresponding payload to the set of target compute nodes in an order defined by the plan.
11. The computing device of claim 10 , wherein the operations further comprise:
generating the assignment by, for at least a subset of the plurality of tree nodes, randomly selecting a target compute node from at least a subset of the set of target compute nodes and assigning the target compute node to a tree node in the subset of the plurality of tree nodes.
12. The computing device of claim 10 , wherein the operations further comprise:
generating the assignment by partitioning the plurality of target compute nodes into a plurality of partitions based on an attribute of the plurality of target compute nodes and assigning each of the plurality of partitions to a corresponding subtree of the tree hierarchy.
13. The computing device of claim 10 , wherein the plurality of payload groups is based on a last update of each of the plurality of target compute nodes.
14. The computing device of claim 10 , wherein the corresponding payload for a first payload group of the plurality of payload groups is a delta payload, and the corresponding payload for a first payload group of the plurality of payload groups is a full payload.
15. The computing device of claim 10 , wherein the operations further comprise:
generating a table mapping a tree height to a number of replication threads and number of compute nodes in the set of compute nodes, wherein the table defines the tree height such that a resulting tree has an initial fanout equal to the number of replication threads, and the resulting tree is a complete tree before a leaf level of the tree hierarchy.
16. The computing device of claim 10 , wherein the operations further comprise:
generating the tree hierarchy by:
determining a tree height of the tree hierarchy from a table mapping the tree height to a number of replication threads and a number of compute nodes, wherein the number of compute nodes is a total of the number of the set of target compute nodes and a source compute node, and
setting a fanout of the tree hierarchy as a floor of the number of compute nodes to a power of (1/tree height−1), wherein the tree height is greater than one, and the floor is a function that rounds down to a nearest whole number.
17. The computing device of claim 10 , wherein the operations further comprise:
performing a second replication operation for replicating a payload across the plurality of target compute nodes.
18. The computing device of claim 10 , wherein the operations further comprise:
performing a second replication operation for replicating a payload across the plurality of target compute nodes, wherein the sets of target compute nodes are different between the first replication operation and the second replication operation.
19. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform operations including:
determining a plurality of payload groups, wherein each payload group comprises a corresponding payload;
partitioning a plurality of target compute nodes into sets of target compute nodes based on the plurality of payload groups; and
performing a first replication operation for each payload group of the plurality of payload groups, wherein performing the first replication operation for a payload group comprises:
generating, for the first replication operation, a plan for replicating the corresponding payload on a set of target compute nodes corresponding to the payload group, the plan comprising an assignment of the set of target compute nodes to a plurality of tree nodes in a tree hierarchy,
distributing the plan to the set of target compute nodes, and
distributing, according to the plan, the corresponding payload to the set of target compute nodes in an order defined by the plan.
20. The non-transitory computer-readable medium of claim 19 , wherein the operations further comprise:
generating the assignment by, for at least a subset of the plurality of tree nodes, randomly selecting a target compute node from at least a subset of the set of target compute nodes and assigning the target compute node to a tree node in the subset of the plurality of tree nodes.