System and method for parallel processing using a Type I Howard Cascade

1. A method using for performing a scatter-type data distribution among a cluster of computational devices, the method comprising:

receiving a value T indicating a number of time steps;

receiving a value Cg indicating a number of tree generator channels, each generating a root of a respective tree structure;

receiving a value Ct indicating a number of communication channels per node in each said tree structure;

initially generating Cg nodes, each connected to an initial generator, to create respective initial root nodes of an initial tree structure;

transmitting data, from the initial generator to each of the initial root nodes;

(a) generating Cg root nodes, each connected to a respective new generator, to create respective roots of Cg newly generated tree structures;

(b) expanding each of the tree structures by generating Ct new nodes connected to each said node generated in each previous step;

(c) transmitting data, to each of the new nodes from an immediately preceding one of the nodes, and from each said new generator to an associated said root node; and

repeating steps (a) through (c) T−1 times;

wherein each of the nodes comprises one of the computational devices.

2. The method of claim 1 , wherein each of the generators and each of the nodes comprises one of the computational devices.

3. The method of claim 1 , wherein the data is transmitted in reverse order relative to the order of transmission in step (c).

4. A method used for performing a scatter-type data distribution among a cluster of computational devices, the method comprising:

receiving a value T indicating a number of time steps;

(a) generating an initial node connected to a new generator, to create a root node of a newly generated tree structure;

(b) expanding each said tree structure by generating a variable number V new nodes connected to the node, if any, generated in the step prior to the immediately previous execution of step (a);

wherein V is determined by a function which creates a predetermined number of new nodes at each said time step;

(c) transmitting data, to each of the new nodes, respectively, from an immediately preceding said node; and

repeating steps (a) through (c) T−1 times;

wherein each of the nodes comprises one of the computational devices.

5. The method of claim 4 , wherein the function creates an N-nomial tree.

6. The method of claim 4 , wherein the function creates a non-N-nomial tree.

7. The method of claim 4 , wherein the function is variable on a per-tree basis.

8. The method of claim 4 , wherein the data is transmitted in reverse order relative to the order of transmission in step (c).

9. The method of claim 4 , wherein the processing performance of two or more slow computational devices is slower than the average speed of all the computational devices working on a common task;

wherein the performance of the cluster is balanced by generating an increasing rate SUM function monotonic tree structure comprising the slow computational devices as nodes connected, via virtual channels, to a node generated in a next-previous time step, to distribute the data;

wherein the time-step-previous node comprises a root node for the monotonic tree structure.

10. The method of claim 4 , wherein the processing performance of two or more fast computational devices is faster than the average speed of all the computational devices working on a common task;

wherein the performance of the cluster is balanced by substituting, for an existing said tree structure containing the fast computational devices, a relatively decreased rate SUM function tree structure to distribute the data.

11. The method of claim 4 , wherein the processing performance of two or more fast computational devices is faster than the average speed of all the computational devices working on a common task;

wherein the performance of the cluster is balanced by removing a sufficient number of the computational devices from the tree structure containing the fast computational devices.

12. The method of claim 4 , wherein the performance of each of two slow communication channels is slower than the average channel performance of all the communication channels in the cluster;

wherein the performance of the cluster is balanced by substituting, for an existing said tree structure containing the slow communication channels, a relatively increased rate SUM function tree structure.

13. The method of claim 4 , wherein the performance of each of two slow communication channels is slower than the average channel performance of all the communication channels in the cluster;

wherein the performance of the cluster is balanced by adding at least one virtual channel between the computational devices connected by one of the slow communication channels.

14. The method of claim 4 , wherein the performance of each of two fast communication channels is faster than the average channel performance of all the communication channels in the cluster;

wherein the performance of the cluster is balanced by substituting, for an existing said tree structure containing the fast communication channels, a relatively decreased rate SUM function tree structure.

15. The method of claim 9 , wherein the data is transmitted in reverse order relative to the order of transmission in step (c).

16. The method of claim 10 , wherein the data is transmitted in reverse order relative to the order of transmission in step (c).

17. The method of claim 11 , wherein the data is transmitted in reverse order relative to the order of transmission in step (c).

18. The method of claim 12 , wherein the data is transmitted in reverse order relative to the order of transmission in step (c).

19. The method of claim 13 , wherein the data is transmitted in reverse order relative to the order of transmission in step (c).

20. The method of claim 14 , wherein the data is transmitted in reverse order relative to the order of transmission in step (c).