Dynamically reassigning a connected node to a block of compute nodes for re-launching a failed job
View Patent ↗Methods, systems, and products for dynamically reassigning a connected node to a block of compute nodes for re-launching a failed job that include: identifying that a job failed to execute on the block of compute nodes because connectivity failed between a compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node; and re-launching the job, including selecting an alternative connected node that is actively coupled for data communications with an active I/O node; and assigning the alternative connected node as the connected node for the block of compute nodes running the re-launched job.
1. A method of dynamically reassigning a connected node to a block of compute nodes for re-launching a failed job, the method comprising:
identifying that a job failed to execute on the block of compute nodes because connectivity failed between a compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node; and
re-launching the job, including:
selecting an alternative connected node that is actively coupled for data communications with an active I/O node; and
assigning the alternative connected node as the connected node for the block of compute nodes running the re-launched job.
2. The method of claim 1 wherein connectivity failed between the compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node because the I/O node failed.
3. The method of claim 1 wherein connectivity failed between the compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node because a link between the compute node assigned as at least one of the connected nodes for the block of compute nodes and the I/O node failed.
4. The method of claim 1 wherein selecting an alternative connected node that is actively coupled for data communications with an active I/O node further comprises selecting a predetermined alternative connected node previously identified as being coupled for data communications through a point-to-point network to the block of compute nodes.
5. The method of claim 1 wherein selecting an alternative connected node that is actively coupled for data communications with an active I/O node further comprises identifying another connected node that is coupled for data communications through a point-to-point network to the block of compute nodes.
6. The method of claim 1 wherein identifying that a job failed to execute on the block of compute nodes because connectivity failed between a compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node further comprises receiving a failure event from the supporting I/O node.
7. Apparatus for dynamically reassigning a connected node to a block of compute nodes for re-launching a failed job, the apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having stored on it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:
identifying that a job failed to execute on the block of compute nodes because connectivity failed between a compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node; and
re-launching the job, including:
selecting an alternative connected node that is actively coupled for data communications with an active I/O node; and
assigning the alternative connected node as the connected node for the block of compute nodes running the re-launched job.
8. The apparatus of claim 7 wherein connectivity failed between the compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node because the I/O node failed.
9. The apparatus of claim 7 wherein connectivity failed between the compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node because a link between the compute node assigned as at least one of the connected nodes for the block of compute nodes and the I/O node failed.
10. The apparatus of claim 7 wherein selecting an alternative connected node that is actively coupled for data communications with an active I/O node further comprises selecting a predetermined alternative connected node previously identified as being coupled for data communications through a point-to-point network to the block of compute nodes.
11. The apparatus of claim 7 wherein selecting an alternative connected node that is actively coupled for data communications with an active I/O node further comprises identifying another connected node that is coupled for data communications through a point-to-point network to the block of compute nodes.
12. The apparatus of claim 7 wherein identifying that a job failed to execute on the block of compute nodes because connectivity failed between a compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node further comprises receiving a failure event from the supporting I/O node.
13. A computer program product for dynamically reassigning a connected node to a block of compute nodes for re-launching a failed job, the computer program product stored on a non-transitory computer readable storage medium, the computer program product comprising computer program instructions that, when executed, cause a computer to carry out the steps of:
identifying that a job failed to execute on the block of compute nodes because connectivity failed between a compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node; and
re-launching the job, including:
selecting an alternative connected node that is actively coupled for data communications with an active I/O node; and
assigning the alternative connected node as the connected node for the block of compute nodes running the re-launched job.
14. The computer program product of claim 13 wherein connectivity failed between the compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node because the I/O node failed.
15. The computer program product of claim 13 wherein connectivity failed between the compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node because a link between the compute node assigned as at least one of the connected nodes for the block of compute nodes and the I/O node failed.
16. The computer program product of claim 13 wherein selecting an alternative connected node that is actively coupled for data communications with an active I/O node further comprises selecting a predetermined alternative connected node previously identified as being coupled for data communications through a point-to-point network to the block of compute nodes.
17. The computer program product of claim 13 wherein selecting an alternative connected node that is actively coupled for data communications with an active I/O node further comprises identifying another connected node that is coupled for data communications through a point-to-point network to the block of compute nodes.
18. The computer program product of claim 13 wherein identifying that a job failed to execute on the block of compute nodes because connectivity failed between a compute node assigned as at least one of the connected nodes for the block of compute nodes and its supporting I/O node further comprises receiving a failure event from the supporting I/O node.