System, method and computer program product for resource management in a distributed computation system

1. A computer-implemented method in a distributed computation system, comprising:

determining a task resource consumption predicted for each of one or more tasks being executed on a node, wherein the task resource consumption is a function of time, each of the one or more tasks having respective computation logics;

predicting a node resource consumption of the node based at least on the predicted task resource consumption, wherein the node resource consumption is a function of time; and

pre-storing, in advance, for each of the one or more tasks having the respective computation logics a functional mathematical relationship between a value of a duration time, a stage, and an input data size,

wherein, when the predicting predicts, the value of the duration time of the stage corresponding to the input data size are retrieved and a duration of a stage is determined for a different task having a same computation logic to the respective computation log of the functional mathematical relationship, wherein the functional mathematical relationship is created in advance in the pre-storing,

the method further comprising allocating a resource to a new task and setting a start time of execution of the new task based at least on the predicted node resource consumption.

2. The computer-implemented method according to claim 1 , further comprising:

determining timing information of start of execution of each of the one or more tasks,

wherein the predicting the node resource consumption of the node is based at least on the predicted task resource consumption and the timing information for each of the one or more tasks.

3. The computer-implemented method according to claim 2 , wherein the determining the timing information comprises:

monitoring an actual node resource consumption of the node; and

determining time differences between start times of executions of the one or more tasks based on the predicted task resource consumptions and the monitored actual node resource consumption.

4. The computer-implemented method according to claim 2 , wherein the predicting a node resource consumption of the node comprises:

combining the predicted task resource consumptions based on time differences which are determined from the timing information of the one or more tasks.

5. The computer-implemented method according to claim 1 , wherein the predicted task resource consumption of at least one of the one or more tasks is predicted based on one or more of a pre-stored task resource consumptions of one or more monitored tasks, and

wherein the one or more monitored tasks and the at least one of the one or more tasks have the same computation logics.

6. The computer-implemented method according to claim 1 , wherein the task resource consumption of a task selected from the one or more tasks being executed on a node is predicted based on the computation logic of the task and the input data size of the task.

7. The computer-implemented method according to claim 1 , wherein the task resource consumption of a task selected from the one or more tasks being executed on a node is predicted by:

obtaining a way of change of the task resource consumption with time for each stage of the task resource consumption; and

determining the value of the duration of each stage of the task resource consumption based on the input data size of the task.

8. The computer-implemented method according to claim 1 , further comprising:

determining a task resource consumption predicted for a further task that is newly executed on the node; and

updating the predicted node resource consumption of the node based at least on the predicted task resource consumption for the further task.

9. The computer-implemented method of claim 1 , embodied in a cloud-computing environment.

10. The computer-implemented method according to claim 1 , wherein, for the tasks for which the input data size is fixed, predicting the task resource consumption based on monitored tasks of a same type of task in advance, and to store, in advance, a sequence of discrete values of the task resource consumption in association with points of time in the duration of time.

11. A distributed computation system, comprising:

a processor, and

a memory, the memory storing instructions to cause the processor to perform:

determining a task resource consumption predicted for each of one or more tasks being executed on a node, wherein the task resource consumption is a function of time, each of the one or more tasks having respective computation logics;

predicting a node resource consumption of the node based at least on the predicted task resource consumption, wherein the node resource consumption is a function of time; and

pre-storing, in advance, for each of the one or more tasks having the respective computation logics a functional mathematical relationship between a value of a duration time, a stage, and an input data size,

wherein, when the predicting predicts, the value of the duration time of the stage corresponding to the input data size are retrieved and a duration of a stage is determined for a different task having a same computation logic to the respective computation log of the functional mathematical relationship, wherein the functional mathematical relationship is created in advance in the pre-storing, and

wherein the memory further stores instructions to cause the processor to perform allocating a resource to a new task and setting a start time of execution of the new task based at least on the predicted node resource consumption.

12. The system according to claim 11 , wherein the memory further stores instructions to cause the processor to perform:

determining timing information of start of execution of each of the one or more tasks, and

wherein the predicting the node resource consumption of the node is based at least on the predicted task resource consumption and the timing information for each of the one or more tasks.

13. The system according to claim 12 , wherein to determine the timing information, the memory further stores instructions to cause the processor to perform:

monitoring an actual node resource consumption of the node; and

determining time differences between start times of executions of the one or more tasks based on the predicted task resource consumptions and the monitored actual node resource consumption.

14. The system according to claim 11 , wherein the predicted task resource consumption of at least one of the one or more tasks is predicted based on one or more of a pre-stored task resource consumptions of one or more monitored tasks, and

wherein the one or more monitored tasks and the at least one of the one or more tasks have the same computation logics.

15. The system according to claim 11 , wherein the task resource consumption of a task selected from the one or more tasks being executed on a node is predicted based on the computation logic of the task and the input data size of the task.

16. The system of claim 11 , embodied in a cloud-computing environment.

17. A non-transitory computer program product for a distributed computation system, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions being executable by a computer to cause the computer to perform:

determining a task resource consumption predicted for each of one or more tasks being executed on a node, wherein the task resource consumption is a function of time, each of the one or more tasks having respective computation logics;

predicting a node resource consumption of the node based at least on the predicted task resource consumption, wherein the node resource consumption is a function of time; and

pre-storing, in advance, for each of the one or more tasks having the respective computation logics a functional mathematical relationship between a value of a duration time, a stage, and an input data size,

wherein, when the predicting predicts, the value of the duration time of the stage corresponding to the input data size are retrieved and a duration of a stage is determined for a different task having a same computation logic to the respective computation, log of the functional mathematical relationship, wherein the functional mathematical relationship is created in advance in the pre-storing, and

wherein the program further stores instructions executable by a computer to cause the computer to perform allocating a resource to a new task and setting a start time of execution of the new task based at least on the predicted node resource consumption.

18. The non-transitory computer program product according to claim 17 , wherein the program further stores instructions executable by a computer to cause the computer to perform:

determining timing information of start of execution of each of the one or more tasks, and

wherein the predicting the node resource consumption of the node is based on the predicted task resource consumption and the timing information for each of the one or more tasks.

19. The non-transitory computer program product according to claim 17 , wherein the predicted task resource consumption of at least one of the one or more tasks is predicted based on one or more of a pre-stored task resource consumptions of one or more monitored tasks, and

wherein the one or more monitored tasks and the at least one of the one or more tasks have the same computation logics.