Method and system for the dynamic scheduling of a stream of computing jobs based on priority and trigger threshold

1. A method of executing jobs comprising:

obtaining a first plurality of jobs having a first plurality of priorities;

generating, by the hardware processor and based on the first plurality of priorities, a first schedule for executing the first plurality of jobs;

identifying a lowest priority in the first plurality of priorities;

setting a first trigger threshold to a value exceeding the lowest priority;

initiating execution of the first plurality of jobs according to the first schedule;

obtaining a second plurality of jobs having a second plurality of priorities during execution of at least one of the first plurality of scheduled jobs, wherein the second plurality of priorities comprises a priority satisfying the first trigger threshold;

generating, based on the second plurality of priorities and in response to the priority satisfying the first trigger threshold, a second schedule for executing the second plurality of jobs and a job of the first plurality of jobs;

setting a second trigger threshold based on at least one of the second plurality of priorities; and

initiating execution of the second plurality of jobs and the job of the first plurality of jobs according to the second schedule.

2. The method of claim 1 , wherein the job of the first plurality of jobs is a pending job.

3. The method of claim 1 , wherein the first trigger threshold varies over an execution interval.

4. The method of claim 1 , wherein the value equals the lowest priority multiplied by two.

5. The method of claim 1 , further comprising:

assigning the first plurality of priorities to the first plurality of jobs based on at least one selected from a group consisting of an existing priority and time since submission.

6. A non-transitory computer readable medium storing instructions to execute jobs, the instructions comprising functionality to:

obtain a first plurality of jobs having a first plurality of priorities;

generate, based on the first plurality of priorities, a first schedule for executing the first plurality of jobs;

identify a lowest priority in the first plurality of priorities;

set a first trigger threshold to a value exceeding the lowest priority;

initiate execution of the first plurality of jobs according to the first schedule;

obtain a second plurality of jobs having a second plurality of priorities during execution of at least one of the first plurality of scheduled jobs, wherein the second plurality of priorities comprises a priority satisfying the first trigger threshold;

generate, based on the second plurality of priorities and in response to the priority satisfying the first trigger threshold, a second schedule for executing the second plurality of jobs and a job of the first plurality of jobs;

set a second trigger threshold based on at least one of the second plurality of priorities; and

initiate execution of the second plurality of jobs and the job of the first plurality of jobs according to the second schedule.

7. The non-transitory computer readable medium of claim 6 , wherein generating the second scheduled is further based on at least one selected from the group consisting of an execution interval, a plurality of resources, and a completion time.

8. The non-transitory computer readable medium of claim 6 , wherein the instructions further comprise functionality to:

assign the first plurality of priorities to the first plurality of jobs based on at lest one selected from the group consisting of an existing priority and time since submission.

9. The non-transitory computer readable medium of claim 6 , wherein the first trigger threshold varies over an execution interval.

10. The non-transitory computer readable medium of claim 6 , wherein the at least one job of the first plurality of jobs is a pending job from the first plurality of scheduled jobs.

11. A system for executing jobs comprising:

a hardware processor;

a priority generator configured to assign a first plurality of priorities to a first plurality of jobs and a second plurality of priorities to a second plurality of jobs; and

a scheduler executing on the hardware processor and configured to:

generate, based on the first plurality of priorities, a first schedule for executing the first plurality of jobs;

identify a lowest priority in the first plurality of priorities;

set, before the second plurality of priorities are assigned to the second plurality of jobs, a first trigger threshold to a value exceeding the lowest priority, wherein the second plurality of priorities includes a priority satisfying the first trigger threshold; and

generate, based on the second plurality of priorities and in response to the priority satisfying the first trigger threshold, a second schedule for executing the second plurality of priorities and a job of the first plurality of jobs; and

a plurality of resources configured to execute the first plurality of jobs according to the first schedule and to execute the second plurality of jobs and the job of the first plurality of jobs according to the second schedule.

12. The system of claim 11 , wherein the job of the first plurality of jobs is a pending job.

13. The system of claim 11 , wherein the first trigger threshold varies over an execution interval.

14. The system of claim 11 , wherein the value equals the lowest priority multiplied by two.