System and method for sorting and scheduling workflows
In general, embodiments of the invention relate to processing backup jobs. The processing of backup jobs includes ordering a first plurality of jobs in a priority queue, where the first plurality of jobs is associated with a first workflow, ordering a second plurality of jobs in the priority queue, where the second plurality of jobs is associated with a second workflow, where the first workflow is associated with a higher weight than the second workflow, where the first plurality of jobs is scheduled to be serviced before the second plurality of jobs, and initiating servicing of the first plurality of jobs.
1. A method for processing backup jobs, the method comprising:
ordering a first plurality of jobs in a priority queue, wherein the first plurality of jobs is associated with a first workflow;
ordering a second plurality of jobs in the priority queue, wherein the second plurality of jobs is associated with a second workflow, wherein the first workflow is associated with a higher weight than the second workflow, wherein the first plurality of jobs is scheduled to be serviced before the second plurality of jobs;
initiating servicing of the first plurality of jobs;
after initiating the servicing of the first plurality of jobs:
receiving a third plurality of jobs to be placed in the priority queue, wherein the third plurality of jobs is associated with a third workflow;
in response to receiving the third plurality of jobs, determining that an ordering cap for the priority queue has been reached; and
in response to the determination, appending the third plurality of jobs to the priority queue, wherein the third plurality of jobs is not ordered prior to the appending.
2. The method of claim 1 , wherein a first job of the first plurality of jobs is associated with a first priority and a second job of the first plurality of jobs is associated with a second priority, wherein the first priority is greater than the second priority, and wherein the first job is ordered in the priority queue before the second job.
3. The method of claim 2 , wherein a third job of the second plurality of jobs is associated with a third priority, wherein the third priority is greater than the first priority, and wherein the first job and second job are ordered in the priority queue before the third job.
4. The method of claim 1 , wherein ordering the first plurality of jobs in the priority queue is based, at least in part, on a workflow property associated with the first workflow.
5. The method of claim 4 , wherein the workflow property is at least one selected from a group consisting of application type, application version, size of data, and read sampling performance.
6. A non-transitory computer readable medium (CRM) storing instructions for processing backup jobs, the instructions comprising functionality for:
ordering a first plurality of jobs in a priority queue, wherein the first plurality of jobs is associated with a first workflow;
ordering a second plurality of jobs in the priority queue, wherein the second plurality of jobs is associated with a second workflow, wherein the first workflow is associated with a higher weight than the second workflow, wherein the first plurality of jobs is scheduled to be serviced before the second plurality of jobs;
initiating servicing of the first plurality of jobs;
after initiating the servicing of the first plurality of jobs:
receiving a third plurality of jobs to be placed in the priority queue, wherein the third plurality of jobs is associated with a third workflow;
in response to receiving the third plurality of jobs, determining that an ordering cap for the priority queue has been reached; and
in response to the determination, appending the third plurality of jobs to the priority queue, wherein the third plurality of jobs is not ordered prior to the appending.
7. The CRM of claim 6 , wherein a first job of the first plurality of jobs is associated with a first priority and a second job of the first plurality of jobs is associated with a second priority, wherein the first priority is greater than the second priority, and wherein the first job is ordered in the priority queue before the second job.
8. The CRM of claim 7 , wherein a third job of the second plurality of jobs is associated with a third priority, wherein the third priority is greater than the first priority, and wherein the first job and second job are ordered in the priority queue before the third job.
9. The CRM of claim 6 , wherein ordering the first plurality of jobs in the priority queue is based, at least in part, on a workflow property associated with the first workflow.
10. The CRM of claim 9 , wherein the workflow property is at least one selected from a group consisting of application type, application version, size of data, and read sampling performance.
11. A system, comprising:
a processor;
a scheduler; and
a priority queue,
wherein the scheduler when, executed by the processor enables the scheduler to perform a method, the method comprising:
ordering a first plurality of jobs in the priority queue, wherein the first plurality of jobs is associated with a first workflow;
ordering a second plurality of jobs in the priority queue, wherein the second plurality of jobs is associated with a second workflow, wherein the first workflow is associated with a higher weight than the second workflow, wherein the first plurality of jobs is scheduled to be serviced before the second plurality of jobs;
initiating servicing of the first plurality of jobs;
after initiating the servicing of the first plurality of jobs:
receiving a third plurality of jobs to be placed in the priority queue, wherein the third plurality of jobs is associated with a third workflow;
in response to receiving the third plurality of jobs, determining that an ordering cap for the priority queue has been reached; and
in response to the determination, appending the third plurality of jobs to the priority queue, wherein the third plurality of jobs is not ordered prior to the appending.
12. The system of claim 11 , wherein a first job of the first plurality of jobs is associated with a first priority and a second job of the first plurality of jobs is associated with a second priority, wherein the first priority is greater than the second priority, and wherein the first job is ordered in the priority queue before the second job.
13. The system of claim 12 , wherein a third job of the second plurality of jobs is associated with a third priority, wherein the third priority is greater than the first priority, and wherein the first job and second job are ordered in the priority queue before the third job.
14. The system of claim 11 , wherein ordering the first plurality of jobs in the priority queue is based, at least in part, on a workflow property associated with the first workflow.
15. The system of claim 14 , wherein the workflow property is at least one selected from a group consisting of application type, application version, size of data, and read sampling performance.