Workload manager managing a workload of an enterprise data warehouse
View Patent ↗Described herein is a workload manager for managing a workload in a database that includes: an admission controller operating to divide the workload into a plurality of batches, with each batch having at least one workload process to be performed in the database, and each batch having a memory requirement based on the available memory for processing workloads in the database; a scheduler operating to assign a unique priority to each of the at least one workload process in each of the plurality of batches, the unique priority provides an order in which each workload process is executed in the database; and an execution manager operating to execute the at least one workload process in each of the plurality of batches in accordance with the unique priority assigned to each workload process.
1. A workload manager for managing a workload in a database, comprising:
an admission controller to divide the workload into a plurality of batches, with each batch having at least one workload process to be performed in the database, and each batch having a memory requirement based on available memory for processing the workload;
a scheduler, executed by a hardware processor, to assign a unique priority to each workload process in each of the plurality of batches, the unique priorities to provide an order in which each workload process is executed in the database, wherein each assigned unique priority is different from any other of the assigned unique priorities; and
an execution manager to execute each workload process in each of the plurality of batches in accordance with the unique priority assigned to each workload process.
2. The workload manager of claim 1 , wherein the workload comprises a plurality of database queries, and each of the plurality of batches includes at least one database query as the at least one workload process.
3. The workload manager of claim 1 , wherein the admission controller is further to arrange each workload process in each of the plurality of batches in order of memory requirement so as to divide the workload into the plurality of batches.
4. The workload manager of claim 1 , wherein the scheduler is to randomly assign the unique priority to each workload process in each of the plurality of batches.
5. The workload manager of claim 1 , wherein the scheduler is to assign the unique priority to each workload process in each of the plurality of batches based on a memory requirement of the workload process.
6. The workload manager of claim 5 , wherein each workload process in a first batch of the plurality of batches comprises a plurality of workload processes, and a workload process of the plurality of workload processes with the highest memory requirement in the first batch is assigned a highest priority as its unique priority.
7. The workload manager of claim 1 , wherein the scheduler is to assign the unique priority to each of the at least one workload process in each of the plurality of batches based on a resource characteristic in the database.
8. A method for managing a workload of queries in a database, comprising:
dividing the workload of queries into a plurality of batches of queries such that each batch has a memory requirement of no more than an average available memory for processing workloads in the database;
assigning a unique priority to each of the queries, the unique priorities providing an order in which each query is executed in the database, wherein each assigned unique priority is different from any other of the assigned unique priorities; and
executing each of the plurality of batches by:
a) executing at least one query in one batch based on the unique priority assigned to the at least one query in the one batch;
b) determining whether the execution of the one batch is done; and
c) executing another batch of the plurality of batches once the execution of the selected batch is done.
9. The method of claim 8 , wherein determining whether the execution of the one batch is done comprises at least one of:
first determining whether a predetermined threshold number of queries in the one batch is done;
second determining whether a usage of memory of the database by the one batch falls below a predetermined memory threshold; and
third determining whether an average utilization of a processing power of the database by the one batch falls below a predetermined processing threshold.
10. The method of claim 9 , wherein determining whether the execution of the one batch is done comprises the steps of first determining, second determining, and third determining.
11. The method of claim 8 , wherein dividing the workload of queries into a plurality of batches of queries comprises:
arranging the queries in the workload in descending order of memory requirement;
assigning a first one of the queries in the arranged descending order to one of the plurality of batches that is able to accommodate each query without exceeding an average available memory for processing workloads in the database; and
repeating the step of assigning each query, in the arranged descending order, until all of the queries in the workload are assigned to the plurality of batches.
12. The method of claim 8 , wherein assigning a unique priority to each of the queries in each of the plurality of batches comprises:
uniquely ordering the queries in each of the plurality of batches; and
assigning the unique priority to each of the queries in each batch from highest priority to lowest priority in accordance with the unique ordering of the queries in each batch.
13. The method of claim 12 , wherein uniquely ordering the queries comprises:
uniquely ordering the queries in each of the plurality of batches according to an ordering function that randomly orders the queries.
14. The method of claim 12 , wherein unique ordering the queries comprises:
uniquely ordering the queries in each of the plurality of batches according to an ordering function based on a memory requirement of each of the queries in each batch.
15. The method of claim 14 , wherein uniquely ordering the queries in each of the plurality of batches according to an ordering function further comprises:
ranking a query that has a highest memory requirement in the batch as a first query, and ranking a query that has a lowest memory requirement in the batch as a last query.
16. The method of claim 15 , wherein assigning the unique priority to each of the queries in each batch includes assigning the unique priority from highest priority to lowest priority, wherein the highest priority is assigned to the first ranked query and the lowest priority is assigned to the last-ranked query.
17. A non-transitory computer readable medium on which is encoded code that when executed by a processing unit, the code is to:
divide the workload of queries into a plurality of batches of queries such that each batch has a memory requirement of no more than an average available memory for processing workloads in the database;
assign a unique priority to each of the queries in each of the plurality of batches, the unique priorities to provide an order in which each workload process is executed in the database, wherein each assigned unique priority is different from any other of the assigned unique priorities; and
execute each of the plurality of batches, wherein to execute each of the plurality of batches, the code is to:
a) execute one of the plurality of batches based on the unique priority assigned to each of the queries in the one batch;
b) determine whether the execution of the one batch is done; and
c) execute another one of the plurality of batches once the execution of the one batch is done.
18. The non-transitory computer readable medium of claim 17 , wherein the code to divide the workload of queries into a plurality of batches of queries comprises code to:
arrange the queries in the workload in descending order of memory requirement;
assign a first one of the queries in the arranged descending order to one of the plurality of batches that is able to accommodate each query without exceeding an average available memory for processing the workloads in the database; and
repeat the assignment of each query, in the arranged descending order, until all of the queries in the workload are assigned to the plurality of batches.
19. The non-transitory computer readable medium of claim 17 , wherein the code to assign a unique priority to each of the queries in each of the plurality of batches comprises code to:
uniquely order the queries in each of the plurality of batches; and
assign the unique priority to each of the queries in each batch from highest priority to lowest priority in accordance with the unique ordering of the queries in each batch.
20. The method of claim 12 , wherein uniquely ordering the queries comprises:
uniquely ordering the queries in each of the plurality of batches according to an ordering function that randomly orders the queries; and
uniquely ordering the queries in each of the plurality of batches according to an ordering function that is based a memory requirement of each of the queries in each batch.