Method and system for controlling distribution of work items to threads in a server
View Patent ↗A system and method are presented to control distribution of work items to threads in a server. The system and method include a permit dispenser that keeps track of permits, and a plurality of thread pools each including a queue with a configurable size, being configured with a desired concurrency and a size of the queue that is equal to a total number of work items to be executed by pool threads in the thread pool. The number of permits specifies a total number of threads available for executing the work items in the server. Each pool thread executes a work item in the thread pool, determines whether a thread surplus or a thread deficit exists, and shrinks or grows the thread pool respectively.
1. A computer-implemented system for controlling distribution of work items to
threads in a server, comprising: hardware computer processor to execute:
a permit dispenser that keeps track of permits, a number of permits specifying a total number of threads available for executing the work items in the server;
a workflow authoring component that determines a desired concurrency, wherein the desired concurrency is equal to a maximum number of work items that can be executed in parallel;
a thread pool manager to manage one or more thread pools and to request a number of permits equal to the desired concurrency; and wherein if no permits are available, a calling thread to: determine if work permits are available; if no work permits are available, execute one work item and re-request from the permit dispenser the number of permits that is equal to a lesser of the desired concurrency and a number of remaining unexecuted work items.
2. The system of claim 1 , wherein if the system obtains one or more permits, the thread pool manager is arranged to create thread pool that comprises:
the one or more pool threads, wherein a number of the one or more pool threads is equal to an initial concurrency that is equal to a number of permits obtained from the permit dispenser and a queue size that is equal to a total number of work items in a set of work items the thread pool is to execute.
3. The system of claim 1 , wherein a thread surplus exists when a number of pool
threads in the thread pool is more than a number of remaining unexecuted work items.
4. The system of claim 3 , wherein if the thread surplus exists, the pool thread is arranged to return the permit to the permit dispenser and terminate, thereby shrinking the thread pool.
5. The system of claim 1 , wherein a thread deficit exists when a number of pool
threads working on the work items in the thread pool is less than both the desired
concurrency and a number of remaining unexecuted work items.
6. The system of claim 5 , wherein if the thread deficit is greater than zero, the pool thread is arranged to request a number of permits from the permit dispenser that is equal to the thread deficit.
7. The system of claim 6 , wherein the pool thread is arranged to add acquired new pool threads to the thread pool.
8. The system of claim 1 , wherein the thread pool manager ensures that each pool thread in the thread pool executes work items without relying on another pool thread in a same thread pool to complete a sub-item, and wherein the sub-item is executed by threads in a different thread pool, thereby preventing a deadlock.
9. A method for controlling distribution of work items to threads in a server, the method being implemented by a computer including a processor and a memory, comprising:
using the processor to provide a permit dispenser that keeps track of a plurality of permits, a number of the plurality of permits specifying a total number of threads available for executing the work items in the server;
determining a desired concurrency, using the processor, the desired concurrency being equal to a maximum number of work items enabled to execute in parallel; and
creating a plurality of thread pools stored in the memory using the processor; the thread pool manager requesting a number of permits equal to the desired concurrency; and
if no permits are available, executing, using the processor, one work item and re-requesting from the permit dispenser the number of permits equal to a lesser of the desired concurrency and the number of remaining unexecuted work items.
10. The method of claim 9 , wherein a thread surplus exists when a number of pool threads in the thread pool is more than a number of remaining unexecuted work items.
11. The method of claim 10 , further comprising if the thread surplus exists, the pool thread returning the permit to the permit dispenser and terminating, thereby shrinking the thread pool.
12. The method of claim 9 , wherein a thread deficit exists when a number of pool threads working on the work items in the thread pool is less than both the desired concurrency and a number of remaining unexecuted work items.
13. The method of claim 12 , further comprising:
if the thread deficit is greater than zero, the pool thread requesting a number of permits from the permit dispenser that is equal to the thread deficit; and
adding newly acquired pool threads to the thread pool.
14. A non-transitory computer readable medium providing instructions for controlling distribution of work items to threads in a server, the instructions, if executed, causes a processor to:
provide a permit dispenser to keep track of a plurality of permits, a number of the plurality of permits specifying a total number of threads available for executing work items in a server;
provide a workflow authoring component to determine a desired concurrency, the desired concurrency being equal to a maximum number of work items that can be executed in parallel; and
use a thread pool manager to create a plurality of thread pools stored in a memory and to request a number of permits equal to the desired concurrency; and
if no permits are available, execute one work item and re-request from the permit dispenser the number of permits equal to a lesser of the desired concurrency and the number of remaining unexecuted work items.