Fast concurrent array-based stacks, queues and deques using fetch-and-increment-bounded, fetch-and-decrement-bounded and store-on-twin synchronization primitives
Implementation primitives for concurrent array-based stacks, queues, double-ended queues (deques) and wrapped deques are provided. In one aspect, each element of the stack, queue, deque or wrapped deque data structure has its own ticket lock, allowing multiple threads to concurrently use multiple elements of the data structure and thus achieving high performance. In another aspect, new synchronization primitives FetchAndIncrementBounded (Counter, Bound) and FetchAndDecrementBounded (Counter, Bound) are implemented. These primitives can be implemented in hardware and thus promise a very fast throughput for queues, stacks and double-ended queues.
1. A method for concurrent operation in a multithreaded computing environment, the method comprising:
fetching a counter and comparing the counter to a boundary, by one or more of a plurality of synchronization primitives, wherein the plurality of synchronization primitives comprise at least:
FetchAndIncrementBounded, the FetchAndIncrementBounded outputting the counter and then incrementing the counter if the counter is less than the boundary, the FetchAndIncrementBounded outputting an indication of reaching a bound of a data structure if the counter is not less than the boundary;
FetchAndDecrementBounded, the FetchAndDecrementBounded outputting the counter and then decrementing the counter if the counter is greater than the boundary, the FetchAndDecrementBounded outputting an indication of reaching a bound of a data structure if the counter is not greater than the boundary; and
StoreOnTwin, the StoreOnTwin setting the counter and the boundary to a given value if a value of the counter and a value of the boundary are equivalent to each other, said StoreOnTwin not changing the value of the counter or the value of the boundary if the value of the counter and the value of the boundary are not equivalent;
constructing a concurrent array-based data structure, each element of the concurrent array-based data structure having a ticket lock for granting an access to the element of the concurrent array-based data structure if a thread has a corresponding ticket; and
concurrently operating said concurrent array-based data structure using at least one of the plurality of synchronization primitives, the at least one of the plurality of synchronization primitives being one of the FetchAndIncrementBounded and the FetchAndDecrementBounded,
wherein said concurrent array-based data structure is a double-ended queue (deque), and said step of concurrently operating comprises obtaining an index to an element for putting an object on a top of the deque by executing the FetchAndIncrementBounded, obtaining an index to an element for taking an object on the top of the deque by executing the FetchAndDecrementBounded, obtaining an index to an element for putting an object on a bottom of the deque by executing the FetchAndDecrementBounded and obtaining an index to an element for taking an element on the bottom of the deque by executing the FetchAndIncrementBounded; and
wherein the StoreOnTwin resets a first pointer referencing a top of said deque and a second pointer referencing a bottom of said deque to reference a middle of the deque if the first pointer equals the second pointer.
2. The method of claim 1 , wherein the FetchAndIncrementBounded and the FetchAndDecrementBounded are implemented in a memory system and wherein the counter and the boundary are stored in adjacent memory locations.
3. A system for concurrent operation in a multithreaded computing environment, the system comprising:
a hardware processor;
one or more synchronization primitives of a plurality of synchronization primitives executable on the hardware processor, each of the one or more synchronization primitives fetching a counter and comparing the counter to a boundary, wherein the plurality of synchronization primitives comprise at least:
FetchAndIncrementBounded, the FetchAndIncrementBounded outputting the counter and then incrementing the counter if the counter is less than the boundary, the FetchAndIncrementBounded outputting an indication of reaching a bound of a data structure if the counter is not less than the boundary;
FetchAndDecrementBounded, the FetchAndDecrementBounded outputting the counter and then decrementing the counter if the counter is greater than the boundary, the FetchAndDecrementBounded outputting an indication of reaching a bound of a data structure if the counter is not greater than the boundary; and
StoreOnTwin, the StoreOnTwin setting the counter and the boundary to a given value if a value of the counter and a value of the boundary are equivalent to each other, said StoreOnTwin not changing the value of the counter or the value of the boundary if the value of the counter and the value of the boundary are not equivalent;
a processing unit for constructing a concurrent array-based data structure, each element of the concurrent array-based data structure having a ticket lock that grants an access to the element of the concurrent array-based data structure if a thread has a corresponding ticket; and
a functional unit for concurrently operating the concurrent array-based data structure using at least one of the plurality of synchronization primitives,
wherein said concurrent array-based data structure is a double-ended queue (deque), and said functional unit is further operable to obtain an index to an element for putting an object on a top of the deque by executing the FetchAndIncrementBounded, obtain an index to an element for taking an object on the top of the deque by executing the FetchAndDecrementBounded, obtain an index to an element for putting an object on a bottom of the deque by executing the FetchAndDecrementBounded, and obtain an index to an element for taking an element on the bottom of the deque by executing the FetchAndIncrementBounded; and
wherein the StoreOnTwin resets a first pointer referencing a top of said deque and a second pointer referencing a bottom of said deque to reference a middle of the deque if the first pointer equals the second pointer.
4. The system according to claim 3 , wherein said deque is a wrapped deque, wherein if said top of said wrapped deque is full, said functional unit is further operable to put to said top of said wrapped deque using FetchAndIncrementBounded to move a free element from said bottom of said wrapped deque to said top of said wrapped deque if a free element on said bottom of said deque is found, and wherein if said bottom of said wrapped deque is full, said functional unit is further operable to put to said bottom of said wrapped deque using FetchAndDecrementBounded to move a free element from said top of said wrapped deque to said bottom of said wrapped queue if a free element on said top of said deque is found.
5. The system of claim 3 , further comprising a memory system wherein the FetchAndIncrementBounded and the FetchAndDecrementBounded are implemented in the memory system and wherein the counter and the boundary are stored in adjacent memory locations of the memory system.
6. A non-transitory program memory device readable by a machine, the device tangibly embodying a program of instructions executable by the machine to perform a method for concurrent operation in a multithreaded computing environment, the method comprising:
fetching a counter and comparing the counter to a boundary, by one or more synchronization primitives of a plurality of synchronization primitives, wherein the plurality of synchronization primitives comprise at least:
FetchAndIncrementBounded, the FetchAndIncrementBounded outputting the counter and then incrementing the counter if the counter is less than the boundary, the FetchAndIncrementBounded outputting an indication of reaching a bound of a data structure if the counter is not less than the boundary;
FetchAndDecrementBounded, the FetchAndDecrementBounded outputting the counter and then decrementing the counter if the counter is greater than the boundary, the FetchAndDecrementBounded outputting an indication of reaching a bound of a data structure if the counter is not greater than the boundary; and
StoreOnTwin, the StoreOnTwin setting the counter and the boundary to a given value if a value of the counter and the value of the boundary are equivalent to each other, said StoreOnTwin not changing the value of the counter or the value of the boundary if the value of the counter and the value of the boundary are not equivalent.
7. The program memory device of claim 6 , wherein the method further comprises:
constructing a concurrent array-based data structure, each element of the concurrent array-based data structure having a ticket lock for granting an access to the element of the concurrent array-based data structure if a thread has a corresponding ticket; and
concurrently operating said concurrent array-based data structure using at least one of the plurality of synchronization primitives, the at least one of the plurality of synchronization primitives being one of the FetchAndIncrementBounded and the FetchAndDecrementBounded.
8. The program memory device of claim 7 , wherein said concurrent array-based data structure is a queue, and said step of concurrently operating comprises obtaining an index to an element for a put operation by executing the FetchAndIncrementBounded and obtaining an index to an element for a take operation by executing the FetchAndIncrementBounded.
9. The program memory device according to claim 8 , wherein said index into the concurrent array-based queue for at least one of the put operation and the take operation is also used to calculate an expected ticket lock value for said element for the at least one of the put operation and the take operation.
10. The program memory device of claim 7 , wherein said concurrent array-based data structure is a stack, and said step of concurrently operating comprises obtaining an index to an element for a put operation by executing the FetchAndIncrementBounded and obtaining an index to an element for a take operation by executing the FetchAndDecrementBounded.
11. The program memory device of claim 7 , wherein said concurrent array-based data structure is a double-ended queue (deque), and said step of concurrently operating comprises obtaining an index to an element for putting an object on a top of the deque by executing the FetchAndIncrementBounded, obtaining an index to an element for taking an object on the top of the deque by executing the FetchAndDecrementBounded, obtaining an index to an element for putting an object on a bottom of the deque by executing the FetchAndDecrementBounded and obtaining an index to an element for taking an element on the bottom of the deque by executing the FetchAndIncrementBounded.
12. The program memory device according to claim 11 , wherein said deque is a wrapped deque, wherein if said top of said wrapped deque is full, putting to said top of said wrapped deque uses FetchAndIncrementBounded to move a free element from said bottom of said wrapped deque to said top of said wrapped deque if a free element on said bottom of said deque is found, and wherein if said bottom of said wrapped deque is full, putting to said bottom of said wrapped deque uses FetchAndDecrementBounded to move a free element from said top of said wrapped deque to said bottom of said wrapped queue if a free element on said top of said deque is found.
13. The program memory device of claim 11 , further comprising the StoreOnTwin resetting a first pointer referencing a top of said deque and a second pointer referencing a bottom of said deque to reference a middle of the deque if the first pointer equals the second pointer.