High performance synchronization mechanisms for coordinating operations on a computer system
To synchronize operations of a computing system, a new type of synchronization barrier is disclosed. In one embodiment, the disclosed synchronization barrier provides for certain synchronization mechanisms such as, for example, “Arrive” and “Wait” to be split to allow for greater flexibility and efficiency in coordinating synchronization. In another embodiment, the disclosed synchronization barrier allows for hardware components such as, for example, dedicated copy or direct-memory-access (DMA) engines to be synchronized with software-based threads.
1. A synchronization barrier comprising:
a data structure stored in memory, the data structure comprising a counter;
the counter being advanced by a first operation performed by an execution thread and the counter being further advanced by a second operation performed by a hardware based controlled operator that can advance the counter independently of the first operation performed by the execution thread,
wherein the data structure stored in the memory comprises an arrival counter and a further value used to reinitialize the arrival counter upon reset of the synchronization barrier, and the hardware operator comprises hardware that performs copying.
2. The synchronization barrier of claim 1 wherein the data structure stored in the memory comprises a phase flag.
3. The synchronization barrier of claim 1 wherein the first operation comprises an ARRIVE that is distinct from a WAIT and/or a WAIT that is distinct from an ARRIVE.
4. The synchronization barrier of claim 1 wherein the data structure is structured to be reset in response to a fused load/store atomic that can be initiated by either a hardware engine or a software thread.
5. The synchronization barrier of claim 1 further comprising circuitry associated with the counter that enables gating of further execution of the same or different execution thread based on a state of the counter.
6. A computing system comprising:
a synchronization barrier primitive stored in memory, the primitive including a counter and a phase indicator, wherein the counter is associated with a collection of threads and at least one copy operation performed a hardware based controlled operator; and
a memory access circuit that resets the counter and changes the phase indicator in response to the counter indicating that all threads in the collection of threads and the at least one copy operation have reached a synchronization point and all operations in said collection of threads have completed.
7. The system of claim 6 wherein the counter counts an aggregate of a number of copy operation completions and the number of execution thread arrive calls.
8. The system of claim 6 wherein the memory access circuit resets the counter and changes the phase indicator in response to executing of an instruction by a software thread, the instruction consisting of an ARRIVE operation that does not include a WAIT operation or a WAIT operation that does not include an ARRIVE operation.
9. The system of claim 6 wherein primitive stored in the memory further comprises a predetermined value, and hardware resets the counter by loading the predetermined value when the counter indicates that all threads in a collection of threads and copy operations have reached a synchronization point and all copy operations in said collection have completed.
10. The system of claim 9 wherein the system is structured to allow a thread to dynamically change the predetermined value.
11. The system of claim 6 wherein the synchronization barrier primitive is stored in shared memory of a Graphics Processing Unit (GPU).
12. The system of claim 6 wherein the synchronization barrier primitive is stored in a memory hierarchy which determines access by threads to the primitive.
13. The system of claim 6 further including a comparator that compares the count of the counter with a predetermined value and resets the primitive based on results of the comparison.
14. The system of claim 6 wherein the primitive's phase indicator is structured to be read first by an ARRIVE command and then by a WAIT command, so that a thread can determine whether the primitive's phase indicator has changed phase state.
15. A non-transitory readable medium storing a Graphics Processing Unit (GPU) instruction set architecture comprising:
an ARRIVE operation that reads at least a phase indicator portion of a synchronization barrier primitive stored in memory and causes the barrier primitive to advance a counter by a first operation performed by an execution thread and to advance the counter by a second operation performed by a hardware based controlled operator;
a WAIT operation that reads at least the phase indicator portion of the primitive stored in the memory and compares the phase indicator portion read by the ARRIVE operation with the phase indicator portion of the primitive read by the WAIT operation to determine whether a phase state of a barrier has changed, and
a fused load/store instruction that permits a hardware-based engine to reset the synchronization barrier primitive when the hardware-based engine completes a task assigned to it.
16. The non-transitory readable medium of claim 15 , wherein the GPU instruction set architecture further includes an ADD operation that adds to a field stored with the synchronization barrier primitive, the field being used to reinitialize the primitive upon reset to a next phase state.
17. The non-transitory readable medium of claim 15 , wherein the GPU instruction set architecture further comprises a CREATE instruction that initializes and stores the synchronization barrier primitive to the memory.
18. A synchronization method comprising:
storing in memory synchronization barrier indicia including a phase indicator and a counter associated with a set of threads and a task performed by a hardware initiated controller;
executing an arrive instruction with at least one thread of the set of threads, thereby causing the counter to count and enabling the thread to read the phase indicator;
completing the task with the hardware initiated controller, thereby causing the counter to count;
resetting the counter when the counter count indicates that the set of threads have executed arrive instructions and the hardware initiated controller has completed the task; and
executing a wait instruction with the at least one thread, thereby enabling the at least one thread to again read the phase indicator, the at least one thread conditioning blocking on whether the phase indicator has changed values.
19. The synchronization method of claim 18 including opening a window of execution from when at least one thread executes the arrive instruction to when the at least one thread executes the wait instruction, the at least one thread performing work that is asynchronous with respect to a synchronization barrier within the window of execution.
20. A synchronization barrier comprising:
a counter providing a synchronization barrier count, wherein the counter resides in memory; and
circuitry operatively connected to the counter that advances the synchronization barrier count in response to completion of software initiated operations performed by execution threads and advances the synchronization barrier count in response to completion of operations performed by hardware initiated operators,
wherein the circuitry is structured to reset the synchronization barrier count in response to a fused load/store atomic that can be initiated by each of copy hardware and software thread execution.
21. The synchronization barrier of claim 20 wherein the memory comprises memory shared by the execution threads and the hardware initiated operators.
22. The synchronization barrier of claim 20 wherein the counter is stored in shared memory of a Graphics Processing Unit (GPU).