Distributed shared memory
Distributed shared memory (DSMEM) comprises blocks of memory that are distributed or scattered across a processor (such as a GPU). Threads executing on a processing core local to one memory block are able to access a memory block local to a different processing core. In one embodiment, shared access to these DSMEM allocations distributed across a collection of processing cores is implemented by communications between the processing cores. Such distributed shared memory provides very low latency memory access for processing cores located in proximity to the memory blocks, and also provides a way for more distant processing cores to also access the memory blocks in a manner and using interconnects that do not interfere with the processing cores' access to main or global memory such as hacked by an L2 cache. Such distributed shared memory supports cooperative parallelism and strong scaling across multiple processing cores by permitting data sharing and communications previously possible only within the same processing core.
1. A processing system comprising:
a first semiconductor memory block co-located with at least one associated first processing core executing a first cooperative thread array (CTA) of a cooperative group array (CGA);
a second semiconductor memory block co-located with at least one associated second processing core executing a second CTA of the CGA; and
an interconnect operatively connecting the first processing core with the second processing core,
wherein the first CTA is enabled to access the second semiconductor memory block via the second processing core.
2. The processing system of claim 1 wherein the processing system conditions access of the second semiconductor memory block by the first CTA on allocation of the second semiconductor memory block to the second CTA.
3. The processing system of claim 1 wherein the processing system is configured to guarantee that the first and second CTAs run concurrently.
4. The processing system of claim 1 wherein the first and second processing cores each comprise streaming multiprocessors, the first semiconductor memory block is part of a first streaming multiprocessor, the second semiconductor memory block is part of a second streaming multiprocessor, and the interconnect provides intercommunications between the first streaming multiprocessor and the second streaming multiprocessor.
5. The processing system of claim 1 wherein the processing system is configured to enable the first CTA to issue read, write, and atomic commands to access the second semiconductor memory block.
6. The processing system of claim 1 wherein the interconnect comprises a switch or a network.
7. The processing system of claim 1 wherein the interconnect enables direct communication between processing cores to all shared memory allocated to CTAs that are part of the same CGA.
8. The processing system of claim 1 wherein the processing system is configured to provide segmented addressing of distributed shared memory blocks based on a unique identifier per executing CGA.
9. The processing system of claim 1 wherein the processing system is configured to support hardware resident CGA identifier allocation and recycling protocols.
10. The processing system of claim 1 wherein the first and second processing cores each store an associating structure to enable distributed shared memory addressing from processing cores remote thereto.
11. The processing system of claim 1 wherein the interconnect comprises a low-latency processor-to-processor communication network.
12. The processing system of claim 1 wherein the processing system coalesces write acknowledgements for communications across the interconnect.
13. The processing system of claim 1 wherein the processing system includes hardware barriers to synchronize usage of the first semiconductor memory block and the second semiconductor memory block at CGA creation and during CGA context switching.
14. The processing system of claim 13 wherein the hardware barriers ensure the first semiconductor memory block and the second semiconductor memory block are available in all CTAs of a CGA before any CTA references the memory blocks over the interconnect.
15. The processing system of claim 13 wherein the hardware barriers ensure all CTAs in a CGA have completed all accesses before state save, and all distributed shared memory has been restored during state restore.
16. The processing system of claim 1 wherein the processing system is further configured to provide CGA/CTA exit and error handling protocols with distributed shared memory.
17. A method of operating a processing system comprising:
launching a first thread block on a first processing core;
allocating to the first thread block a first semiconductor memory block local to the first processing core;
launching a second thread block on a second processing core;
allocating to the second thread block a second semiconductor memory block local to the second processing core;
launching a third thread block on a third processing core;
allocating to the third thread block a third semiconductor memory block local to the third processing core; and
enabling the first thread block to access each of the allocated second and third semiconductor memory blocks via the second and third processing cores, respectively; enabling the second thread block to access each of the allocated first and third semiconductor memory blocks via the first and third processing cores, respectively; and enabling the third thread block to access each of the first and second semiconductor memory blocks via the first and second processing cores, respectively.
18. The method of claim 17 wherein the processing system is disposed on a substrate, and the first, second and third memory blocks are disposed on different portions of the substrate and are separated from one another.
19. The method of claim 17 further including using a hardware barrier to ensure that the first, second and third semiconductor memory blocks are allocated before permitting access by any of the first, second and third thread blocks.
20. The method of claim 17 wherein the enabling comprises providing communications between the first, second and third processing cores.
21. A processing system comprising:
a hardware work distributor circuit that launches a collection of thread groups concurrently on plural processors; and
a communications interface between the plural processors that enables each thread group to access memory allocated to any other thread group in the collection,
wherein the communications interface comprises communication links between the plural processors and inter-processor messaging functions performed by the plural processors, and each thread group is enabled to access memory allocated to other thread groups via respective processor(s) executing the other thread groups to which memory to be accessed is allocated.
22. The processing system of claim 21 wherein the communications interface prohibits thread groups not in the collection from accessing memory portions allocated to any other thread group in the collection.
23. The processing system of claim 21 wherein the plural processors comprise streaming multiprocessors.
24. The processing system of claim 21 wherein the plural processors each comprise a group of processing cores and memory local to and directly addressable by each of the processing cores in the group.