Managing an out-of-order asynchronous heterogeneous remote direct memory access (RDMA) message queue

1. A system, comprising:

a first processing device having a first local memory for incoming messages from a second processing device;

the second processing device having a second local memory for incoming messages from the first processing device;

a memory storage location, wherein the first and second local memories are mapped to the memory storage location; and

a data bus coupled to the first processing device, the second processing device, and the memory storage location, wherein:

the first processing device and the second processing device cooperate to process queue data within a shared message queue of the memory storage location, wherein the queue data comprises first processing device queue data and second processing device queue data;

the first processing device and the second processing device access the queue data within the shared message queue via remote direct memory access (RDMA); and

responsive to at least one of the first and second processing devices accessing the shared message queue via an RDMA operation, a lock is automatically placed on the shared message queue, and wherein the at least one of the first and second processing devices obtaining the lock obtains exclusive use of the queue data and obtains both the first processing device queue data and the second processing device queue data by copying to the respective first or second local memory the first processing device queue data and the second processing device queue data.

2. The system of claim 1 , wherein the queue data acquired from the shared message queue comprises all data necessary to manage the queue.

3. The system of claim 1 , wherein the first processing device comprises a power processing unit (PPU).

4. The system of claim 3 , wherein the second processing device comprises a synergistic processing unit (SPU).

5. The system of claim 1 , wherein, to retrieve the queue data in the shared message queue, at least one of the first and second processing devices:

checks to see if the queue data is locked;

responsive to the queue data being unlocked, performs an atomic GET to retrieve and lock the queue data in a single RDMA;

operate on the queue data to produce updated queue data; and

performs an atomic PUT to return the updated queue data.

6. The system of claim 1 , wherein another processing device cannot operate on the queue data when the queue data is locked.

7. A method comprising:

determining, by a first processing device having a first local memory for incoming messages from a second processing device, if a shared queue data in a memory storage location to be processed by the first processing device is locked, the second processing device having a second local memory for incoming messages from the first processing device;

responsive to determining that the shared queue data is unlocked, acquiring, by the first processing device, the shared queue data by copying the shared queue data to the first local memory and acquiring a shared queue lock in a single RDMA providing exclusive use of the shared queue data by the first processing device, wherein the shared queue data acquired comprises queue data for the first processing device and queue data for the second processing device;

processing the shared queue data, by the first processing device, to produce updated queue data; and

outputting the updated queue data to the memory storage location and releasing the shared queue lock.

8. The method of claim 7 , wherein:

acquiring the shared queue data and the shared queue lock comprises an atomic GET; and

outputting the updated queue data and releasing the shared queue lock comprises an atomic PUT.

9. The method of claim 7 , wherein acquiring the shared queue data and the shared queue lock when the shared queue data is unlocked comprises:

fetching the queue data from the memory storage location;

setting a lock flag to locked; and

writing the queue data with a locked lock flag to the memory storage location.

10. The method of claim 7 , wherein the queue data comprises actual data to be processed.

11. The method of claim 7 , wherein the queue data comprises an address and size associated with actual data to be processed.

12. A system comprising:

a first processing device wherein a first memory domain is associated with the first processing device;

a second processing device wherein a second memory domain is associated with the second processing device;

a memory storage location, wherein the first memory domain and the second memory domain are mapped to locations within the memory storage location;

a data bus coupled to the first processing device, the second processing device, and the memory storage location, wherein:

the first processing device and the second processing device cooperate to process queue data within a shared message queue of the memory storage location, wherein the queue data comprises first processing device queue data and second processing device queue data;

the first processing device and the second processing device access the queue data within the shared message queue via direct memory access (DMA); and

responsive to at least one of the first and second processing devices accessing the shared message queue, a lock is automatically placed on the shared message queue and the queue data retrieved in a single RDMA, and wherein the at least one of the first and second processing devices obtaining the lock obtains exclusive use of the queue data and obtains both the first processing device queue data and the second processing device queue data; and

wherein a queue order in the shared message queue is managed by an ordering array of indices that correspond to entries in the shared message queue, and wherein queue entries may be removed from the queue out-of-order by:

traversing a queue order array by ascending index up to a free index looking for a message match at each corresponding message queue slot;

in response to a message match being found, shifting contents of the queue order array up, in order, starting at a message queue slot following the slot of the message match;

moving the contents of the message match slot to a last slot entry; and

decrementing the free index.

13. The system of claim 12 , wherein the queue data acquired when the queue data is locked comprises first processing device queue data and second processing device queue data necessary to manage the shared message queue.

14. The system of claim 12 , wherein the first processing device comprises:

a power processing unit (PPU); and

a main memory location.

15. The system of claim 14 , wherein the second processing device comprises:

a synergistic processing unit (SPU); and

a local memory location.

16. The system of claim 12 , wherein, to retrieve the queue data in the shared message queue, at least one of the first and second processing devices:

checks to see if queue data is locked;

responsive to determining that the shared queue data is unlocked, performs an atomic GET to retrieve and lock the queue data in the single RDMA;

operate on the queue data to produce updated queue data; and

performs an atomic PUT to return the updated queue data.

17. The system of claim 12 , wherein another processing device cannot operate on the queue data when the queue data is locked.

18. A system, comprising:

a first processing device;

a second processing device;

a memory storage location; and

a data bus coupled to the first processing device, the second processing device, and the memory storage location, wherein:

the first processing device and the second processing device cooperate to process queue data within a shared message queue of the memory storage location, wherein the queue data comprises first processing device queue data and second processing device queue data;

the first processing device and the second processing device access the queue data within the shared message queue via remote direct memory access (RDMA); and

responsive to at least one of the first and second processing devices accessing the shared message queue via an RDMA operation, a lock is automatically placed on the shared message queue, and wherein the at least one of the first and second processing devices obtaining the lock obtains exclusive use of the queue data and obtains both the first processing device queue data and the second processing device queue data; and

wherein a queue order in the shared message queue is managed by an ordering array of indices that correspond to entries in the shared message queue, and wherein queue entries may be removed from the queue out-of-order by:

traversing a queue order array by ascending index up to a free index looking for a message match at each corresponding message queue slot;

in response to a message match being found, shifting contents of the queue order array up, in order, starting at a message queue slot following the slot of the message match;

moving the contents of the message match slot to a last slot entry; and

decrementing the free index.