Source ordering in device interconnects
In one embodiment, an apparatus includes a port to transmit and receive data over a link; and protocol stack circuitry to implement one or more layers of a load-store input/output (I/O)-based protocol (e.g., PCIe or CXL) across the link. The protocol stack circuitry constructs memory write request transaction layer packets (TLPs) for memory write transactions, wherein fields of the memory write request TLPs indicate a virtual channel (VC) other than VC0, that a completion is required in response to the memory write transaction, and a stream identifier associated with the memory write transaction. The memory write request TLP is transmitted over the link and a completion TLP is received over the link in response, indicating a completion for the memory write request TLP.
1. An apparatus comprising:
a port to transmit and receive data over a link; and
protocol stack circuitry to implement one or more layers of a Peripheral Component Interconnect Express (PCIe)-based or a Compute Express Link (CXL)-based protocol across the link, wherein protocol stack circuitry is to:
receive a request to initiate a memory write transaction on the link;
construct a memory write request transaction layer packet (TLP) for the memory write transaction, wherein fields of the memory write request TLP indicate a virtual channel (VC) other than VC0 for the TLP, that a completion is required in response to the memory write transaction, and a stream identifier associated with the memory write transaction;
cause the memory write request TLP to be transmitted over the link; and
process a completion TLP received over the link, the completion TLP indicating a completion for the memory write request TLP.
2. The apparatus of claim 1 , wherein the memory write request TLP is a Memory Write TLP, and the indication that a completion is required in response to the memory write transaction is included in two upper bits of a tag field of the Memory Write TLP, two reserved bits of the Memory Write TLP, or a combination of 1 tag field bit and 1 reserved bit of the Memory Write TLP.
3. The apparatus of claim 1 , wherein the memory write request TLP is a Streamed Write TLP, and the indication that a completion is required in response to the memory write transaction is included in one or more fields of the Streamed Write TLP.
4. The apparatus of claim 3 , wherein the Streamed Write TLP is indicated with a value other than 0100_0000b.
5. The apparatus of claim 4 , wherein the Streamed Write TLP is indicated with a value of 0101_1100b or 0101_1101b.
6. The apparatus of claim 3 , wherein the protocol stack circuitry is to construct a plurality of Streamed Write TLPs associated with a same stream identifier, the completion TLP is a Streamed Write Completion TLP that indicates a completion for each the plurality of Streamed Write TLPs.
7. The apparatus of claim 6 , wherein the Streamed Write Completion TLP is indicated with a value other than 0000_1010b or 0100_1010b.
8. The apparatus of claim 7 , wherein the Streamed Write Completion TLP is indicated with a value of 0000_1100b.
9. The apparatus of claim 6 , wherein a tag field of the completion TLP indicates the stream identifier associated with the plurality of Streamed Write TLPs.
10. The apparatus of claim 6 , wherein a byte count field of the completion TLP indicates a number of completions indicated by the completion TLP.
11. The apparatus of claim 1 , wherein the stream identifier is indicated in a tag field of the memory write request TLP.
12. The apparatus of claim 1 , wherein the memory write request TLP further includes a field indicating that completion coalescence is allowed.
13. The apparatus of claim 1 , wherein the memory write request TLP is tracked using a non-posted transaction type buffer.
14. A method comprising:
receiving a request to initiate a memory write transaction on a link implementing a Peripheral Component Interconnect Express (PCIe)-based or a Compute Express Link (CXL)-based protocol;
constructing a memory write request transaction layer packet (TLP) for the memory write transaction, wherein fields of the memory write request TLP indicate a virtual channel (VC) other than VC0 for the TLP, that a completion is required in response to the memory write transaction, and a stream identifier associated with the memory write transaction;
transmitting the memory write request TLP over the link; and
receiving a completion TLP over the link, the completion TLP indicating a completion for the memory write request TLP.
15. The method of claim 14 , wherein the memory write request TLP is a Memory Write TLP, and the indication that a completion is required in response to the memory write transaction is included in two upper bits of a tag field of the Memory Write TLP, two reserved bits of the Memory Write TLP, or a combination of 1 tag field bit and 1 reserved bit of the Memory Write TLP.
16. The method of claim 14 , wherein the memory write request TLP is a Streamed Write TLP, and the indication that a completion is required in response to the memory write transaction is included in one or more fields of the Streamed Write TLP.
17. The method of claim 16 , further comprising constructing a plurality of Streamed Write TLPs associated with a same stream identifier, the completion TLP is a Streamed Write Completion TLP that indicates a completion for each the plurality of Streamed Write TLPs.
18. The method of claim 17 , wherein a tag field of the completion TLP indicates the stream identifier associated with the plurality of Streamed Write TLPs.
19. The method of claim 17 , wherein a byte count field of the completion TLP indicates a number of completions indicated by the completion TLP.
20. The method of claim 14 , wherein the stream identifier is indicated in a tag field of the memory write request TLP.
21. A system comprising:
a processor;
a first device; and
a second device;
wherein the processor, first device, and second device are coupled to one another via a Peripheral Component Interconnect Express (PCIe)-based or Compute Express Link (CXL)-based interconnect, and each includes protocol circuitry to:
construct memory write request transaction layer packets (TLPs) for memory write transactions to be performed on the interconnect, wherein fields of the memory write request TLPs indicate a virtual channel (VC) other than VC0 for the TLPs, that a completion is required in response to the memory write transaction, and a stream identifier associated with the memory write transaction; and
process completion TLPs received over the interconnect indicating completions for the memory write request TLPs.
22. The system of claim 21 , wherein the processor is a root port of the PCIe-based or CXL-based interconnect, and the system further comprises a switch positioned in the interconnect between the processor, the first device, and the second device, wherein the switch is to transmit memory write request TLPs from the first device to the second device without involving the processor.
23. The system of claim 21 , further comprising a CXL-based switch positioned in the interconnect between the processor, the first device, and the second device, wherein the switch is to transmit memory write request TLPs from the first device to the second device using a CXL.io protocol.