Low latency host processor to coherent device interaction
In a computer system, a processor and an I/O device controller communicate with each other via a coherence interconnect and according to a cache coherence protocol. Registers of the I/O device controllers are mapped to the cache coherent memory space to allow the processor to treat the registers as cacheable memory. As a result, latency of processor commands executed by the I/O device controller is decreased, and size of data stored in the I/O device controller that can be accessed by the processor is increased from the size of a single register to the size of an entire cache line.
1 . A method of operating an I/O device controller that is connected to a processor via a coherence interconnect, wherein the I/O device controller includes registers, the method comprising:
mapping the registers to a cache cohernt memory space of the processor, and allocating a command queue and a completion queue in the cache coherent memory space, wherein processor interactions with I/O device controller, including reading from and writing to the registers of I/O device controller, are carried out thorugh the cache coherent memory space;
tracking accesses of the processor to cache line of registers, the command queue, and completion queue in the cache coherent memory space and detecting, as a result of said tracking, that the processor acquired ownership a first cache line, wherein the first cache line is one of a plurality of cache lines of the command queue stored in the cache coherent memory space and retrieved into a cache of the processor;
in response to a write-back transaction issued on the coherence interconnect, writing the first cache line to the cache coherent memory space, retrieving a command from the first cache line and executing the command; and
writing a result of executing the command to a second cache line, wherein the second cache line is one of a plurality of cache lines of the completion queue stored in the cache coherent memory space.
2 . The method of claim 1 , wherein
the first cache line is updated by the processor to add the command to a free slot of the command queue after the processor acquired ownership of the first cache line, and
the result is added to a free slot of the completion queue by the I/O device controller.
3 . The method of claim 2 , wherein each cache line of the command queue and the completion queue includes a plurality of slots.
4 . The method of claim 1 , further comprising:
causing the write-back transaction to be issued by the processor on the coherence interconnect.
5 . The method of claim 4 , further comprising:
sending a snoop message on the coherence interconnect to cause the processor to issue the write-back transaction on the coherence interconnect.
6 . The method of claim 1 , further comprising:
detecting a request on the coherence interconnect for the first cache line; and
sending the first cache line on the coherence interconnect.
7 . The method of claim 6 , further comprising:
storing an address of the first cache line in a buffer; and
in response to another request on the coherence interconnect for the first cache line,
sending a message to invalidate the first cache line on the coherence interconnect and
updating a state of the first cache line in the buffer to indicate that the first cache line is in an invalid state.
8 . A non-transitory computer-readable medium comprising instructions that are executable in an I/O device controller that is connected to a processor via a coherence interconnect, wherein the I/O device controller executes the instructions to carry out the steps of:
mapping the registers to a cache conherent memory space of the processor, and allocating a command queue and a completion in the cache coherent memory space, wherein processory interactions with the I/O device controller, including reading from and writing to the registers of the I/O device controller, are carried out through the cache coherent memory space;
tracking accesses of the processor to cache lines of the registers, the command queue, and the completion queue in the cache coherent memory space detecting, as a result of said tracking, that the processor acquired ownership of a a first cache line, wherein the first cache line is on of plurality of cache lines of the command queue stored in a the cache coherent memory space and retrieved into a cache of the processor;
in response to a write-back transaction issued on the coherence interconnect, writing the first cache line to the cache coherent memory space, retrieving a command from the first cache line and executing the command; and
writing a result of executing the command to a second cache line, wherein the second cache line is one of a plurality of cache lines of the completion queue stored in the cache coherent memory space.
9 . The non-transitory computer-readable medium of claim 8 , wherein
the first cache line is updated by the processor to add the command to a free slot of the command queue after the processor acquired ownership of the first cache line, and
tthe result is added to a free slot of the completion queue by the I/O device controller.
10 . The non-transitory computer-readable medium of claim 9 , wherein each cache line of the command queue and the completion queue includes a plurality of slots.
11 . The non-transitory computer-readable medium of claim 8 , wherein the steps further include:
causing the write-back transaction to be issued by the processor on the coherence interconnect.
12 . The non-transitory computer-readable medium of claim 11 , wherein the steps further include:
sending a snoop message on the coherence interconnect to cause the processor to issue the write-back transaction on the coherence interconnect.
13 . The non-transitory computer-readable medium of claim 8 , wherein the steps further include:
detecting a request on the coherence interconnect for the first cache line; and
sending the first cache line on the coherence interconnect.
14 . The non-transitory computer-readable medium of claim 13 , wherein the steps further include:
storing an address of the first cache line in a buffer; and
in response to another request on the coherence interconnect for the first cache line, sending a message to invalidate the first cache line on the coherence interconnect and updating a state of the first cache line in the buffer to indicate that the first cache line is in an invalid state.
15 . A computer system comprising:
a processor; and
an I/O device controller communicating with the processor via a coherence interconnect and according to a cache coherence protocol, wherein the I/O device controller includes registers that are mapped to a cache coherent memory space of the processor and a command queue and a completion queue that are also allocated in the cache coherent memory space, and wherein processor interactions with I/O device controller, including reading from and writing to the registers of the I/O device controller, are carried out through the cache coherent memory space,
the I/O device controller configured to:
track accesses of the processor to cache lines of the registers, the command queue, and the completion queue in the cache coherent memory space and detect, as a result of said tracking, that the processor acquired ownership of a first cache line, wherein the first cache line is one of plurality of cache lines of the command queue stored in the cache coherent memory space and retrieved into a cache of the processor;
in response to a write-back transaction issued on the coherence interconnect, write the first cache line to the cache coherent memory space, retrieve a command from the first cache line, and execute the command; and
write a result of executing the command to a second cache line, wherein the second cache line is one of plurality of cache lines of the completion queue stored in the cache coherent memory space.
16 . The computer system of claim 15 , wherein
the first cache line is updated by the processor to add the command to a free slot of the command queue after the processor acquired ownership of the first cache line, and
the result is added to a free slot of the completion queue by the I/O device controller.
17 . The computer system of claim 16 , wherein each cache line of the command queue and the completion queue includes a plurality of slots.
18 . The computer system of claim 15 , wherein the I/O device controller is futher configured to:
cause the write-back transaction to be issues by the processor on the coherence interconnect.
19 . The computer system of claim 18 , wherein the I/O device controller is further configured to:
send a snoop message on the coherence interconnect to cause the processor to issue the write-back transaction on the coherence interconnect.