Read control in a computer I/O interconnect
View Patent ↗In one embodiment, a method for controlling reads in a computer input/output (I/O) interconnect is provided. A read request is received over the computer I/O interconnect from a first device, the request requesting data of a first size. Then it is determined whether fulfilling the read request would cause the total size of a completion queue to exceed a first predefined threshold. If fulfilling the read request would cause the total size of the completion queue to exceed the first predefined threshold, then the read request is temporarily restricted from being forwarded upstream.
1. A method for controlling reads in a packetized computer input/output (I/O) interconnect compatible with a packetized computer I/O interconnect standard, the method comprising:
receiving a read request at a root complex over the packetized computer I/O interconnect from a first device compatible with the packetized computer I/O interconnect standard, the request requesting data of a first size;
determining whether adding the read request to a read request queue at the root complex would cause the total size of requested data in read requests from the first device currently stored in the read request queue to exceed a first predefined threshold, wherein the first predefined threshold is based on a round trip time between the root complex and a destination switch of the read request; and
temporarily restricting the read request from being forwarded upstream to the destination switch of the read request if adding the read request to the read request queue would cause the total size of the requested data in read requests from the first device currently stored in the read request queue to exceed the first predefined threshold.
2. The method of claim 1 , further comprising:
receiving a second read request at the root complex over the packetized computer I/O interconnect from a second device compatible with the packetized computer I/O interconnect standard;
determining whether adding the read request from the second device to the second read request queue would cause the total size of requested data in read requests from the second device stored in the read request queue to exceed a second predefined threshold, wherein the first predefined threshold is based on a round trip time between the root complex and a destination switch of the second read request;
if the second read request from the second device would not cause the total size of requested data in read requests from the second device stored in the read request queue to exceed the second predefined threshold, determining if forwarding the second read request upstream would cause the rate at which read requests are forwarded upstream to exceed a drain rate of a completion queue by more than a third predefined threshold; and
temporarily restricting the second read request from the second device from being forwarded upstream if forwarding the second read request from the second device upstream would cause the rate at which read requests are forwarded upstream to exceed the drain rate of the completion queue by more than the third predefined threshold.
3. The method of claim 2 , wherein the third predefined threshold is expressed as a multiplication factor between the rate at which read requests are forwarded upstream and the drain rate of the completion queue.
4. The method of claim 1 , wherein the packetized computer I/O interconnect standard is PCIe.
5. The method of claim 1 , wherein the first predefined threshold is based on the round trip time between the root complex and the destination switch of the read request and is determined by dividing a memory size available for a completion queue by a number of ports of the switch that are connected to aggressive reading devices.
6. The method of claim 1 , wherein the determining includes adding the first size to a read counter corresponding to the first device and comparing the read counter to the first predetermined threshold.
7. The method of claim 6 , further comprising, if fulfilling the read request would not cause the total size of the requested data in read requests from the first device currently stored in the read request queue to exceed the first predefined threshold:
forwarding the read request upstream; and
adding the first size to the read counter.
8. The method of claim 7 , further comprising:
when the read request is fulfilled, subtracting the first size from the read counter.
9. A root complex compatible with a packetized computer I/O interconnect standard, the root complex comprising:
an upstream request queue;
a downstream completion queue;
an interface to a switch;
an interface to a first device compatible with the packetized computer I/O interconnect standard; and
a memory controller adapted to:
receive a read request from the first device first device, the request requesting data of a first size from a device coupled to the switch;
determine whether adding the read request to the read request queue would cause the total size of requested data in read requests from the first device currently stored in the read request queue to exceed a first predefined threshold, wherein the first predefined threshold is based on a round trip time between the root complex and the switch; and
temporarily restrict the read request from being forwarded upstream to the switch if adding the read request to the read request queue would cause the total size of the requested data in read requests from the first device currently stored in the read request queue to exceed the first predefined threshold.
10. The root complex of claim 9 , further comprising a different upstream read request queue for each connected device.
11. The root complex of claim 9 , further comprising a different downstream completion queue for each connected switch.
12. An apparatus for controlling reads in a packetized computer input/output (I/O) interconnect compatible with a packetized computer I/O interconnect standard, the apparatus comprising:
means for receiving a read request over the packetized computer I/O interconnect from a first device compatible with the packetized computer I/O interconnect standard, the request requesting data of a first size;
means for determining whether adding the read request to a read request queue would cause the total size of requested data in read requests from the first device currently stored in the read request queue to exceed a first predefined threshold, wherein the first predefined threshold is based on a round trip time between the means for receiving and a destination switch of the read request; and
means for temporarily restricting the read request from being forwarded upstream to the destination switch of the read request if adding the read request to the read request queue would cause the total size of the requested data in read requests from the first device currently stored in the read request queue to exceed the first predefined threshold.
13. The apparatus of claim 12 , wherein the first predetermined threshold is determined by dividing a memory size available for a completion queue by a number of ports of the switch that are connected to aggressive reading devices.
14. The apparatus of claim 12 , wherein the apparatus is a switch.
15. The apparatus of claim 12 , wherein the means for determining includes means for adding the first size to a read counter corresponding to the first device and comparing the read counter to the first predetermined threshold.
16. The apparatus of claim 15 , further comprising, means for, if fulfilling the read request would not cause the total size of the requested data in read requests from the first device currently stored in the read request queue to exceed the first predefined threshold:
forwarding the read request upstream; and
adding the first size to the read counter.
17. The apparatus of claim 16 , further comprising:
means for, when the read request is fulfilled, subtracting the first size from the read counter.
18. The apparatus of claim 12 , wherein the first predetermined threshold is set based on a size of memory available for a completion queue.