Multi-port network device using lookup cost backpressure
View Patent ↗In one embodiment of a network device, multiple packet sources contend for access to a packet processing pipeline. The packet processing pipeline tracks the usage of lookup resources by each of the multiple packet sources. When a packet source is detected to be using more than an acceptable allocation of the lookup resources, access to the packet processing pipeline for that source is limited or curtailed to bring that source back within an acceptable allocation of resources. This backpressure mechanism can be used to control sources that, although within a bandwidth limit, are submitting a packet type mix that is consuming unfair percentages of lookup resources in an oversubscribed system. Other embodiments are described and claimed.
1. A packet network device comprising:
multiple packet port buffers, at least one of which is assigned a lookup credit which is a relative cost associated with lookup operations performed by a lookup engine over a fixed period of time on a packet data type;
an arbiter for selecting packet data from the multiple packet port buffers; a packet processor;
a lookup credit manager to maintain a lookup credit account associated with the at least one packet port buffer which is assigned the lookup credit, the lookup credit account credited with the costs of the lookup operations and storing a current lookup credit value;
the lookup engine performing the lookup operations for packets received from the arbiter and sending the results of the lookup operations to the packet processor which the packet processor uses to modify at least one of the received packets and to send the cost of the lookup operations to the lookup credit manager; and
a backpressure channel to request the arbiter to reduce the relative rate of packets submitted to the lookup engine from the at least one packet port buffer that is assigned the lookup credit when the lookup credit value in the lookup credit account assigned to the at least one packet port buffer falls below a lower credit threshold.
2. The packet network device of claim 1 , wherein the credit manager adjusts the lookup credit account by a lookup cost when the lookup engine performs a lookup operation for a packet received from the first packet port buffer, the credit manager recurrently replenishing the first lookup credit account at a rate reflecting an allowable consumption of lookup resources for the first packet buffer, wherein the backpressure channel requests for reduction in a relative rate of packets are based on the value of the first lookup credit account.
3. The packet network device of claim 2 , further comprising:
a parser to determine a packet type for packets submitted to the lookup engine; and
an estimator to estimate the lookup cost from the packet type.
4. The packet network device of claim 3 , wherein the estimator comprises a lookup engine cost lookup table indexed by packet type.
5. The packet network device of claim 2 , wherein the lookup engine calculates the lookup cost for the packet when the lookup engine performs the lookup operation for that packet.
6. The packet network device of claim 2 , wherein the credit manager maintains a corresponding lookup credit account for each of the multiple packet port buffers, and wherein the backpressure channel submits requests for reduction in a relative rate of packets for each of the multiple packet buffers based on the value of the corresponding lookup credit account.
7. The packet network device of claim 2 , further comprising a backpressure threshold circuit, the backpressure threshold circuit signaling the backpressure channel to request a reduction in the relative rate of packets when the value of the first lookup credit account drops below a lower threshold.
8. The packet network device of claim 2 , the credit manager comprising a refresh credit timer, the refresh credit timer recurrently issuing credits for the first lookup credit account to replenish the first lookup credit account.
9. The packet network device of claim 2 , wherein the credit manager caps the lookup credit account at a maximum credit.
10. The packet network device of claim 1 , wherein once the backpressure threshold circuit signals the backpressure channel to request a reduction in the relative rate of packets, the backpressure threshold circuit will signal an increase in the relative rate of packets when the value of the first lookup credit account rises above an upper threshold.
11. The packet network device of claim 1 , wherein the packet port buffers reside on a first integrated circuit and the lookup engine resides on a second integrated circuit, the packet network device further comprising:
the arbiter on the first integrated circuit to select packet data from the multiple packet port buffers for transmission to the second integrated circuit;
a forward data bus connected from the first integrated circuit to the second integrated circuit, the forward data bus transmitting the packet data selected by the arbiter to the second integrated circuit; and
a reverse status bus connected from the second integrated circuit to the first integrated circuit, the backpressure channel operating over the reverse status bus to notify the arbiter of requests for reduction in the relative rate of packets selected from the first packet port buffer.
12. The packet network device of claim 11 , the second integrated circuit further comprising a status manager for the reverse status bus, the status manager merging the backpressure channel requests with requests for relative packet rate changes based on criteria other than lookup engine resource usage.
13. A method of operating a packet network device, the method comprising:
assigning a lookup credit to at least one of a plurality of packet port buffers comprising the packet network device, the lookup credit is a relative cost associated with lookup operations performed by a lookup engine over a fixed period of time on a packet data type;
maintaining a lookup credit account for tracking the relative costs associated with the lookup engine usage by at least a first packet port buffer;
performing lookup operations for packet data selected from the multiple port buffers, using the results of the lookup operations to modify at least some of the packet data and storing the cost of the lookup operations in the lookup credit account associated with the at least one packet port buffer; and
reducing the relative rate of packets submitted to the lookup engine from the first one of the plurality of packet port buffers when the lookup credit value in the lookup credit account assigned to that buffer fall below a lower credit threshold.
14. The method of claim 13 , wherein the relative rate of packets submitted to the lookup engine is reduced to zero when the lookup credit account assigned to that buffer falls below the lower credit limit.
15. The method of claim 13 , wherein tracking lookup resource usage comprises maintaining a first credit account for the first packet port buffer, and adjusting the value of the first credit account by a lookup cost when lookup resources are consumed by a packet from the first packet port buffer.
16. The method of claim 15 , further comprising recurrently issuing credits to the first credit account at a rate related to an acceptable lookup resource usage profile for the first packet port buffer.
17. The method of claim 16 , further comprising limiting the amount of credit that can be stored in the first credit account.
18. The method of claim 15 , further comprising comparing the value of the first credit account to a first threshold and setting a backpressure condition when the value of the first credit account does not pass the first threshold, the backpressure condition affecting the arbitration success of the first packet port buffer.
19. The method of claim 18 , further comprising, when the backpressure condition is set, comparing the value of the first credit account to a second threshold and clearing the backpressure condition when the value of the first credit account passes the second threshold.
20. The method of claim 15 , wherein tracking lookup resource usage further comprises maintaining corresponding credit accounts for each packet port buffer, and, when lookup resources are consumed by a packet from one of the packet port buffers, adjusting the value of a corresponding credit account by a lookup cost.
21. The method of claim 15 , further comprising calculating the lookup cost based on a number of clock cycles required to complete lookup operations for a packet.
22. The method of claim 15 , further comprising classifying a packet according to a packet type, and estimating the lookup cost for the packet according to the packet type.
23. The method of claim 22 , wherein estimating the lookup cost comprises storing estimated lookup costs in a cost lookup table indexed by packet type, and indexing the cost lookup table using the packet type to obtain a lookup cost.