Selective load balancing for spraying over fabric paths
A network device includes a memory, a plurality of packet processors, a switch fabric coupling the plurality of processors, and processing circuitry. The processing circuitry is configured to receive a data stream to be transmitted on a switch fabric and determine a plurality of credit counts, each credit count being assigned to a respective subchannel of a plurality of subchannels. The packet processor is further configured to determine per-subchannel occupancy of the memory for the plurality of subchannels, select, based on the plurality of credit counts and the per-subchannel occupancy of the memory, a subchannel of the plurality of subchannels for transmitting a cell of a plurality of cells for the data stream, and output data for the cell to the memory for output by the selected subchannel.
1. A method comprising:
receiving, by processing circuitry of a network device, a data stream to be transmitted on a switch fabric of the network device, the switch fabric coupling a plurality of packet processors to a plurality of fabric endpoints of the switch fabric;
determining, by the processing circuitry, a plurality of credit counts, each credit count being assigned to a respective subchannel of a plurality of subchannels, each subchannel of the plurality of subchannels corresponding to a respective fabric path through the switch fabric from an ingress fabric endpoint to an egress fabric endpoint of the plurality of fabric endpoints;
determining, by the processing circuitry, per-subchannel occupancy of a memory for the plurality of subchannels, wherein the per-subchannel occupancy indicates whether a number of bytes for each subchannel of the plurality of subchannels is less than an occupancy threshold;
selecting, by the processing circuitry, a subchannel of the plurality of subchannels for transmitting a cell of a plurality of cells for the data stream in response to determining a credit count of the plurality of credit counts that is assigned to the selected subchannel is greater than a credit threshold for the selected subchannel and in response to determining that the per-subchannel occupancy indicates that the number of bytes for the subchannel is less than the occupancy threshold; and
outputting, by the processing circuitry, data for the cell to the memory for output by the selected subchannel.
2. The method of claim 1 , further comprising:
retrieving, by the processing circuitry, after outputting the data for the cell to the memory, the data for the cell from the memory; and
outputting, by the processing circuitry, the data for the cell at the selected subchannel.
3. The method of claim 1 , wherein a packet processor of the plurality of packet processors comprises the processing circuitry.
4. The method of claim 1 , further comprising:
updating, by the processing circuitry and based on selecting the subchannel, the per-subchannel occupancy of the memory.
5. The method of claim 4 , wherein updating the per-subchannel occupancy of the memory comprises:
determining an indication of an occupancy for the data for the cell in the memory; and
modifying an entry in a table of occupancy per-subchannel and per destination identifier that corresponds to the selected subchannel and the egress fabric endpoint using the indication of the occupancy for the data for the cell.
6. The method of claim 1 , further comprising:
updating, by the processing circuitry, after selecting the subchannel, the plurality of credit counts.
7. The method of claim 6 , wherein updating the plurality of credit counts comprises:
determining a credit cost for the data for the cell; and
modifying an entry in a table of credits per-subchannel and per destination identifier that corresponds to the selected subchannel and the egress fabric endpoint using the credit cost.
8. The method of claim 7 , wherein updating the plurality of credit counts comprises:
determining the table indicates that each subchannel for the egress fabric endpoint has negative credits; and
setting, for each subchannel of the plurality of subchannels, an entry in the table that corresponds to a respective subchannel and the egress fabric endpoint to a respective replenished credit count.
9. A network device comprising:
a memory;
a plurality of packet processors;
a switch fabric coupling the plurality of packet processors to a plurality of fabric endpoints of the switch fabric; and
processing circuitry configured to:
receive a data stream to be transmitted on the switch fabric;
determine a plurality of credit counts, each credit count being assigned to a respective subchannel of a plurality of subchannels, each subchannel of the plurality of subchannels corresponding to a respective fabric path through the switch fabric from an ingress fabric endpoint to an egress fabric endpoint of the plurality of fabric endpoints;
determine per-subchannel occupancy of the memory for the plurality of subchannels, wherein the per-subchannel occupancy indicates whether a number of bytes for each subchannel of the plurality of subchannels is less than an occupancy threshold;
select a subchannel of the plurality of subchannels for transmitting a cell of a plurality of cells for the data stream in response to a determination that a credit count of the plurality of credit counts that is assigned to the selected subchannel is greater than a credit threshold for the selected subchannel and in response to a determination that the per-subchannel occupancy indicates that the number of bytes for the subchannel is less than the occupancy threshold; and
output data for the cell to the memory for output by the selected subchannel.
10. The network device of claim 9 , wherein the processing circuitry is configured to:
retrieve, after outputting the data for the cell to the memory, the data for the cell from the memory; and
output the data for the cell at the selected subchannel.
11. The network device of claim 9 , wherein the ingress fabric endpoint comprises the processing circuitry.
12. The network device of claim 9 , wherein a packet processor of the plurality of packet processors comprises the processing circuitry.
13. The network device of claim 9 , wherein the processing circuitry is configured to:
update, based on selecting the subchannel, the per-subchannel occupancy of the memory.
14. The network device of claim 13 , wherein, to update the per-subchannel occupancy of the memory, the processing circuitry is configured to:
determine an indication of an occupancy for the data for the cell in the memory; and
modify an entry in a table of occupancy per-subchannel and per destination identifier that corresponds to the selected subchannel and the egress fabric endpoint using the indication of the occupancy for the data for the cell.
15. The network device of claim 9 , wherein the processing circuitry is configured to:
update, after selecting the subchannel, the plurality of credit counts.
16. The network device of claim 15 , wherein, to update the plurality of credit counts, the processing circuitry is configured to:
determine a credit cost for the data for the cell; and
modify an entry in a table of credits per-subchannel and per destination identifier that corresponds to the selected subchannel and the egress fabric endpoint using the credit cost.
17. The network device of claim 16 , wherein, to update the plurality of credit counts, the processing circuitry is configured to:
determine the table indicates that each subchannel for the egress fabric endpoint has negative credits; and
set, for each subchannel of the plurality of subchannels, an entry in the table that corresponds to a respective subchannel and the egress fabric endpoint to a respective replenished credit count.
18. A non-transitory computer-readable storage medium comprising instructions for causing processing circuitry of a network device to:
receive a data stream to be transmitted on a switch fabric of the network device, the switch fabric coupling a plurality of packet processors to a plurality of fabric endpoints of the switch fabric;
determine a plurality of credit counts, each credit count being assigned to a respective subchannel of a plurality of subchannels, each subchannel of the plurality of subchannels corresponding to a respective fabric path through the switch fabric from an ingress fabric endpoint to an egress fabric endpoint of the plurality of fabric endpoints;
determine per-subchannel occupancy of a memory for the plurality of subchannels, wherein the per-subchannel occupancy indicates whether a number of bytes for each subchannel of the plurality of subchannels is less than an occupancy threshold;
select a subchannel of the plurality of subchannels for transmitting a cell of a plurality of cells for the data stream in response to a determination that a credit count of the plurality of credit counts that is assigned to the selected subchannel is greater than a credit threshold for the selected subchannel and in response to a determination that the per-subchannel occupancy indicates that the number of bytes for the subchannel is less than the occupancy threshold; and
output data for the cell to the memory for output by the selected subchannel.