IP Library Granted Patent US 6,963,576
Granted Patent B1
US 6,963,576 · App. 09/676,046 · Granted Nov 8, 2005

Scheduling and arbitration scheme for network processing device

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 6,963,576
App. No.
09/676,046
Granted
Nov 8, 2005
Kind
B1
Abstract

An arbitration scheme is used for scheduling connections between input ports and output ports. Input ports request connections to the output ports for a next time slot. Arbitration parameters, such as priority and weight, are identified for the buffer requests. Output port arbitrations are conducted for each one of the output ports according to the arbitration parameters. If there are more than two input buffers with the same priority and weight, a round robin arbitration is used. Grants are issued to the input port buffers winning the output port arbitrations. Input port arbitrations are conducted using the same arbitration parameters for input ports receiving multiple grants. The grants are accepted by the input port buffers winning the input port arbitrations. The input port buffers accepting the grants are connected to the requested output ports. Virtual output buffers are used to prevent packet blocking at the input ports and a dual multicast and unicast arbitration is used for multicast and unicast packets.

Claims (94)

1. A packet scheduler, comprising:

inputs configured to receive requests for connecting multiple input ports to multiple output ports;

outputs for configuring multiple concurrent connections according to a matching between the multiple input ports and the multiple output ports; and

an arbitration circuit configured to conduct a first arbitration phase for a next time slot where multiple independent arbitrations are conducted concurrently for each of the multiple output ports, the arbitration circuit selecting the input ports for each of the output ports for the next time slot according to both a priority and weight of packets at the input ports used for the multiple independent arbitrations, wherein the input port weight of packets for each output port arbitration is based on the amount of data accumulated at that input port for forwarding to the output port related to that arbitration.

2. A scheduler according to claim 1 wherein the input ports that win multiple ones of the multiple arbitrations accept one of the arbitration wins and reject the other arbitration wins, the arbitration circuit then conducting another arbitration phase for the next time slot with independent arbitrations for each of the output ports that did not receive arbitration win acceptance from input ports during the first arbitration phase.

3. A scheduler according to claim 2 including timers that are activated anytime one of the input ports requests a connection to one of the output ports, the arbitration circuit increasing the priority for any input ports having unserviced connection requests extending beyond a timer period.

4. A scheduler according to claim 1 wherein the arbitration circuit conducts output port arbitrations for each one of the output ports and conducts input port arbitrations for each one of the input ports winning multiple output port arbitrations.

5. A scheduler according to claim 4 wherein the output port arbitrations and the input port arbitrations are conducted for both multicast packets and unicast packets for the next time slot.

6. A scheduler according to claim 1 wherein each one of the input ports has associated virtual output queues each dedicated to a different one of the output ports.

7. A scheduler according to claim 6 wherein the arbitration circuit conducts output port arbitrations for all of the virtual output queues dedicated to the same output ports and conducts input port arbitrations between the virtual output queues for the same input port issued grants during the output port arbitrations.

8. A scheduler according to claim 7 wherein the output port arbitrations and the input port arbitrations are conducted according to both the priority and number of bytes of the packets associated with the virtual output queues.

9. A scheduler according to claim 8 including output port pointers for conducting round robin arbitrations during the output port arbitrations and input port pointers for conducting round robin arbitrations during the input port arbitrations.

10. A scheduler according to claim 9 including a cross switch that is configured by the arbitration circuit to connect the input ports to the output ports during individual time slots according to the output port arbitrations and the input port arbitrations.

11. A scheduler according to claim 1 wherein the arbitration circuit conducts a multicast arbitration that establishes connections for multicast packets during a next time slot and then conducts a unicast arbitration that establishes connections for unicast packets during the next time slot for any remaining unassigned output ports.

12. A scheduler according to claim 11 wherein the multicast arbitration and the unicast arbitration are conducted for both output port arbitrations and also for input port arbitrations.

13. A packet scheduler, comprising:

inputs configured to receive requests for connecting input ports to output ports;

outputs for configuring connections between the input ports and the output ports; and

an arbitration circuit configured to arbitrate between the input ports for connections to the output ports, the arbitration circuit selecting the input ports for a next time slot according to both a priority and weight of packets at the input ports,

wherein the arbitration circuit conducts a multicast arbitration that establishes connections for multicast packets during the next time slot and then conducts a unicast arbitration that establishes connections for unicast packets during the next time slot for any remaining unassigned output ports, the multicast arbitration and the unicast arbitration conducted for both output port arbitrations and also for input port arbitrations and the arbitration circuit is programmable to vary a percentage of output ports assignable during the multicast arbitration.

14. A method for scheduling multiple concurrent connections between multiple input ports and multiple output ports, comprising:

receiving requests from multiple input port buffers for connections to the multiple output ports during a next time slot;

identifying arbitration parameters for the requests;

conducting multiple independent output port arbitrations for each one of the multiple output ports according to the arbitration parameters;

issuing grants to the input port buffers winning the output port arbitrations for each of the multiple output ports;

conducting input port arbitrations for input ports receiving grants from multiple output port arbitrations;

accepting one of the multiple grants for each of the input port arbitrations; and

connecting the multiple output ports to the input port buffers accepting the grants;

wherein the arbitration parameters include a weight that varies according to a number of packet bytes in the input port buffers and a priority of the packets in the input port buffers.

15. A method according to claim 14 including:

selecting one of the weight and the priority to identify a first highest arbitration status;

using the other nonselected weight and priority to identify a second highest arbitration status for two or more input port buffers having the same first highest arbitration status; and

issuing grants to the input port buffers according to the first highest arbitration status when packet for only one input port buffer has the first highest arbitration status and issuing grants to the input port buffers according to the second highest arbitration status when two or more input port buffers have the first highest arbitration status.

16. A method according to claim 15 including using a round robin arbitration when two or more input port buffers have the same first highest arbitration status and the same second highest arbitration status.

17. A method according to claim 14 including dedicating one input port buffer in each input port to one of the output ports and using the input port buffers as virtual output buffers for temporarily storing packets assigned to the output ports to which the input port buffers are dedicated.

18. A method according to claim 14 including conducting a first round robin arbitration when multiple packets have a same highest priority and a same weight during the output port arbitrations and conducting a second round robin arbitration during the input port arbitrations when multiple input port buffers for the same input ports have been issued grants during the output port arbitration and have the same highest priority and the same weight.

19. A method for scheduling connections between multiple input ports and multiple output ports, comprising:

receiving requests from multiple input port buffers for connections to the multiple output ports during a next time slot;

identifying arbitration parameters for the requests;

conducting multiple independent output port arbitrations for each one of the multiple output ports according to the arbitration parameters;

issuing grants to the input port buffers winning the output port arbitrations for each of the multiple output ports;

conducting input port arbitrations for input ports receiving grants from multiple output port arbitrations;

accepting one of the multiple grants for each of the input port arbitrations;

conducting second independent arbitrations between each of the output ports that did not receive a grant acceptance from one of the input ports and any of the input port buffers that did not previously receive grants; and

connecting the multiple output ports to the input port buffers accepting the grants.

20. A method for scheduling connections between multiple input ports and multiple output ports, comprising:

receiving requests from multiple input port buffers for connections to the multiple output ports during a next time slot;

identifying arbitration parameters for the requests;

conducting a multicast arbitration for multicast packets in the input port buffers;

conducting multiple independent output port arbitrations for each one of the multiple output ports according to the arbitration parameters;

granting one of the input port buffers winning the multicast arbitration all output ports identified in an associated multicast group vector;

issuing grants to the input port buffers winning the output port arbitrations for each of the multiple output ports;

conducting input port arbitrations for input ports receiving grants from multiple output port arbitrations;

accepting one of the multiple grants for each of the input port arbitrations; and

connecting the multiple output ports to the input port buffers accepting the grants.

21. A method according to claim 20 including conducting a unicast arbitration after the multicast arbitration.

22. A method for scheduling connections between input ports and output ports, comprising:

receiving requests from input port buffers for connections to the output ports during a next time slot;

identifying arbitration parameters for the requests;

conducting output port arbitrations for each one of the output ports according to the arbitration parameters;

issuing grants to the input port buffers winning the output port arbitrations;

conducting input port arbitrations for input port buffers receiving grants from the output port arbitrations;

accepting one of the grants to one of the input port buffers at each input port buffer winning at least one output port arbitration;

connecting the input port buffers accepting the grants to the requested output ports;

conducting a multicast arbitration for multicast packets in the input port buffers;

granting one of the input port buffers winning the multicast arbitration all output ports identified in an associated multicast group vector;

identifying multicast group vectors for the multicast packets;

conducting multicast input port arbitrations for identifying a highest one of the multicast group vectors for each one of the input port buffers;

conducting multicast output port arbitrations using the highest multicast group vectors for the input port buffers; and

establishing connections for the multicast group vector winning the multicast output port arbitrations.

23. A method according to claim 22 including:

comparing grants issued from the output port arbitrations with the multicast group vectors; and

accepting the grants matching the multicast group vectors.

24. A network processing device, comprising:

multiple input ports for receiving incoming packets;

multiple output ports for outputting packets;

a cross switch coupled to the input ports and the output ports;

a scheduler that configures the cross switch for connecting multiple selected ones of the input ports to multiple selected ones of the output ports at the same time; and

multiple virtual output buffers associated with each one of the input ports, each one of the virtual output buffers dedicated to a different one of the output ports, the scheduler conducting multiple independent output port arbitrations for each one of the multiple output ports and the associated dedicated virtual output buffers and configuring the cross-switch according to the multiple independent arbitrations, wherein the scheduler conducts a multicast arbitration before each time slot to select virtual output buffers to connect to multiple output ports, the scheduler then conducting a unicast arbitration for connecting any unselected virtual output buffers to unselected output ports.

25. A network processing device according to claim 24 wherein the scheduler conducts the multicast arbitration and the unicast arbitration both for the virtual output buffers associated with the same input ports and for the virtual output buffers assigned to the same output ports.

26. A network processing device according to claim 25 wherein the multicast arbitration and the unicast arbitration:

issue grants according to priority of the packets in the virtual output buffers;

issue grants according to a weight based on a number of bytes in the packets for a virtual output buffer when packets for two or more virtual output buffers have a same highest priority; and

issue grants according to a round robin order when packets for two or more virtual output buffers have the same highest priority and a same weight.

27. A network processing device according to claim 26 wherein the multicast arbitration and the unicast arbitration determine connections between the input ports and the output ports before each time slot.

28. A network processing device, comprising:

multiple input ports for receiving incoming packets;

multiple output ports for outputting packets;

a cross switch coupled to the input ports and the output ports;

a scheduler that configures the cross switch for connecting multiple selected ones of the input ports to multiple selected ones of the output ports at the same time; and

multiple virtual output buffers associated with each one of the input ports, each one of the virtual output buffers dedicated to a different one of the output ports, the scheduler conducting multiple independent output port arbitrations for each one of the multiple output ports and the associated dedicated virtual output buffers and configuring the cross-switch according to the multiple independent arbitrations, the scheduler conducting a second separate set of independent arbitrations between any of the output ports that did not receive connection acceptance from an input port arbitration winner and the virtual output buffers associated with the input ports that did not win one of the previous independent output port arbitrations.

29. A packet scheduler comprising:

input circuitry to receive queue status for a plurality of input ports, the queue status for each input port identifying, for a plurality of output ports, the output ports for which that input port has data queued for transmission; and

an arbitration circuit to conduct a multi-phase arbitration to produce a matching between the input ports and the output ports, each arbitration phase comprising conducting an independent output port arbitration for each unmatched output port to select an unmatched input port having data queued for that output port, and conducting an independent input port arbitration for each input port selected by at least one output port arbitration to select an input port-output port matching, the arbitration circuit terminating the multi-phase arbitration when no further input port-output port matches are possible.

Assignments (15)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
MERGER Recorded Mar 4, 2021
From: FORCE10 NETWORKS, INC.
To: DELL MARKETING CORPORATION
Reel/Frame 056104/0988 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
RELEASE OF SECURITY INTEREST Recorded Sep 14, 2016
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: DELL SOFTWARE INC.; FORCE10 NETWORKS, INC.; DELL MARKETING L.P.; ASAP SOFTWARE EXPRESS, INC.; APPASSURE SOFTWARE, INC.; COMPELLENT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL INC.; DELL PRODUCTS L.P.; DELL USA L.P.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040040/0001 →
RELEASE OF SECURITY INTEREST Recorded Sep 14, 2016
From: BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
To: DELL MARKETING L.P.; ASAP SOFTWARE EXPRESS, INC.; APPASSURE SOFTWARE, INC.; COMPELLENT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL INC.; DELL PRODUCTS L.P.; DELL USA L.P.; DELL SOFTWARE INC.; FORCE10 NETWORKS, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040065/0618 →
RELEASE OF SECURITY INTEREST Recorded Sep 13, 2016
From: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
To: DELL MARKETING L.P.; ASAP SOFTWARE EXPRESS, INC.; APPASSURE SOFTWARE, INC.; COMPELLANT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL INC.; DELL PRODUCTS L.P.; DELL USA L.P.; DELL SOFTWARE INC.; FORCE10 NETWORKS, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040065/0216 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Jan 2, 2014
From: APPASSURE SOFTWARE, INC.; ASAP SOFTWARE EXPRESS, INC.; BOOMI, INC.; COMPELLENT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL INC.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL USA L.P.; FORCE10 NETWORKS, INC.; GALE TECHNOLOGIES, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
To: BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS FIRST LIEN COLLATERAL AGENT
Reel/Frame 031897/0348 →
PATENT SECURITY AGREEMENT (ABL) Recorded Jan 2, 2014
From: DELL INC.; APPASSURE SOFTWARE, INC.; ASAP SOFTWARE EXPRESS, INC.; BOOMI, INC.; COMPELLENT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL USA L.P.; FORCE10 NETWORKS, INC.; GALE TECHNOLOGIES, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 031898/0001 →
PATENT SECURITY AGREEMENT (TERM LOAN) Recorded Jan 2, 2014
From: DELL INC.; APPASSURE SOFTWARE, INC.; ASAP SOFTWARE EXPRESS, INC.; BOOMI, INC.; COMPELLENT TECHNOLOGIES, INC.; CREDANT TECHNOLOGIES, INC.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL USA L.P.; FORCE10 NETWORKS, INC.; GALE TECHNOLOGIES, INC.; PEROT SYSTEMS CORPORATION; SECUREWORKS, INC.; WYSE TECHNOLOGY L.L.C.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 031899/0261 →