Interleaved weighted fair queuing mechanism and system
View Patent ↗Methods and apparatus for interleaved weighted fair data packet queue sequencing are disclosed. An interleaving table specifies a queue sequence. A queue sequencer follows the table order on an epoch-by-epoch basis, selecting a queue for each epoch based on the table order. If the selected queue does not have enough data to fill its epoch, the sequencer can step to the next queue in the table order. Because the table is interleaved, higher-priority queues can be visited frequently, improving jitter and latency for packets associated with these queues. The table structure allows all queues at least some portion of the available output bandwidth, and can be organized to afford some queues a much larger portion without having those queues monopolize the output stream for inordinate amounts of time. In some embodiments, each table entry has a programmable epoch value associated with it. The epoch value can be used to weight each table entry respective to the other entries.
1. A method of scheduling packets from multiple queues onto a common output packet stream, wherein the common output packet stream serves multiples sets of queues, the method comprising:
maintaining a programmable interleaving table having multiple entries, each entry identifying one of the queues with an epoch value, each set of queues having its own programmable interleaving table;
maintaining a separate pointer for each set of queues to a current table entry in the programmable interleaving table for that set of queues;
selecting, for each epoch in the sequence of epochs, one of the sets of queues for packet scheduling during that epoch;
selecting the pointer corresponding to the selected set of queues for use during that epoch;
scheduling packets onto the output packet stream from the queue identified with the current table entry of the selected pointer, and charging that entry for use of the epoch; and
moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry.
2. The method of claim 1 , wherein the number of entries in the programmable interleaving table exceeds the number of queues, such that at least some queues are identified more than once in the table, each entry containing a field explicitly identifying the queue associated with that entry.
3. The method of claim 2 , wherein the epoch value for each entry is a common value shared by all entries.
4. The method of claim 2 , wherein the epoch value for each entry is based on the queue identified with the entry.
5. The method of claim 2 , wherein the epoch value for each entry is a programmable value occupying a field of that entry.
6. The method of claim 2 , wherein each entry comprises an epoch counter field, and wherein charging an entry for use of an epoch comprises changing the value of the epoch counter field by one.
7. A method of scheduling packets from multiple queues onto a common output packet stream, the method comprising:
maintaining a programmable interleaving table having multiple entries, each entry identifying one of the queues with an epoch value, each entry comprising an epoch counter field, wherein the number of entries in the programmable interleaving table exceeds the number of queues, such that at least some queues are identified more than once in the table, each entry containing a field explicitly identifying the queue associated with that entry;
maintaining a pointer to a current table entry in the programmable interleaving table;
for each epoch in a sequence of epochs, scheduling packets onto the output packet stream from the queue identified with the current table entry, and charging that entry for use of the epoch by changing the value of the epoch counter field by one; and
moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry, wherein moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry comprises moving the pointer after the value in the epoch counter has been changed a number of times corresponding to the current table entry's epoch value.
8. A method of scheduling packets from multiple queues onto a common output packet stream, the method comprising:
maintaining a programmable interleaving table having multiple entries, each entry identifying one of the queues with an epoch value, each entry comprising an epoch counter field, wherein the number of entries in the programmable interleaving table exceeds the number of queues, such that at least some queues are identified more than once in the table, each entry containing a field explicitly identifying the queue associated with that entry;
maintaining a pointer to a current table entry in the programmable interleaving table;
for each epoch in a sequence of epochs, scheduling packets onto the output packet stream from the queue identified with the current table entry, and charging that entry for use of the epoch by changing the value of the epoch counter field by one; and
moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry, wherein moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry comprises moving the pointer after each epoch to the next table entry that has not yet changed the value of its epoch counter field a number of times corresponding to that entry's epoch value.
9. The method of claim 8 , wherein when all table entries have been charged with use of a number of epochs corresponding to their respective epoch values, re-initializing each epoch counter field and setting the pointer to point to the head of the table.
10. The method of claim 2 , wherein when the pointer is moved to a next table entry an epoch register is initialized, wherein charging an entry for use of an epoch comprises changing the value of the epoch register by one, and wherein moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry comprises moving the pointer after the value in the epoch register has been changed a number of times corresponding to the current table entry's epoch value.
11. The method of claim 2 , wherein when the queue identified with the current table entry is empty and the epoch has not yet ended, moving the pointer to the next table entry that is not empty, and scheduling packets onto the output packet stream from the queue identified with that next table entry for the remainder of the epoch.
12. The method of claim 11 , wherein when the pointer is moved during an epoch, only the first table entry of the epoch is charged for use of the epoch.
13. A method of scheduling packets from multiple queues onto a common output packet stream, the method comprising:
maintaining a programmable interleaving table having multiple entries, each entry identifying one of the queues with an epoch value, wherein the number of entries in the programmable interleaving table equals the number of queues, wherein each entry comprises an epoch value field and an epoch counter field;
maintaining a pointer to a current table entry in the programmable interleaving table;
for each epoch in a sequence of epochs scheduling packets onto the output packet stream from the queue identified with the current table entry and charging that entry for use of the epoch, wherein charging an entry for use of an epoch comprises changing the value of that field's epoch counter by one; and
moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry, wherein moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry comprises moving the pointer, after each epoch, to the next table entry that has not yet changed the value of its epoch counter field a number of times corresponding to that entry's epoch value.
14. An integrated circuit having a scheduler comprising:
a programmable interleaving table having multiple entries, each entry identifying a queue with an epoch value;
a pointer register identifying a current entry in the interleaving table; and
a queue sequencer to supply, for each epoch, a queue identifier based on the current entry in the programmable interleaving table, to charge the current entry for use of the epoch, and to step the pointer register to a next table entry when the current entry has been charged with use of a number of epochs set aside for that entry wherein the queue sequencer has access to a queue status register that indicates whether each queue identified in the programmable interleaving table is empty or not, the queue sequencer using the queue status register contents to step the pointer register to a next table entry within an epoch, when the queue status of the queue identified with the current entry indicates that that queue is empty.
15. The circuit of claim 14 , wherein the pointer register comprises multiple pointer entries, each pointer entry identifying a current entry, in the programmable interleaving table, corresponding to that pointer entry.
16. The circuit of claim 15 , wherein the programmable interleaving table comprises multiple interleaving table segments, each pointer entry identifying table entries within a corresponding one of the interleaving table segments.
17. An apparatus comprising a computer-readable medium containing computer instructions that, when executed, cause a processor or multiple communicating processors to perform a method for scheduling packets from multiple queues onto a common output packet stream, wherein the common output packet stream serves multiples sets of queues the method comprising:
maintaining a programmable interleaving table having multiple entries, each entry identifying one of the queues with an epoch value, each set of queues having its own programmable interleaving table;
maintaining a separate pointer for each set of queues to a current table entry in the programmable interleaving table for that set of queues;
selecting, for each epoch in the sequence of epochs, one of the sets of queues for packet scheduling during that epoch;
selecting the pointer corresponding to the selected set of queues for use during that epoch;
scheduling packets onto the output packet stream from the queue identified with the current table entry of the selected pointer, and charging that entry for use of the epoch; and
moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry.
18. The apparatus of claim 17 , wherein the number of entries in the programmable interleaving table exceeds the number of queues, such that at least some queues are identified more than once in the table, each entry containing a field explicitly identifying the queue associated with that entry.
19. The apparatus of claim 18 , wherein the epoch value for each entry is a programmable value occupying a field of that entry.
20. The apparatus of claim 19 , wherein each entry comprises an epoch counter field, and wherein charging an entry for use of an epoch comprises changing the value of the epoch counter field by one.
21. An apparatus comprising a computer-readable medium containing computer instructions that, when executed, cause a processor or multiple communicating processors to perform a method for scheduling packets from multiple queues onto a common output packet stream, the method comprising:
maintaining a programmable interleaving table having multiple entries, each entry identifying one of the queues with an epoch value, each entry comprising an epoch counter field, wherein the number of entries in the programmable interleaving table exceeds the number of queues, such that at least some queues are identified more than once in the table, each entry containing a field explicitly identifying the queue associated with that entry;
maintaining a pointer to a current table entry in the programmable interleaving table;
for each epoch in a sequence of epochs, scheduling packets onto the output packet stream from the queue identified with the current table entry, and charging that entry for use of the epoch by changing the value of the epoch counter field by one; and
moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry, wherein moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry comprises moving the pointer after the value in the epoch counter has been changed a number of times corresponding to the current table entry's epoch value.
22. An apparatus comprising a computer-readable medium containing computer instructions that, when executed, cause a processor or multiple communicating processors to perform a method for scheduling packets from multiple queues onto a common output packet stream, the method comprising:
maintaining a programmable interleaving table having multiple entries, each entry identifying one of the queues with an epoch value, each entry comprising an epoch counter field, wherein the number of entries in the programmable interleaving table exceeds the number of queues, such that at least some queues are identified more than once in the table, each entry containing a field explicitly identifying the queue associated with that entry;
maintaining a pointer to a current table entry in the programmable interleaving table;
for each epoch in a sequence of epochs, scheduling packets onto the output packet stream from the queue identified with the current table entry and charging that entry for use of the epoch by changing the value of the epoch counter field by one; and
moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry, wherein moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry comprises moving the pointer after each epoch to the next table entry that has not yet changed the value of its epoch counter field a number of times corresponding to that entry's epoch value.
23. The apparatus of claim 22 , wherein when all table entries have been charged with use of a number of epochs corresponding to their respective epoch values, re-initializing each epoch counter field and setting the pointer to point to the head of the table.
24. The apparatus of claim 18 , wherein when the queue identified with the current table entry is empty and the epoch has not yet ended, moving the pointer to the next table entry that is not empty, and scheduling packets onto the output packet stream from the queue identified with that next table entry for the remainder of the epoch.
25. The apparatus of claim 17 , wherein the number of entries in the programmable interleaving table equals the number of queues, wherein each entry comprises an epoch value field and an epoch counter field, wherein charging an entry for use of an epoch comprises changing the value of that field's epoch counter by one, and wherein moving the pointer to a next table entry when the current table entry has been charged with use of a number of epochs set aside for that entry comprises moving the pointer, after each epoch, to the next table entry that has not yet changed the value of its epoch counter field a number of times corresponding to that entry's epoch value.