Monitoring resource congestion in a network processor
View Patent ↗Embodiments of the invention are directed to monitoring resources of a network processor to detect a condition of exhaustion in one or more of the resources over a predetermined time interval and to provide an indication of the condition. Some embodiments periodically sample various resources of a network processor and from the samples calculate utilization of the network processor's memory bus and core processor, and determine if an interworking FIFO packet queue error has occurred. Such information may help network operators and/or support engineers to quickly zero in on the root cause and take corrective actions for network failures which previously could have been attributed to many different causes and that would have required significant time and effort to troubleshoot.
1. A method of monitoring resources of a network processor comprising the steps of:
retrieving a first set of utilization data of the resources;
determining from the first set of utilization data if an exhaustion condition of any of the resources exists during a sampling interval, wherein determining if an exhaustion condition exists comprises determining if an error has occurred on a first-in first-out (FIFO) queue of the network processor during the sampling interval; and
performing the following steps responsive to said exhaustion condition existing:
determining if the exhaustion condition persists for a predetermined first interval of time, and
setting an indication of the exhaustion condition responsive to the exhaustion condition persisting for the first interval.
2. The method of claim 1 further comprising:
determining, responsive to setting the indication, if no exhaustion condition exists on all of the resources for a second interval of time; and
clearing the indication of exhaustion responsive to no exhaustion condition existing for the second interval.
3. The method of claim 2 wherein determining if no exhaustion condition exists comprises:
retrieving a third set of utilization data of the resources over the second interval; and
determining from the third set of utilization data if no exhaustion condition exists on all of the resources for the second interval.
4. The method of claim 3 wherein determining from the third set of utilization data if no exhaustion condition exists on all of the resources comprises determining if an error has occurred on a first-in first-out (FIFO) queue of the network processor during the second interval.
5. The method of claim 1 wherein the step of determining if the exhaustion condition of any of the resources exists comprises:
calculating from the first set of utilization data a utilization of a given resource over the sampling interval;
comparing the utilization for the given resource to a respective threshold; and
determining an exhaustion condition exists on the given resource responsive to the utilization exceeding the respective threshold.
6. The method of claim 5 wherein calculating comprises calculating a utilization of a core processor over the sampling interval.
7. The method of claim 5 wherein calculating comprises calculating a utilization of a memory bus over the sampling interval.
8. The method of claim 1 wherein determining if the exhaustion condition persists comprises:
retrieving a second set of utilization data of the resources over the first interval; and
determining from the second set of utilization data if an exhaustion condition of any of the resources persists for the first interval.
9. The method of claim 8 wherein determining from the second set of utilization data if an exhaustion condition of any of the resources persists comprises determining if an error has occurred on a first-in first-out (FIFO) queue of the network processor during the first interval.
10. The method of claim 1 wherein setting the indication comprises recording in a log an entry corresponding to the indication.
11. The method of claim 1 , wherein the method is executed on a microprocessor configured with instructions which when executed cause the microprocessor to perform the method.
12. A device comprising:
a processor for executing program instructions;
a memory; and
an interface for retrieving utilization data concerning resources of a network processor,
wherein the memory includes program instructions for execution by the processor thereby making the processor operable to:
retrieve a first set of utilization data of the resources;
determine from the first set of utilization data if an exhaustion condition of any of the resources exists during a sampling interval, wherein determining if an exhaustion condition exists comprises determining whether an error has occurred on a first-in first-out (FIFO) queue of the network processor during the sampling interval; and
perform the following steps responsive to said exhaustion condition existing:
determine if the exhaustion condition persists for a predetermined first interval of time, and
set an indication of the exhaustion condition responsive to the exhaustion condition persisting for the first interval.
13. The device of claim 12 wherein the memory further includes program instructions for execution by the processor thereby making the processor operable to:
determine, responsive to setting the indication, if no exhaustion condition exists on all of the resources for a second interval of time; and
clear the indication of exhaustion responsive to no exhaustion condition existing for the second interval.
14. The device of claim 13 wherein the memory further includes program instructions for execution by the processor thereby making the processor, when determining if no exhaustion condition exists, operable to:
retrieve a third set of utilization data of the resources over the second interval; and
determine from the third set of utilization data if no exhaustion condition exists on all of the resources for the second interval.
15. The device of claim 14 wherein the memory further includes program instructions for execution by the processor thereby making the processor, when determining from the third set of utilization data if no exhaustion condition exists on all of the resources, operable to determine if an error has occurred on a first-in first-out (FIFO) queue of the network processor during the second interval.
16. The device of claim 12 wherein the memory further includes program instructions for execution by the processor thereby making the processor, when determining if the exhaustion condition of any of the resources exists, operable to:
calculate from the first set of utilization data a utilization of a given resource over the sampling interval;
compare the utilization for the given resource to a respective threshold; and
determine an exhaustion condition exists on the given resource responsive to the utilization exceeding the respective threshold.
17. The device of claim 12 wherein the memory further includes program instructions for execution by the processor thereby making the processor, when determining if the exhaustion condition persists, operable to:
retrieve a second set of utilization data of the resources over the first interval; and
determine from the second set of utilization data if an exhaustion condition of any of the resources persists for the first interval.
18. The device of claim 17 wherein the memory further includes program instructions for execution by the processor thereby making the processor, when determining from the second set of utilization data if an exhaustion condition of any of the resources persists, operable to determine if an error has occurred on a first-in first-out (FIFO) queue of the network processor during the first interval.