Monitoring resource congestion in a network processor
View Patent ↗Embodiments of the invention are directed to monitoring resources of a network processor to detect a condition of exhaustion in one or more of the resources over a predetermined time interval and to provide an indication of the condition. Some embodiments periodically sample various resources of a network processor and from the samples calculate utilization of the network processor's memory bus and core processor, and determine if an interworking FIFO packet queue error has occurred. Such information may help network operators and/or support engineers to quickly zero in on the root cause and take corrective actions for network failures which previously could have been attributed to many different causes and that would have required significant time and effort to troubleshoot.
1. A method of monitoring resources of a network processor, the method comprising:
retrieving a first set of utilization data regarding network processor resources;
determining, from the first set of utilization data, that an exhaustion condition exists for the network processor resources during a sampling interval;
determining whether the exhaustion condition persists for a first interval of time; and
recording a log entry based on the exhaustion condition persisting for the first interval.
2. The method of claim 1 , further comprising:
entering an exhaustion state based on determining that the exhaustion condition exists;
wherein determining whether the exhaustion condition persists and recording a log entry are performed while occupying the exhaustion state.
3. The method of claim 2 , wherein the exhaustion state is one of a group of possible exhaustion states and the group of possible exhaustion states comprises:
a core processor exhaustion state,
a memory bus exhaustion state, and
a first-in first-out (FIFO) error state.
4. The method of claim 1 , further comprising:
providing a debug tool including a history of operating conditions, wherein the history includes a reference to the exhaustion condition persisting for the first interval.
5. The method of claim 4 , wherein the history of operating conditions is a history of worst case operating conditions.
6. The method of claim 1 , further comprising:
determining that no exhaustion condition has been detected during a second interval of time; and
recording a log entry based on no exhaustion condition being detected during the second interval.
7. The method of claim 6 , wherein the second interval of time is at least two orders of magnitude longer than the first interval of time.
8. A device for monitoring resources of a network processor, the device comprising:
a memory; and
a processor in communication with the memory, the processor being configured to:
retrieve a first set of utilization data regarding network processor resources;
determine, from the first set of utilization data, that an exhaustion condition exists for the network processor resources during a sampling interval;
determine whether the exhaustion condition persists for a first interval of time; and
record a log entry based on the exhaustion condition persisting for the first interval.
9. The device of claim 8 , wherein the processor is further configured to:
enter an exhaustion state based on determining that the exhaustion condition exists;
wherein the processor is configured to determine whether the exhaustion condition persists and record a log entry while occupying the exhaustion state.
10. The device of claim 9 , wherein the exhaustion state is one of a group of possible exhaustion states and the group of possible exhaustion states comprises:
a core processor exhaustion state,
a memory bus exhaustion state, and
a first-in first-out (FIFO) error state.
11. The device of claim 8 , wherein the processor is further configured to:
provide a debug tool including a history of operating conditions, wherein the history includes a reference to the exhaustion condition persisting for the first interval.
12. The device of claim 11 , wherein the history of operating conditions is a history of worst case operating conditions.
13. The device of claim 8 , wherein the processor is further configured to:
determine that no exhaustion condition has been detected during a second interval of time; and
record a log entry based on no exhaustion condition being detected during the second interval.
14. The device of claim 13 , wherein the second interval of time is at least two orders of magnitude longer than the first interval of time.
15. A non-transitory machine-readable medium encoded with instructions for monitoring resources of a network processor, the medium comprising:
instructions for retrieving a first set of utilization data regarding network processor resources;
instructions for determining, from the first set of utilization data, that an exhaustion condition exists for the network processor resources during a sampling interval;
instructions for determining whether the exhaustion condition persists for a first interval of time; and
instructions for recording a log entry based on the exhaustion condition persisting for the first interval.
16. The non-transitory machine-readable medium of claim 15 , further comprising:
instructions for entering an exhaustion state based on determining that the exhaustion condition exists;
wherein the instructions for determining whether the exhaustion condition persists and recording a log entry are configured to be executed while occupying the exhaustion state.
17. The non-transitory machine-readable medium of claim 16 , wherein the exhaustion state is one of a group of possible exhaustion states and the group of possible exhaustion states comprises:
a core processor exhaustion state,
a memory bus exhaustion state, and
a first-in first-out (FIFO) error state.
18. The non-transitory machine-readable medium of claim 15 , further comprising:
instructions for providing a debug tool including a history of operating conditions, wherein the history includes a reference to the exhaustion condition persisting for the first interval.
19. The non-transitory machine-readable medium of claim 18 , wherein the history of operating conditions is a history of worst case operating conditions.
20. The non-transitory machine-readable medium of claim 15 , further comprising:
instructions for determining that no exhaustion condition has been detected during a second interval of time; and
instructions for recording a log entry based on no exhaustion condition being detected during the second interval.