Systems and methods for scalable N-core stats aggregation
The present invention is directed towards systems and methods for aggregating and providing statistics from cores of a multi-core system intermediary between one or more clients and servers. The system may maintain in shared memory a global device number for each core of the multi-core system. The system may provide a thread for each core of the multi-core system to gather data from the corresponding core. A first thread may generate aggregated statistics from a corresponding core by parsing the gathered data from the corresponding core. The first thread may transfer the generated statistics to a statistics log according to a schedule. The system may adaptively reschedule the transfer by monitoring the operation of each computing thread. Responsive to a request from a client, an agent of the client may obtain statistics from the statistics log.
1. A method comprising:
(a) maintaining, in shared memory of a multi-core system intermediary between one or more clients and servers, a global device number for each core of the multi-core system, each core comprising one or more packet engines processing network traffic between the one or more clients and servers;
(b) executing, by an aggregator of the multi-core system, a computing thread for each core of the multi-core system;
(c) collecting, by a first computing thread of the aggregator, first statistics of network traffic processed by one or more packet engines on a first core and collecting, by a second computing thread of the aggregator, second statistics of network traffic processed by one or more packet engines on a second core; and
(d) maintaining, by the multi-core system, a statistics log for two or more cores of the multi-core system, including at least the first core and the second core, transferring, by the first computing thread, the first statistics with a marker to the statistics log, the marker corresponding to a global device number of the first core.
2. The method of claim 1 , wherein (a) further comprises assigning, for each core of the multi-core system, a global device number to a virtual machine executing on the core.
3. The method of claim 1 , wherein (a) further comprises assigning, for each core of the multi-core system, a global device number to each virtual machine of the core, each virtual machine comprising one or more packet engines processing network traffic between the one or more clients and servers.
4. The method of claim 1 , wherein (c) further comprises consolidating, by the aggregator, at least a portion of statistics collected from two or more cores of the multi-core system into a buffer.
5. The method of claim 1 , wherein (d) further comprises writing the collected statistics to the statistics log according to a schedule of the aggregator.
6. The method of claim 5 , wherein (d) further comprises adaptively rescheduling, by the aggregator, the transfer by monitoring the operation of the first core.
7. The method of claim 1 , wherein (d) further comprises transferring the collected statistics unchanged to the statistics log.
8. The method of claim 1 , wherein (d) further comprises transferring, by the second computing thread, the second statistics with a second marker to the statistics log, the second marker corresponding to a global device number of the second core.
9. The method of claim 1 , further comprising sending, by the aggregator, a notification to a first client of the one or more clients responsive to the transfer.
10. The method of claim 1 , further comprising providing, by the multi-core system to the one or more clients, access to the statistics log.
11. A system comprising:
an intermediary device intermediary between one or more clients and servers;
shared memory between multiple cores of the intermediary device, the shared memory maintaining a global device number for each of the multiple cores, each core comprising one or more packet engines processing network traffic between the one or more clients and servers; and
an aggregator of the intermediary device, executing a computing thread for each of the multiple cores, comprising at least a first computing thread collecting first statistics of network traffic processed by one or more packet engines on a first core and a second computing thread collecting second statistics of network traffic processed by one or more packet engines on a second core, and transferring the first statistics with a marker to a statistics log of the multi-core intermediary device, the marker corresponding to a global device number of the first core, the statistics log maintained for two or more cores of the multi-core intermediary device, including at least the first core and the second core.
12. The system of claim 11 , wherein the aggregator assigns a global device number to a virtual machine executing on each core.
13. The system of claim 12 , wherein the aggregator assigns a global device number to each virtual machine of each core, each virtual machine comprising one or more packet engines processing network traffic between the one or more clients and servers.
14. The system of claim 11 , wherein the aggregator further consolidates at least a portion of statistics collected from two or more cores of the multi-core intermediary device into a buffer.
15. The system of claim 11 , wherein the first computing thread writes the collected statistics to the statistics log according to a schedule of the aggregator.
16. The system of claim 15 , wherein the aggregator adaptively reschedules the transfer by monitoring the operation of the first core.
17. The system of claim 11 , wherein the first computing thread transfers the collected statistics unchanged to the statistics log.
18. The system of claim 11 , wherein the aggregator transfers the second statistics with a second marker to the statistics log, the second marker corresponding to a global device number of the second core.
19. The system of claim 11 , wherein the aggregator sends a notification to a first client of the one or more clients responsive to the transfer.
20. The system of claim 11 , wherein the aggregator provides the one or more clients access to the statistics log.