Adaptive event aggregation
An application network is monitored using a plurality of agents. Adaptive event aggregation is performed to determine retaining values for an aggregation dimension. A report of the application network is generated based on the aggregation dimension.
1. A method, comprising:
copying event records that describe API call events between applications in an application network into a learning buffer,
wherein the event records comprise keys and metrics, and
wherein the event records are aggregated in the learning buffer based on an aggregation dimension that indicates retaining values determined using adaptive event aggregation;
copying an additional event record into an overflow buffer when the learning buffer reaches a size limit; and
generating a visualization of the application network based on the learning buffer and the overflow buffer.
2. The method of claim 1 , further comprising:
receiving configurable parameters that specify the aggregation dimension and the size limit.
3. The method of claim 1 , wherein the aggregation dimension can be lossy, lossless, or collapsed.
4. The method of claim 1 , further comprising:
determining the retaining values for the aggregation dimension using an adaptive learning algorithm.
5. The method of claim 1 , the copying the additional event record into the overflow buffer further comprising:
when the additional event record comprises keys not indicated by the retaining values, updating the event record to include a special token prior to copying the additional event record into the overflow buffer.
6. The method of claim 5 , further comprising:
creating an overflow key using the updated event record; and
using the overflow key to determine if a corresponding aggregated event exists in the overflow buffer.
7. The method of claim 1 , wherein an event record in the event records comprises a timestamp, a source IP address, a destination IP address, a response code, a response duration, and a response size.
8. A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
copy event records that describe API call events between applications in an application network into a learning buffer,
wherein the event records comprise keys and metrics, and
wherein the event records are aggregated in the learning buffer based on an aggregation dimension that indicates retaining values determined using adaptive event aggregation;
copy an additional event record into an overflow buffer when the learning buffer reaches a size limit; and
generate a visualization of the application network based on the learning buffer and the overflow buffer.
9. The system of claim 8 , the at least one processor further configured to:
receive configurable parameters that specify the aggregation dimension and the size limit.
10. The system of claim 8 , wherein the aggregation dimension can be lossy, lossless, or collapsed.
11. The system of claim 8 , the at least one processor further configured to:
determine the retaining values for the aggregation dimension using an adaptive learning algorithm.
12. The system of claim 8 , wherein to copy the additional event record into the overflow buffer the at least one processor is further configured to:
update the event record to include a special token prior to copying the additional event record into the overflow buffer when the additional event record comprises keys not indicated by the retaining values.
13. The system of claim 12 , the at least one processor further configured to:
create an overflow key using the updated event record; and
use the overflow key to determine if a corresponding aggregated event exists in the overflow buffer.
14. The system of claim 8 , wherein an event record in the event records comprises a timestamp, a source IP address, a destination IP address, a response code, a response duration, and a response size.
15. A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:
copying event records that describe API call events between applications in an application network into a learning buffer,
wherein the event records comprise keys and metrics, and
wherein the event records are aggregated in the learning buffer based on an aggregation dimension that indicates retaining values determined using adaptive event aggregation;
copying an additional event record into an overflow buffer when the learning buffer reaches a size limit; and
generating a visualization of the application network based on the learning buffer and the overflow buffer.
16. The non-transitory computer-readable device of claim 15 , the operations further comprising: receiving configurable parameters that specify the aggregation dimension and the size limit.
17. The non-transitory computer-readable device of claim 15 , wherein the aggregation dimension can be lossy, lossless, or collapsed.
18. The non-transitory computer-readable device of claim 15 , the operations further comprising:
determining the retaining values for the aggregation dimension using an adaptive learning algorithm.
19. The non-transitory computer-readable device of claim 15 , the operations further comprising:
when the additional event record comprises keys not indicated by the retaining values, updating the event record to include a special token prior to copying the additional event record into the overflow buffer.
20. The non-transitory computer-readable device of claim 19 , the operations further comprising:
creating an overflow key using the updated event record; and
using the overflow key to determine if a corresponding aggregated event exists in the overflow buffer.