System and method for providing end-to-end observability for distributed event-driven applications
A method and system for providing end-to-end observability for distributed event-driven applications are disclosed. The method includes generating a trace for an executed transaction and decomposing the generated trace into multiple synthetic traces or sub-traces. Each of the synthetic traces or sub-traces represents an operation performed for a discrete application function contained within that transaction. For each sub-trace a synthetic root span is generated, the root span representing an end-to-end time to process the respective application function.
1. A method for providing end-to-end observability for distributed event-driven applications, the method comprising:
executing, by a processor, a transaction including a plurality of operations and generating a trace for the executed transaction;
receiving, by a plurality of local collectors, a plurality of spans generated from the plurality of operations executed in differing host servers;
filtering, by at least one local collector among the plurality of local collectors, asynchronous spans among the plurality of spans, such that only spans representing inter-process communications are forwarded to a global collector cluster including a plurality of global collectors;
identifying, by a load balancer, a target global collector among the plurality of global collectors for routing the spans associated with the generated trace;
transmitting, by the load balancer to the target global collector, the spans associated with the generated trace;
waiting, by the target global collector, until all of the asynchronous spans associated with a discrete path through the generated trace are received;
decomposing, by the target global collector, the trace into a plurality of sub-traces; and
deriving, by the target global collector, end-to-end metrics for each of the sub-traces.
2. The method according to claim 1 , further comprising:
re-parenting the internal spans, such that the internal spans may be related to one another in a parent-child relationship.
3. The method according to claim 1 , wherein the global collector cluster is a global cluster mode.
4. The method according to claim 1 , wherein the global collector cluster is regional cluster mode.
5. The method according to claim 1 , wherein the global collector cluster is a line of business cluster mode.
6. The method according to claim 1 , wherein at least one of the plurality of local collectors and at least one of the plurality of global collectors reside on a same network.
7. The method according to claim 1 , wherein at least one of the plurality of local collectors and at least one of the plurality of global collectors reside on different networks.
8. The method according to claim 1 , further comprising:
sampling, by the target global collector, one or more exemplary sub-traces.
9. The method according to claim 1 , further comprising:
sampling, by the target global collector, the generated trace.
10. The method according to claim 1 , wherein at least one of the sub-trace corresponds to an operation performed for a discrete application function contained within the transaction.
11. The method according to claim 1 , wherein at least one of the plurality of operations includes one or more sub-operations.
12. The method according to claim 11 , wherein a span is generated for each of the plurality of operations, and a sub-span is generated for each of the one or more sub-operations.
13. The method according to claim 1 , wherein the trace is defined by a tree of the plurality of spans.
14. The method according to claim 1 , wherein each of the sub-traces corresponds to an operation that is unrelated from one another.
15. The method according to claim 1 , wherein each of the sub-traces include different spans from one another.
16. The method according to claim 1 , wherein at least one of the plurality of local collectors reside in one of the host servers.
17. The method according to claim 1 , wherein each of the sub-traces resembles a synchronous trace.
18. The method according to claim 1 , wherein a span identifies a specific operation being executed, and a start time and an end time of the specific operation executed.
19. A system to provide for providing end-to-end observability for distributed event-driven applications, the system comprising:
a memory;
a display;
a processor;
a plurality of local collectors;
a plurality of global collectors; and
a load balancer,
wherein the system is configured to perform:
executing, by the processor, a transaction including a plurality of operations and generating a trace for the executed transaction;
receiving, by the plurality of local collectors, a plurality of spans generated from the plurality of operations executed in differing host servers;
filtering, by at least one local collector among the plurality of local collectors, asynchronous spans among the plurality of spans, such that only spans representing inter-process communications are forwarded to a global collector cluster including the plurality of global collectors;
identifying, by the load balancer, a target global collector among the plurality of global collectors for routing the spans associated with the generated trace;
transmitting, by the load balancer to the target global collector, the spans associated with the generated trace;
waiting, by the target global collector, until all of the asynchronous spans associated with a discrete path through the generated trace are received;
decomposing, by the target global collector, the trace into a plurality of sub-traces; and
deriving, by the target global collector, end-to-end metrics for each of the sub-traces.
20. A non-transitory computer readable storage medium that stores a computer program for providing end-to-end observability for distributed event-driven applications, the computer program, when executed by a processor, causing a system to perform a plurality of processes comprising:
executing a transaction including a plurality of operations and generating a trace for the executed transaction;
receiving, by a plurality of local collectors, a plurality of spans generated from the plurality of operations executed in differing host servers;
filtering, by at least one local collector among the plurality of local collectors, asynchronous spans among the plurality of spans, such that only spans representing inter-process communications are forwarded to a global collector cluster including a plurality of global collectors;
identifying, by a load balancer, a target global collector among the plurality of global collectors for routing the spans associated with the generated trace;
transmitting, by the load balancer to the target global collector, the spans associated with the generated trace;
waiting, by the target global collector, until all of the asynchronous spans associated with a discrete path through the generated trace are received;
decomposing, by the target global collector, the trace into a plurality of sub-traces; and
deriving, by the target global collector, end-to-end metrics for each of the sub-traces.