Application execution path tracing for inline performance analysis
Techniques are provided for application tracing for inline performance analysis. One method comprises obtaining trace events generated by instructions executed in response to trace points in instrumented software; updating, for each trace event, a buffer entry of a sampling buffer that corresponds to a particular processing core and a time window, wherein the buffer entry is identified based on (a) a flow type identifier associated with the instructions, (b) an identifier of a respective trace event, and (c) an identifier of an adjacent trace event to the respective trace event, and wherein the updating comprises updating, for the time window: (i) a first counter indicating a cumulative number of events for the respective and adjacent trace events, and (ii) a second counter indicating a cumulative amount of time between the respective and adjacent trace events; and determining one or more performance metrics associated with the respective and adjacent trace events in the time window using the first and second counters.
1. A method, comprising:
obtaining a plurality of trace events generated by one or more instructions executed in response to trace points placed in instrumented software;
updating, for each trace event, a buffer entry of a particular sampling buffer of a plurality of sampling buffers, wherein the particular sampling buffer corresponds to a particular processing core of a plurality of processing cores and a time window, wherein the particular processing core executes the one or more instructions that generated a respective trace event, wherein the buffer entry of the particular sampling buffer is identified based at least in part on (a) a flow type identifier associated with the one or more instructions, (b) an identifier of the respective trace event, and (c) an identifier of an adjacent trace event to the respective trace event, wherein the adjacent trace event corresponds to an adjacent trace point processed by a given execution instance, and wherein the updating comprises (i) updating a first counter indicating a cumulative number of events for the respective trace event and the adjacent trace event for the time window, and (ii) updating a second counter indicating a cumulative amount of time between the respective trace event and the adjacent trace event in the time window; and
determining one or more performance metrics associated with the respective trace event and the adjacent trace event in the time window using the first counter and the second counter,
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
2. The method of claim 1 , further comprising determining one or more of an average and a variance of the time between the respective trace event and the adjacent trace event in the time window using the first counter and the second counter.
3. The method of claim 1 , further comprising determining one or more of a minimum and a maximum of the amount of time between the respective trace event and the adjacent trace event in the time window for a given flow type.
4. The method of claim 1 , further comprising destaging a content of the buffer entries of the plurality of sampling buffers using at least one destage thread.
5. The method of claim 1 , wherein the flow type identifier is identified in the instrumented software.
6. The method of claim 5 , wherein the flow type identifier identifies a flow type comprising one or more of a read synchronous flow, a write synchronous flow, a disk defragmentation flow, a disk rebuild flow, a synchronous execution flow and a background execution flow.
7. The method of claim 1 , wherein the particular sampling buffer is identified by applying a hash operation to (a) the flow type identifier associated with the one or more instructions, (b) the identifier of the respective trace event, and (c) the identifier of the adjacent trace event to the respective trace event.
8. An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured to implement the following steps:
obtaining a plurality of trace events generated by one or more instructions executed in response to trace points placed in instrumented software;
updating, for each trace event, a buffer entry of a particular sampling buffer of a plurality of sampling buffers, wherein the particular sampling buffer corresponds to a particular processing core of a plurality of processing cores and a time window, wherein the particular processing core executes the one or more instructions that generated a respective trace event, wherein the buffer entry of the particular sampling buffer is identified based at least in part on (a) a flow type identifier associated with the one or more instructions, (b) an identifier of the respective trace event, and (c) an identifier of an adjacent trace event to the respective trace event, wherein the adjacent trace event corresponds to an adjacent trace point processed by a given execution instance, and wherein the updating comprises (i) updating a first counter indicating a cumulative number of events for the respective trace event and the adjacent trace event for the time window, and (ii) updating a second counter indicating a cumulative amount of time between the respective trace event and the adjacent trace event in the time window; and
determining one or more performance metrics associated with the respective trace event and the adjacent trace event in the time window using the first counter and the second counter.
9. The apparatus of claim 8 , further comprising determining one or more of an average and a variance of the time between the respective trace event and the adjacent trace event in the time window using the first counter and the second counter.
10. The apparatus of claim 8 , further comprising determining one or more of a minimum and a maximum of the amount of time between the respective trace event and the adjacent trace event in the time window for a given flow type.
11. The apparatus of claim 8 , further comprising destaging a content of the buffer entries of the plurality of sampling buffers using at least one destage thread.
12. The apparatus of claim 8 , wherein the flow type identifier is identified in the instrumented software.
13. The apparatus of claim 12 , wherein the flow type identifier identifies a flow type comprising one or more of a read synchronous flow, a write synchronous flow, a disk defragmentation flow, a disk rebuild flow, a synchronous execution flow and a background execution flow.
14. The apparatus of claim 8 , wherein the particular sampling buffer is identified by applying a hash operation to (a) the flow type identifier associated with the one or more instructions, (b) the identifier of the respective trace event, and (c) the identifier of the adjacent trace event to the respective trace event.
15. A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform the following steps:
obtaining a plurality of trace events generated by one or more instructions executed in response to trace points placed in instrumented software;
updating, for each trace event, a buffer entry of a particular sampling buffer of a plurality of sampling buffers, wherein the particular sampling buffer corresponds to a particular processing core of a plurality of processing cores and a time window, wherein the particular processing core executes the one or more instructions that generated a respective trace event, wherein the buffer entry of the particular sampling buffer is identified based at least in part on (a) a flow type identifier associated with the one or more instructions, (b) an identifier of the respective trace event, and (c) an identifier of an adjacent trace event to the respective trace event, wherein the adjacent trace event corresponds to an adjacent trace point processed by a given execution instance, and wherein the updating comprises (i) updating a first counter indicating a cumulative number of events for the respective trace event and the adjacent trace event for the time window, and (ii) updating a second counter indicating a cumulative amount of time between the respective trace event and the adjacent trace event in the time window; and
determining one or more performance metrics associated with the respective trace event and the adjacent trace event in the time window using the first counter and the second counter.
16. The non-transitory processor-readable storage medium of claim 15 , further comprising determining one or more of an average and a variance of the time between the respective trace event and the adjacent trace event in the time window using the first counter and the second counter.
17. The non-transitory processor-readable storage medium of claim 15 , further comprising determining one or more of a minimum and a maximum of the amount of time between the respective trace event and the adjacent trace event in the time window for a given flow type.
18. The non-transitory processor-readable storage medium of claim 15 , further comprising destaging a content of the buffer entries of the plurality of sampling buffers using at least one destage thread.
19. The non-transitory processor-readable storage medium of claim 15 , wherein the flow type identifier is identified in the instrumented software and identifies a flow type comprising one or more of a read synchronous flow, a write synchronous flow, a disk defragmentation flow, a disk rebuild flow, a synchronous execution flow and a background execution flow.
20. The non-transitory processor-readable storage medium of claim 15 , wherein the particular sampling buffer is identified by applying a hash operation to (a) the flow type identifier associated with the one or more instructions, (b) the identifier of the respective trace event, and (c) the identifier of the adjacent trace event to the respective trace event.