Grouping log pieces of different context within a log file in virtualization system
A method for log analysis includes receiving log outputs from an application and generating a log file by recording the log outputs in the log file and, for each log output in the log file, attaching a log context record to the log output. The log context record is encoded with a call stack having stack frames and one or more variables in the stack frames. The method further includes grouping the log outputs in the log file based on their log context records.
1. A method for log analysis in a computing system, comprising:
receiving log outputs from an application;
generating a log file, comprising:
recording the log outputs in the log file; and
for each log output in the log file, attaching a log context record to the log output, wherein the log context record is encoded with a call stack comprising stack frames and one or more variables in the stack frames; and
grouping the log outputs in the log file based on the log context record attached to each log output.
2. The method of claim 1 , further comprising retrieving the call stack from a hypervisor.
3. The method of claim 1 , further comprising retrieving the call stack from a guest operating system in a virtual machine.
4. The method of claim 1 , wherein grouping the log outputs in the log file comprises:
determining a distance between every pair of log context records of the log outputs in the log file; and
grouping the log outputs based on distances of their log context records.
5. The method of claim 4 , wherein grouping the log outputs based on the distances of their log context record comprises using machine learning clustering.
6. The method of claim 4 , wherein determining the distance between a pair of log context records comprises:
for corresponding pairs of stack frames from the pair of log context records, calculating numbers of different variables in the stack frames; and
calculating a weighted sum of the numbers of different variables as the distance.
7. The method of claim 6 , wherein determining the distance between a pair of log context records further comprises:
filling any call stack with less than a greatest number of stack frames among the call stacks with a number of empty frames so all the call stacks have a same number of stack frames.
8. The method of claim 1 , further comprising determining a common thread, session, or transaction for each group of log outputs.
9. The method of claim 8 , further comprising determining a cause of an error in the computing system based on the grouping of the log outputs and correcting the error in the computing system.
10. A non-transitory, computer-readable storage medium encoded with instructions executable by a processor to implement the method of claim 1 .
11. A computer system, comprising:
a processor; and
a non-transitory, computer-readable medium having stored thereon instructions that, in response to execution by the processor, cause the processor to:
receive log outputs from an application;
generate a log file by:
recording the log outputs in the log file; and
for each log output in the log file, attaching a log context record to the log output, wherein the log context record is encoded with a call stack comprising stack frames and one or more variables in the stack frames; and
group the log outputs in the log file based on the log context record attached to each log output.
12. The computer system of claim 11 , wherein:
the application runs on a virtual machine implemented by a hypervisor; and
the non-transitory, computer-readable medium having stored thereon additional instructions that, in response to execution by the processor, cause the processor to retrieve the call stack through the hypervisor.
13. The computer system of claim 11 , wherein:
the application runs on a virtual machine; and
the non-transitory, computer-readable medium having stored thereon additional instructions that, in response to execution by the processor, cause the processor to access the call stack through a guest operating system on the virtual machine.
14. The computer system of claim 11 , wherein group the log outputs in the log file comprises:
determining a distance between every pair of log context records of the log outputs in the log file; and
grouping the log outputs based on distances of their log context records.
15. The computer system of claim 14 , wherein grouping the log outputs based on the distances of their log context record comprises using machine learning clustering.
16. The computer system of claim 14 , wherein determining the distance between a pair of log context records comprises:
for corresponding pairs of stack frames from the pair of log context records, calculating numbers of different variables in the same stack frame; and
calculating a weighted sum of the numbers of different variables as the distance.
17. The computer system of claim 16 , wherein determining the distance between a pair of log context records further comprises:
filling any call stack with less than a greatest number of stack frames in all call stacks with a number of empty frames so all the call stacks have a same number of stack frames.
18. The computer system of claim 11 , wherein the non-transitory, computer-readable medium having stored thereon additional instructions that, in response to execution by the processor, cause the processor to determine a common thread, session, or transaction for each group of log outputs.
19. The computer system of claim 18 , further comprising determining a cause of an error in the computing system based on the grouping of the log outputs and correcting the error in the computing system.