N-Gram Analysis of Software Behavior in Production and Testing Environments
Execution sequence information may be analyzed and quantified using n-gram analysis of functions executed by an application. The sequences of functions may be represented by n-grams, and the frequency of the various n-grams may indicate the behavior of the application in production, which may be compared to a test suite whose coverage may be quantified using a similar n-gram analysis. A coverage factor may compare the observed behavior of the application in production to the test suite for the application. The n-grams may be further quantified or prioritized by resource utilization, and several visualizations may be generated from the data.
1 . A method performed on at least one computer processor, said method comprising:
receiving first tracer data observed from an application, said application comprising a plurality of functions;
identifying a plurality of sequences of said functions;
identifying a usage database comprising usage for each of said plurality of sequences from said first tracer data;
receiving second tracer data observed from said application;
identifying a subset of said plurality of sequences contained in said second tracer data;
comparing said subset of said plurality of sequences to said usage frequency database to determine a coverage factor.
2 . The method of claim 1 , said plurality of sequences being identified by a tri-gram of said functions.
3 . The method of claim 2 , said plurality of sequences being identified by a 4-gram of said functions.
4 . The method of claim 1 , said usage being determined by a count of executions.
5 . The method of claim 1 , said usage being determined by resource usage of said sequences.
6 . The method of claim 5 , said resource usage comprising at least one of a group composed of:
processor resources;
memory resources;
storage resources;
network resources;
peripheral resources;
input/output resources;
database resources;
local service resources; and
remote service resources.
7 . The method of claim 1 , said first tracer data having a first schema and said second tracer data having a second schema.
8 . The method of claim 7 , said first schema comprising more elements than said second schema.
9 . The method of claim 7 , said first schema comprising fewer elements than said second schema.
10 . The method of claim 7 , said first schema being the same as said second schema.
11 . The method of claim 1 further comprising:
presenting a graph comprising said usage from said usage database.
12 . The method of claim 11 further comprising:
identifying a union of said usage for each of said plurality of sequences with said subset of said plurality of sequences in said graph.
13 . The method of claim 11 , said graph being a histogram.
14 . The method of claim 1 , said first tracer data being gathered during production usage of said application.
15 . The method of claim 14 , said second tracer data being gathered during testing of said application.
16 . The method of claim 14 , said first tracer data being gathered during production usage of said application during a first usage period and said second tracer data being gathered during production usage of said application during a second usage period.
17 . The method of claim 16 , determining that said coverage factor is less than a predefined value and creating an alert indicating that usage patterns for said application has changed.
18 . A system comprising:
a processor;
an analysis engine operating on said processor, said analysis engine that:
receives first tracer data observed from an application, said application comprising a plurality of functions;
identifies a plurality of sequences of said functions;
identifies a usage database comprising usage for each of said plurality of sequences from said first tracer data;
receives second tracer data observed from said application;
identifies a subset of said plurality of sequences contained in said second tracer data;
compares said subset of said plurality of sequences to said usage frequency database to determine a coverage factor.
19 . The system of claim 18 , said plurality of sequences being identified by a tri-gram of said functions.
20 . The system of claim 19 , said plurality of sequences being identified by a 4-gram of said functions.
21 . The system of claim 19 , said usage being determined by a count of executions.
22 . The system of claim 19 , said usage being determined by resource usage of said sequences.
23 . The system of claim 22 , said resource usage comprising at least one of a group composed of:
processor resources;
memory resources;
storage resources;
network resources;
peripheral resources;
input/output resources;
database resources;
local service resources; and
remote service resources.
24 . The system of claim 18 , said first tracer data having a first schema and said second tracer data having a second schema.
25 . The system of claim 24 , said first schema comprising more elements than said second schema.
26 . The system of claim 30 , said first schema comprising fewer elements than said second schema.
27 . The system of claim 30 , said first schema being the same as said second schema.
28 . The system of claim 18 , said analyzer that further:
presents a graph comprising said usage from said usage database.
29 . The system of claim 28 , said analyzer that further:
identifies a union of said usage for each of said plurality of sequences with said subset of said plurality of sequences in said graph.
30 . The system of claim 28 , said graph being a histogram.
31 . The system of claim 18 , said first tracer data being gathered during production usage of said application.
32 . The system of claim 31 , said second tracer data being gathered during testing of said application.
33 . The system of claim 31 , said first tracer data being gathered during production usage of said application during a first usage period and said second tracer data being gathered during production usage of said application during a second usage period.
34 . The method of claim 33 , said analyzer that further:
determines that said coverage factor is less than a predefined value and creating an alert indicating that usage patterns for said application has changed.