Reconstructing database sessions from a query log
Some embodiments provide a method for analyzing database queries performed on a database. The method receives a log that includes a set of database queries that were performed on the database. The method identifies, from the log, two or more subsets of queries that are each associated with a different connection session between the database and a set of client applications, where each subset is associated with a set of temporary session objects that are not associated with queries in the other subsets of queries. The method performs a separate query interpretation process on each subset of queries to quantify the impact of performing the queries on the database during the connection sessions, where the query interpretation processes are performed separately in order to avoid errors associated with the temporary objects.
1. A method for analyzing database queries performed on a database, the method comprising:
receiving a log comprising a set of database queries that were performed on the database;
identifying, from the log, two or more subsets of queries that are each associated with a different connection session between the database and a set of client applications, wherein each subset is associated with a set of temporary session objects that are not associated with queries in the other subsets of queries; and
performing a separate query interpretation process on each subset of queries to quantify an impact of performing the queries on the database during the connection sessions.
2. The method of claim 1 , wherein at least two separate query interpretation processes are performed on two subsets of queries simultaneously, the method further comprising:
calculating the number of queries in each subset; and
assigning query subsets to query interpretation processes based on a decreasing number of calculated queries in each subset.
3. The method of claim 1 , wherein
the log further comprises a session identifier associated with each database query;
each connection session is associated with a unique session identifier; and
identifying the subsets of queries comprises using the session identifiers to identify, for each query in the second set of queries, the session in which the query was performed.
4. The method of claim 1 , wherein the temporary session objects comprise temporary tables.
5. The method of claim 1 , wherein:
the log further comprises a start time associated with each database query;
said start time indicates an initial time that the query was performed on the database; and
the method further comprises processing each query in a particular subset of queries in order of increasing start time.
6. The method of claim 1 further comprising based on an analysis of the log, removing duplicate database queries from the set of queries.
7. The method of claim 6 , wherein removing duplicate database queries comprises storing in a metadata storage a set of metadata associated with each removed duplicate query, wherein the set of metadata comprises at least one of an application identifier, a session identifier, and a user identifier.
8. The method of claim 6 further comprising determining, for each duplicated database query, whether to remove the duplicate based on a set of criteria that comprises a rule specifying that a particular duplicate database query should not be removed when the particular database query comprises a Data Definition Language (DDL) statement.
9. The method of claim 1 further comprising identifying individual queries in the log file.
10. The method of claim 9 , wherein identifying the individual queries further comprises combining at least two of the queries in the log file into a single query.
11. The method of claim 1 , wherein quantifying the impact on the database comprises computing a complexity indicator representing a complexity expression of each subset of queries.
12. A non-transitory machine readable medium storing a program which when executed by at least one processing unit analyzes database queries performed on a database, the program comprising sets of instructions for:
receiving a log comprising a set of database queries that were performed on the database;
identifying, from the log, two or more subsets of queries that are each associated with a different connection session between the database and a set of client applications, wherein each subset is associated with a set of temporary session objects that are not associated with queries in the other subsets of queries; and
performing a separate query interpretation process on each subset of queries to quantify an impact of performing the queries on the database during the connection sessions.
13. The non-transitory machine readable medium of claim 12 , wherein at least two separate query interpretation processes are performed on two subsets of queries simultaneously, the program further comprising sets of instructions for:
calculating the number of queries in each subset; and
assigning query subsets to query interpretation processes based on a decreasing number of calculated queries in each subset.
14. The non-transitory machine readable medium of claim 12 , wherein:
the log further comprises a session identifier associated with each database query;
each connection session is associated with a unique session identifier; and
the set of instructions for identifying the subsets of queries comprises a set of instructions for using the session identifier to identify, for each query in the second set of queries, the session in which the query was performed.
15. The non-transitory machine readable medium of claim 12 , wherein the temporary session objects comprise temporary tables.
16. The non-transitory machine readable medium of claim 12 , wherein:
the log further comprises a start time associated with each database query;
said start time indicates an initial time that the query was performed on the database; and
the program further comprises a set of instructions for processing each query in a particular subset of queries in order of increasing start time.
17. The non-transitory machine readable medium of claim 12 , the program further comprising a set of instructions for, based on an analysis of the log, removing duplicate database queries from the set of queries.
18. The non-transitory machine readable medium of claim 17 , wherein the set of instructions for removing duplicate database queries comprises a set of instructions for storing in a metadata storage a set of metadata associated with each removed duplicate query, wherein the set of metadata comprises at least one of an application identifier, a session identifier, and a user identifier.
19. The non-transitory machine readable medium of claim 12 , wherein the program further comprises a set of instructions for identifying individual queries in the log file, said set of instructions for identifying individual queries comprising a set of instructions for combining at least two of the queries in the log file into a single query.
20. The non-transitory machine readable medium of claim 12 , wherein the set of instructions for quantifying the impact on the database comprises a set of instructions for computing a complexity indicator representing a complexity expression of each subset of queries.