Unified statistics collection framework using a process-based top-down approach for Postgres-based database systems

1 . A computer-implemented method comprising:

receiving, by data processing hardware, a schema defining a relational database for storing a plurality of statistics corresponding to a query, the relational database comprising a plurality of data tables relationally connected according to the schema, each data table of the plurality of data tables corresponding to a respective statistic, wherein the plurality of statistics includes a first statistic having a first statistic type and a second statistic having a second statistic type that is distinct from the first statistic type, wherein the first and second statistic types are selected from the group consisting of: instance statistics, session statistics, transaction statistics, query statistics, plan statistics, and wait event statistics;

obtaining the query corresponding to data at a data store;

executing, using a query executor, the query on the data at the data store; and

during execution of the query on the data at the data store:

collecting, by the data processing hardware and as the query is executing, from a query execution database, the plurality of statistics indicative of performance characteristics specific to the query that is currently executing, each statistic of the plurality of statistics corresponding to a respective data table of the plurality of data tables of the relational database;

for each statistic of the plurality of statistics, storing, by the data processing hardware, the statistic at the respective data table according to the schema;

receiving an indication to cease collecting the first statistic type; and

responsive to receiving the indication, ceasing collection of the first statistic associated with the first statistic type while continuing to collect at least the second statistic associated with the second statistic type.

2 . The method of claim 1 , wherein:

the schema comprises a tree data structure comprising a plurality of nodes;

each data table of the plurality of data tables corresponds to a respective node of the plurality of nodes of the tree data structure; and

the plurality of statistics includes one or more of: session statistics, transaction statistics, query statistics, plan statistics, wait event statistics, buffer usage statistics, or write-ahead logging (WAL) usage statistics.

3 . The method of claim 1 , wherein, in response to receiving an indication to cease collecting the first statistic type, the method further comprises:

identifying a particular node of a tree data structure of the schema that corresponds to the first statistic; and

ceasing collection of statistics associated with each node in the tree data structure that is descendant from the particular node.

4 . The method of claim 1 , further comprising:

accessing the stored plurality of statistics, by the data processing hardware, from the respective data table according to the schema; and

executing a second query on the data store, the second query being based at least in part on the accessed plurality of statistics.

5 . The method of claim 1 , wherein the plurality of statistics include one or more of:

instance statistics;

session statistics;

auxiliary process statistics;

transaction statistics;

activity statistics;

wait event statistics;

query statistics; or

plan statistics.

6 . The method of claim 1 , further comprising:

normalizing the plurality of statistics to remove identifying information.

7 . The method of claim 6 , further comprising:

aggregating a plurality of normalized statistics corresponding to a plurality of queries to generate a plurality of generalized statistics corresponding to the plurality of queries.

8 . The method of claim 1 , further comprising:

determining whether a size of collected statistics satisfies a threshold size,

wherein receiving the indication to cease collecting the first statistic type is in response to determining that the size of the collected statistics satisfies the threshold size.

9 . The method of claim 1 , further comprising:

determining whether the duration of collected statistics satisfies a threshold size,

wherein receiving the indication to cease collecting the first statistic type is in response to determining that the duration of the collected statistics satisfies the threshold size.

10 . A system comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware, the memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to:

receive a schema defining a relational database for storing a plurality of statistics corresponding to a query, the relational database comprising a plurality of data tables relationally connected according to the schema, each data table of the plurality of data tables corresponding to a respective statistic, wherein the plurality of statistics includes a first statistic having a first statistic type and a second statistic having a second statistic type that is distinct from the first statistic type, wherein the first and second statistic types are selected from the group consisting of: instance statistics, session statistics, transaction statistics, query statistics, plan statistics, and wait event statistics;

obtain the query corresponding to data at a data store;

execute, using a query executor, the query on the data at the data store; and

during execution of the query on the data at the data store:

collect, as the query is executing, from a query execution database, the plurality of statistics indicative of performance characteristics specific to the query as the query executes, each statistic of the plurality of statistics corresponding to a respective data table of the plurality of data tables of the relational database; and

for each statistic of the plurality of statistics, storing the respective statistic at the respective data table according to the schema;

receive an indication to cease collecting first statistic type; and

in response to receiving the indication, cease collection of the first statistic associated with the first statistic type while continuing to collect at least the second statistic associated with the second statistic type.

11 . The system of claim 10 , wherein:

the schema comprises a tree data structure comprising a plurality of nodes;

each data table of the plurality of data tables corresponds to a respective node of the plurality of nodes of the tree data structure; and

the plurality of statistics includes one or more of: session statistics, transaction statistics, query statistics, plan statistics, wait event statistics, buffer usage statistics, or write-ahead logging (WAL) usage statistics.

12 . The system of claim 11 , wherein, in response to receiving the indication to cease collecting the first statistic type, the instructions, when executed on the data processing hardware, further cause the data processing hardware to:

identify a particular node of a tree data structure of the schema that corresponds to the first statistic; and

cease collection of statistics associated with each node in the tree data structure that is descendant from the particular node.

13 . The system of claim 10 , wherein the instructions, when executed on the data processing hardware, further cause the data processing hardware to:

access the stored plurality of statistics, by the data processing hardware, from the respective data table according to the schema; and

execute a second query on the data store, the second query being based at least in part on the accessed plurality of statistics.

14 . The system of claim 11 , wherein the plurality of statistics include one or more of:

instance statistics;

session statistics;

auxiliary process statistics;

transaction statistics;

activity statistics;

wait event statistics;

query statistics; or

plan statistics.

15 . The system of claim 10 , wherein the instructions, when executed on the data processing hardware, further cause the data processing hardware to:

normalize the plurality of statistics to remove identifying information.

16 . The system of claim 15 , wherein the instructions, when executed on the data processing hardware, further cause the data processing hardware to:

aggregate a plurality of normalized statistics corresponding to a plurality of queries; and

generate a plurality of generalized statistics corresponding to the plurality of queries.

17 . The system of claim 10 , wherein the instructions, when executed on the data processing hardware, further cause the data processing hardware to:

determine whether a size of collected statistics satisfies a threshold size,

wherein receiving the indication to cease collecting the first statistic type is in response to determining that the size of the collected statistics satisfies the threshold size.

18 . The system of claim 10 , wherein the instructions, when executed on the data processing hardware, further cause the data processing hardware to:

determine whether the duration of collected statistics satisfies a threshold size,

wherein receiving the indication to cease collecting the first statistic type is in response to determining that the duration of the collected statistics satisfies the threshold size.

19 . A non-transitory computer-readable medium storing instructions that, when executed by processing circuitry, cause the processing circuitry to:

receive a schema defining a relational database for storing a plurality of statistics corresponding to a query, the relational database comprising a plurality of data tables relationally connected according to the schema, each data table of the plurality of data tables corresponding to a respective statistic, wherein the plurality of statistics includes a first statistic having a first statistic type and a second statistic having a second statistic type that is distinct from the first statistic type, wherein the first and second statistic types are selected from the group consisting of: instance statistics, session statistics, transaction statistics, query statistics, plan statistics, and wait event statistics;

obtain the query corresponding to data at a data store;

execute, using a query executor, the query on the data at the data store;

during execution of the query on the data at the data store:

collect, as the query is executing, from a query execution database, the plurality of statistics indicative of performance characteristics specific to the query as the query executes, each statistic of the plurality of statistics corresponding to a respective data table of the plurality of data tables of the relational database; and

for each statistic of the plurality of statistics, storing the respective statistic at the respective data table according to the schema;

receive an indication to cease collecting the first statistic type; and

in response to receiving the indication, cease collection of the first statistic associated with the first statistic type while continuing to collect at least the second statistic associated with the second statistic type.

20 . The non-transitory computer-readable medium of claim 19 , wherein, in response to receiving the indication to cease collecting the first statistic type, the instructions, when executed on the data processing hardware, further cause the data processing hardware to:

identify a particular node of a tree data structure of the schema that corresponds to the first statistic; and

cease collection of statistics associated with each node in the tree data structure that is descendant from the particular node.