Optimizing query performance in a data virtualization system

1 . A computer implemented method for optimizing query performance in a data virtualization system, the computer implemented method comprising:

generating, by a processor set, a number of query collections comprising query statements with different query operators;

determining, by the processor set, network transfer times for data between the data virtualization system and a number of data sources connected with the data virtualization system, and an execution time for executing each query collection from the number of query collections, wherein the data virtualization system delegates queries to the number of data sources for remote execution, and wherein query statements with unsupported query operators are executed locally on the data virtualization system for determining execution times while query statements with supported query operators are executed remotely on the number of data sources for determining execution times;

identifying, by the processor set, a number of factors associated with query performance for the data virtualization system;

training, by the processor set, a machine learning model for estimating execution times for the queries using a training dataset comprising the number of factors, the network transfer times, and the execution times for executing the number of query collections;

evaluating, by the processor set using the machine learning model in real-time, network conditions and performances of the number of data sources for executing queries for the data virtualization system; and

adjusting, by the processor set, delegation of the queries to be executed by the number of data sources for the data virtualization system based on the evaluation.

2 . The computer implemented method of claim 1 , further comprising:

monitoring, by the processor set, query executions on the data virtualization system and the number of data sources over time to generate historical data of query performance for the data virtualization system;

forecasting, by the processor set, workload changes for the number of data sources connected to the data virtualization system based on the historical data; and

adjusting, by the processor set, delegation of the queries to be executed by the number of data sources for the data virtualization system based on the forecasted workload changes.

3 . The computer implemented method of claim 2 , further comprising:

retraining, by the processor set, the machine learning model using the historical data of query performance for the data virtualization system.

4 . The computer implemented method of claim 1 , wherein the network transfer times for data between the data virtualization system and the number of data sources connected with the data virtualization system are determined by simulating transferring datasets of different sizes and types between the data virtualization system and the number of data sources connected with the data virtualization system.

5 . The computer implemented method of claim 1 , wherein the number of factors comprise at least one of operation type, dataset size, data type, available resources in the number of data sources, network bandwidth, and network latency.

6 . The computer implemented method of claim 1 , wherein the number of query collections comprise query statements with operators unsupported by the number of data sources.

7 . The computer implemented method of claim 1 , wherein the machine learning model uses a regression algorithm for estimating execution time of the queries.

8 . A computer system comprising:

a processor set;

a set of one or more computer-readable storage media; and

program instructions stored on the set of one or more storage media to cause the processor set to perform operations comprising:

generating a number of query collections comprising query statements with different query operators;

determining network transfer times for data between a data virtualization system and a number of data sources connected with the data virtualization system, and an execution time for executing each query collection from the number of query collections, wherein the data virtualization system delegates queries to the number of data sources for remote execution, and wherein query statements with unsupported query operators are executed locally on the data virtualization system for determining execution times while query statements with supported query operators are executed remotely on the number of data sources for determining execution times;

identifying a number of factors associated with query performance for the data virtualization system;

training a machine learning model for estimating execution times for the queries using a training dataset comprising the number of factors, the network transfer times, and the execution times for executing the number of query collections;

evaluating network conditions and performances of the number of data sources for executing queries for the data virtualization system using the machine learning model in real-time; and

adjusting delegation of the queries to be executed by the number of data sources for the data virtualization system based on the evaluation.

9 . The computer system of claim 8 , wherein the operations further comprise:

monitoring query executions on the data virtualization system and the number of data sources over time to generate historical data of query performance for the data virtualization system;

forecasting workload changes for the number of data sources connected to the data virtualization system based on the historical data; and

adjusting delegation of the queries to be executed by the number of data sources for the data virtualization system based on the forecasted workload changes.

10 . The computer system of claim 9 , wherein the operations further comprise:

retraining the machine learning model using the historical data of query performance for the data virtualization system.

11 . The computer system of claim 8 , wherein the network transfer times for data between the data virtualization system and the number of data sources connected with the data virtualization system are determined by simulating transferring datasets of different sizes and types between the data virtualization system and the number of data sources connected with the data virtualization system.

12 . The computer system of claim 8 , wherein the number of factors comprise at least one of operation type, dataset size, data type, available resources in the number of data sources, network bandwidth, and network latency.

13 . The computer system of claim 8 , wherein the number of query collections comprise query statements with operators unsupported by the number of data sources.

14 . The computer system of claim 8 , wherein the machine learning model uses a regression algorithm for estimating execution time of the queries.

15 . A computer program product for optimizing query performance in a data virtualization system, the computer program product comprising:

a set of one or more computer-readable storage media;

program instructions stored in the set of one or more storage media to perform operations comprising:

generating, by a processor set, a number of query collections comprising query statements with different query operators;

determining, by the processor set, network transfer times for data between the data virtualization system and a number of data sources connected with the data virtualization system, and an execution time for executing each query collection from the number of query collections, wherein the data virtualization system delegates queries to the number of data sources for remote execution, and wherein query statements with unsupported query operators are executed locally on the data virtualization system for determining execution times while query statements with supported query operators are executed remotely on the number of data sources for determining execution times;

identifying, by the processor set, a number of factors associated with query performance for the data virtualization system;

training, by the processor set, a machine learning model for estimating execution times for the queries using a training dataset comprising the number of factors, the network transfer times, and the execution times for executing the number of query collections;

evaluating, by the processor set using the machine learning model in real-time, network conditions and performances of the number of data sources for executing queries for the data virtualization system; and

adjusting, by the processor set, delegation of the queries to be executed by the number of data sources for the data virtualization system based on the evaluation.

16 . The computer program product of claim 15 , wherein the operations further comprise:

monitoring, by the processor set, query executions on the data virtualization system and the number of data sources over time to generate historical data of query performance for the data virtualization system;

forecasting, by the processor set, workload changes for the number of data sources connected to the data virtualization system based on the historical data; and

adjusting, by the processor set, delegation of the queries to be executed by the number of data sources for the data virtualization system based on the forecasted workload changes.

17 . The computer program product of claim 16 , wherein the operations further comprise:

retraining, by the processor set, the machine learning model using the historical data of query performance for the data virtualization system.

18 . The computer program product of claim 15 , wherein the network transfer times for data between the data virtualization system and the number of data sources connected with the data virtualization system are determined by simulating transferring datasets of different sizes and types between the data virtualization system and the number of data sources connected with the data virtualization system.

19 . The computer program product of claim 15 , wherein the number of factors comprise at least one of operation type, dataset size, data type, available resources in the number of data sources, network bandwidth, and network latency.

20 . The computer program product of claim 15 , wherein the number of query collections comprise query statements with operators unsupported by the number of data sources.