Method and system for dynamically optimizing client queries to read-mostly servers
View Patent ↗A system and method of query optimization are disclosed. A complexity measure of a client query received from a client application is compared to a predetermined threshold, which is determined using at least one of system data and query execution data. An optimized query is generated by modifying the client query to reduce the complexity measure of the client query if the complexity measure of the client query exceeds the predetermined threshold. The optimized query is submitted to a server application on a server and a post-processing routine to be applied to the result set is selected.
1. A method of query optimization, comprising:
comparing a complexity measure of a client query received from a client application to a predetermined threshold, the client query received before the client query reaches a server application on a server, wherein the predetermined threshold is determined based on query execution data collected from at least one previous query;
generating an optimized query by modifying the client query to reduce the complexity measure of the client query if the complexity measure of the client query exceeds the predetermined threshold, wherein an expected result set size of the optimized query is greater than an expected result set size of the client query;
submitting the optimized query to the server application on the server;
selecting a post-processing routine to be applied to a result set of the optimized query to have a filtered result set; and
returning the filtered set to the client application.
2. The method of claim 1 , wherein the complexity measure of the client query is calculated using at least one of a type of operators in the client query, a number of operators in the client query, and a number of sub-queries.
3. The method of claim 1 , wherein the predetermined threshold is determined further based on system data comprising at least one of:
static system data including at least one of server performance characteristics, server capacity characteristics, a server content distribution, a predetermined maximum client load, a predetermined maximum server load, and predetermined maximum network load; and
dynamic system data including at least one of a measure of the server load at the time a query is submitted, a measure of the client load at the time the query is submitted, and a measure of the network load at the time the query is submitted.
4. The method of claim 1 , further comprising intercepting the client query from a client application before the client query reaches the server application.
5. The method of claim 1 , wherein the query execution data comprises, for the at least one previous query, at least one of a result set size, a query response time, a query execution time, a measure of the network load at the time of query execution, and a query complexity measure.
6. The method of claim 1 , wherein the at least one previous query comprises at least one of a client query, an optimized query, or a probe query.
7. The method of claim 1 , further comprising:
submitting a probe query to the server application, wherein the probe query is a user configured query or a frequently used query;
automatically submitting the probe query to the server application at a selected time; and
collecting query execution data for the probe query, wherein the query execution data for the probe query includes at least one of a result set size, a query response time, a measure of the network load at the time of query execution, and a query complexity measure.
8. A system comprising:
a processor;
a computer readable medium communicatively coupled to the processor; and
computer-executable instructions encoded in the computer readable medium, the instructions, when executed by the processor, operable to perform operations comprising:
comparing a complexity measure of a client query received from a client application to a predetermined threshold, the client query received before the client query reaches a server application on a server, wherein the predetermined threshold is determined based on query execution data collected from at least one previous query;
generating an optimized query by modifying the client query to reduce the complexity measure of the client query if the complexity measure of the client query exceeds the predetermined threshold, wherein an expected result set size of the optimized query is greater than an expected result set size of the client query;
submitting the optimized query to the server application on the server;
selecting a post-processing routine to be applied to a result set of the optimized query to have a filtered result set; and
returning the filtered set to the client application.
9. The system of claim 8 , wherein the complexity measure of the client query is calculated using at least one of a type of operators, a number of operators in the client query or a number of sub-queries.
10. The system of claim 8 , wherein:
the predetermined threshold is determined further based on system data comprising at least one of:
static system data including at least one of server performance characteristics, server capacity characteristics, a server content distribution, a predetermined maximum client load, a predetermined maximum server load, and predetermined maximum network load; and
dynamic system data including at least one of a measure of the server load at the time a query is submitted, a measure of the client load at the time the query is submitted, and a measure of the network load at the time the query is submitted; and
the query execution data comprises, for the at least one previous query, at least one of a result set size, a query response time, a measure of the network load at the time of query execution, and a query complexity measure.
11. The system of claim 8 , wherein the at least one previous query comprises at least one of a client query, an optimized query, or a probe query.
12. The system of claim 8 , wherein the instructions are further operable to perform operations comprising:
submitting a probe query to the server application, wherein the probe query is a user configured query or a frequently used query;
automatically submitting the probe query to the server application at a selected time; and
collecting query execution data for the probe query, wherein the query execution data for the probe query includes at least one of a result set size, a query response time, a measure of the network load at the time of query execution, and a query complexity measure.
13. A non-transitory, computer readable medium including computer-executable instructions encoded in the computer readable medium, the instructions, when executed by the processor, operable to perform operations comprising:
comparing a complexity measure of a client query received from a client application to a predetermined threshold, the client query received before the client query reaches a server application on a server, wherein the predetermined threshold is determined based on query execution data collected from at least one previous query;
modifying the client query to generate an optimized query if the complexity measure of the client query exceeds the predetermined threshold, wherein an expected result set size of the optimized query is greater than an expected result set size of the client query;
submitting the optimized query to the server application on the server;
selecting a post-processing routine to be applied to a result set of the optimized query to have a filtered result set; and
returning the filtered result set to the client application.
14. The non-transitory, computer readable medium of claim 13 , wherein the complexity measure of the client query is calculated using at least one of a type of operators, a number of operators in the client query or a number of sub-queries.
15. The non-transitory, computer readable medium of claim 13 , wherein:
the predetermined threshold is determined further based on system data comprising at least one of:
static system data including at least one of server performance characteristics, server capacity characteristics, a server content distribution, a predetermined maximum client load, a predetermined maximum server load, and predetermined maximum network load; and
dynamic system data including at least one of a measure of the server load at the time a query is submitted, a measure of the client load at the time the query is submitted, and a measure of the network load at the time the query is submitted; and
the query execution data comprises, for the at least one previous query, at least one of a result set size, a query response time, a measure of the network load at the time of query execution, and a query complexity measure.
16. The non-transitory, computer readable medium of claim 13 , wherein the at least one previous query comprises at least one of a client query, an optimized query, or a probe query.
17. The non-transitory, computer readable medium of claim 13 , the computer-executable instructions when executed by the processor, further operable to perform operations comprising:
submitting a probe query to the server application, wherein the probe query is a user configured query or a frequently used query;
automatically submitting the probe query to the server application at a selected time; and
collecting query execution data for the probe query, wherein the query execution data for the probe query includes at least one of a result set size, a query response time, a measure of the network load at the time of query execution, and a query complexity measure.