Metadata-based statistics-oriented processing of queries in an on-demand environment
In accordance with embodiments, there are provided mechanisms and methods for facilitating metadata-based statistics-oriented query processing for large datasets in an on-demand services environment. In one embodiment and by way of example, a method comprises evaluating metadata associated with a query placed on behalf of a tenant in a multi-tenant environment, and computing process statistics for the query based on the metadata, where the process statistics reveal an estimation of resources needed for execution of the query within a predictable amount of time and using fewer than or equal to an allocated number of scans of a database. The method may further include associating, based on the process statistics, a set of rules and the estimated resources to process the query, and executing the query based on the set of rules and using the estimated resources such that the query is processed within the predictable amount of time and using fewer than or equal to the allocated number of scans of the database.
1. A method comprising: evaluating metadata associated with a query placed on behalf of a tenant in a multi-tenant environment; computing processing statistics for the query based on the metadata, wherein the processing statistics to identify a maximum number of scans of a database allocated to the query for gathering of data in processing of the query, wherein computing includes estimating a predictable amount of time for processing of the query using fewer than or equal to the allocated maximum number of scans of the database, wherein the allocated maximum number of scans is based on an estimated number of bytes based on a predetermined threshold of bytes to scan the query within the predictable amount of time based on the processing statistics, wherein the query is performed within the predictable amount of time when the estimated number of bytes is less than or equal to the predetermined threshold of bytes, wherein the processing statistics to identify one or more portions of the database subjects to the maximum number of scans without scanning other portions of the database; generating, based on the processing statistics, one or more processing rules, wherein the one or more processing rules are assigned to the query; and processing the query based on the assigned one or more rules such that the query is processed within the predictable amount of time and using fewer than or equal to the allocated maximum number of scans of the database.
2. The method of claim 1 , further comprising:
generating results based on processing of the query;
transmitting the results to a client computing device over a communication network, wherein the query is received from the client computing device.
3. The method of claim 1 , further comprising collecting the metadata from the client computing device, wherein the metadata contains information revealing one or more characteristics of the query.
4. The method of claim 1 , wherein the processing statistics to further identify one or more portions of the database having contents pertinent to the query.
5. The method of claim 4 , further comprising accessing the one or more portions of the database without having to scan or access scanning or accessing other portions of the database.
6. The method of claim 1 , wherein the results are further generated based on efficient classes associated with the query and without having to access or process inefficient classes.
7. A database system comprising: a server computing device having a data processing device coupled to a memory, the data processing device to facilitate operations comprising: evaluating metadata associated with a query placed on behalf of a tenant in a multi-tenant environment; computing processing statistics for the query based on the metadata, wherein the processing statistics to identify a maximum number of scans of a database allocated to the query for gathering of data in processing of the query, wherein computing includes estimating a predictable amount of time for processing of the query using fewer than or equal to the allocated maximum number of scans of the database, wherein the allocated maximum number of scans is based on an estimated number of bytes based on a predetermined threshold of bytes to scan the query within the predictable amount of time based on the processing statistics, wherein the query is performed within the predictable amount of time when the estimated number of bytes is less than or equal to the predetermined threshold of bytes, wherein the processing statistics to identify one or more portions of the database subjects to the maximum number of scans without scanning other portions of the database; generating, based on the processing statistics, one or more processing rules, wherein the one or more processing rules are assigned to the query; and processing the query based on the assigned one or more rules such that the query is processed within the predictable amount of time and using fewer than or equal to the allocated maximum number of scans of the database.
8. The system of claim 7 , wherein the operations further comprise:
generating results based on processing of the query;
transmitting the results to a client computing device over a communication network, wherein the query is received from the client computing device.
9. The system of claim 7 , wherein the operations further comprise collecting the metadata from the client computing device, wherein the metadata contains information revealing one or more characteristics of the query.
10. The system of claim 7 , wherein the processing statistics to further identify one or more portions of the database having contents pertinent to the query.
11. The system of claim 10 , wherein the operations further comprise accessing the one or more portions of the database without having to scan or access other portions of the database.
12. The system of claim 7 , wherein the results are further generated based on efficient classes associated with the query and without having to access or process inefficient classes.
13. A non-transitory computer-readable medium having stored thereon instructions which, when executed, cause a computing device to perform operations comprising: evaluating metadata associated with a query placed on behalf of a tenant in a multi-tenant environment; computing processing statistics for the query based on the metadata, wherein the processing statistics to identify a maximum number of scans of a database allocated to the query for gathering of data in processing of the query, wherein computing includes estimating a predictable amount of time for processing of the query using fewer than or equal to the allocated maximum number of scans of the database, wherein the allocated maximum number of scans is based on an estimated number of bytes based on a predetermined threshold of bytes to scan the query within the predictable amount of time based on the processing statistics, wherein the query is performed within the predictable amount of time when the estimated number of bytes is less than or equal to the predetermined threshold of bytes, wherein the processing statistics to identify one or more portions of the database subjects to the maximum number of scans without scanning other portions of the database; gathering, based on the processing statistics, one or more processing rules, wherein the one or more processing rules are assigned to the query; and processing the query based on the assigned one or more rules such that the query is processed within the predictable amount of time and using fewer than or equal to the allocated maximum number of scans of the database.
14. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise:
generating results based on processing of the query;
transmitting the results to a client computing device over a communication network, wherein the query is received from the client computing device.
15. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise collecting the metadata from the client computing device, wherein the metadata contains information revealing one or more characteristics of the query.
16. The non-transitory computer-readable medium of claim 13 , wherein the processing statistics to further identify one or more portions of the database having contents pertinent to the query.
17. The non-transitory computer-readable medium of claim 16 , wherein the operations further comprise accessing the one or more portions of the database without having to scan or access other portions of the database.
18. The non-transitory computer-readable medium of claim 13 , wherein the results are further generated based on efficient classes associated with the query and without having to access or process inefficient classes.