Dynamic sourcing
Dynamic sourcing, in which a data request that is associated with a query is received and a parameter of data needed for satisfaction of the query is identified. Parameter information defining data available in at least one cube stored in a cache is accessed and the parameter is compared with the parameter information. Based on comparison results, it is determined whether one or more cubes in the cache include sufficient data to satisfy the query. In response to a determination that one or more cubes include sufficient data to satisfy the query, a response to the data request is generated by executing the query against the one or more cubes. In response to a determination that the cubes do not include sufficient data to satisfy the query, a response to the data request is generated by executing at least a portion of the query against a database system.
1. A computer-implemented method comprising:
receiving a data request that is associated with a query;
identifying data needed for satisfaction of the query;
accessing, from electronic storage, information defining data available in data structures stored in memory at a time of the data request;
comparing the identified data needed for satisfaction of the query with the accessed information defining data available in the data structures in the memory at the time of the data request;
based on comparison results, determining that the identified data needed for satisfaction of the query is not represented by a single data structure in the memory at the time of the data request;
based on the determination that the identified data needed for satisfaction of the query is not represented by a single data structure in the memory at the time of the data request, determining that at least a first portion of the identified data needed for satisfaction of the query is represented by a single data structure, and at least a second portion of the identified data needed for satisfaction of the query is obtainable through processing data at a data warehouse;
based on the determination that at least a first portion of the identified data needed for satisfaction of the query is represented by a single data structure, and at least a second portion of the identified data needed for satisfaction of the query is obtainable through processing data at a data warehouse, processing the data at the data warehouse to obtain the second portion of the data needed for satisfaction of the query; and
generating a response to the data request using the data needed for satisfaction of the query by combining the first portion of data from the single data structure and the second portion of data obtained by processing the data at the data warehouse.
2. The method of claim 1 :
wherein identifying data needed for satisfaction of the query comprises identifying one or more attributes of data needed for satisfaction of the query;
wherein accessing, from electronic storage, information defining data available in data structures stored in memory at the time of the data request comprises accessing, from electronic storage, attribute information for data structures stored in the memory at the time of the data request;
wherein comparing the identified data needed for satisfaction of the query with the accessed information defining data available in data structures stored in memory at the time of the data request comprises comparing the identified one or more attributes of data needed for satisfaction of the query with the accessed attribute information for data structures stored in the memory at the time of the data request; and
wherein determining that the identified data needed for satisfaction of the query is not represented by a single data structure in the memory at the time of the data request comprises determining that the one or more attributes of data needed for satisfaction of the query are not included in a single data structure in the memory at the time of the data request.
3. The method of claim 1 :
wherein identifying data needed for satisfaction of the query comprises identifying a filter parameter of the query;
wherein accessing, from electronic storage, information defining data available in data structures stored in memory at the time of the data request comprises accessing, from electronic storage, filter information for data structures stored in the memory at the time of the data request;
wherein comparing the identified data needed for satisfaction of the query with the accessed information defining data available in data structures stored in memory at the time of the data request comprises comparing the identified filter parameter of the query with the accessed filter information for data structures stored in the memory at the time of the data request; and
wherein determining that the identified data needed for satisfaction of the query is not represented by a single data structure in the memory at the time of the data request comprises determining that the identified filter parameter of the query does not match a single data structure in the memory at the time of the data request.
4. The method of claim 1 :
wherein identifying data needed for satisfaction of the query comprises identifying one or more metrics needed for satisfaction of the query;
wherein accessing, from electronic storage, information defining data available in data structures stored in memory at the time of the data request comprises accessing, from electronic storage, metric information for data structures stored in the memory at the time of the data request;
wherein comparing the identified data needed for satisfaction of the query with the accessed information defining data available in data structures stored in memory at the time of the data request comprises comparing the identified one or more metrics needed for satisfaction of the query with the accessed metric information for data structures stored in the memory at the time of the data request; and
wherein determining that the identified data needed for satisfaction of the query is not represented by a single data structure in the memory at the time of the data request comprises determining that the identified one or more metrics needed for satisfaction of the query are not included in a single data structure in the memory at the time of the data request.
5. The method of claim 1 , wherein the data structures stored in memory at a time of the data request comprise cubes, each of the cubes being a pre-processed abstraction of data stored in a database system and being stored in the memory prior to receipt of the data request.
6. The method of claim 1 , wherein accessing, from electronic storage, information defining data available in data structures stored in memory at the time of the data request comprises accessing, from the memory, the data structures stored in memory at the time of the data request.
7. The method of claim 1 , wherein receiving the data request that is associated with the query comprises receiving a query.
8. The method of claim 1 , wherein receiving the data request that is associated with the query comprises receiving a report generation request that requires execution of a query to gather data required to generate the report.
9. A system comprising:
one or more computers; and
at least one computer-readable medium coupled to the one or more computers having instructions stored thereon which, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
receiving a data request that is associated with a query;
identifying data needed for satisfaction of the query;
accessing, from electronic storage, information defining data available in data structures stored in memory at a time of the data request;
comparing the identified data needed for satisfaction of the query with the accessed information defining data available in the data structures in the memory at the time of the data request;
based on comparison results, determining that the identified data needed for satisfaction of the query is not represented by a single data structure in the memory at the time of the data request;
based on the determination that the identified data needed for satisfaction of the query is not represented by a single data structure in the memory at the time of the data request, determining that at least a first portion of the identified data needed for satisfaction of the query is represented by a single data structure, and at least a second portion of the identified data needed for satisfaction of the query is obtainable through processing data at a data warehouse;
based on the determination that at least a first portion of the identified data needed for satisfaction of the query is represented by a single data structure, and at least a second portion of the identified data needed for satisfaction of the query is obtainable through processing data at a data warehouse, processing the data at the data warehouse to obtain the second portion of the data needed for satisfaction of the query; and
generating a response to the data request using the data needed for satisfaction of the query by combining the first portion of data from the single data structure and the second portion of data obtained by processing the data at the data warehouse.
10. The system of claim 9 :
wherein identifying data needed for satisfaction of the query comprises identifying one or more attributes of data needed for satisfaction of the query;
wherein accessing, from electronic storage, information defining data available in data structures stored in memory at the time of the data request comprises accessing, from electronic storage, attribute information for data structures stored in the memory at the time of the data request;
wherein comparing the identified data needed for satisfaction of the query with the accessed information defining data available in data structures stored in memory at the time of the data request comprises comparing the identified one or more attributes of data needed for satisfaction of the query with the accessed attribute information for data structures stored in the memory at the time of the data request; and
wherein determining that the identified data needed for satisfaction of the query is not represented by a single data structure in the memory at the time of the data request comprises determining that the one or more attributes of data needed for satisfaction of the query are not included in a single data structure in the memory at the time of the data request.
11. The system of claim 9 :
wherein identifying data needed for satisfaction of the query comprises identifying a filter parameter of the query;
wherein accessing, from electronic storage, information defining data available in data structures stored in memory at the time of the data request comprises accessing, from electronic storage, filter information for data structures stored in the memory at the time of the data request;
wherein comparing the identified data needed for satisfaction of the query with the accessed information defining data available in data structures stored in memory at the time of the data request comprises comparing the identified filter parameter of the query with the accessed filter information for data structures stored in the memory at the time of the data request; and
wherein determining that the identified data needed for satisfaction of the query is not represented by a single data structure in the memory at the time of the data request comprises determining that the identified filter parameter of the query does not match a single data structure in the memory at the time of the data request.
12. The system of claim 9 :
wherein identifying data needed for satisfaction of the query comprises identifying one or more metrics needed for satisfaction of the query;
wherein accessing, from electronic storage, information defining data available in data structures stored in memory at the time of the data request comprises accessing, from electronic storage, metric information for data structures stored in the memory at the time of the data request;
wherein comparing the identified data needed for satisfaction of the query with the accessed information defining data available in data structures stored in memory at the time of the data request comprises comparing the identified one or more metrics needed for satisfaction of the query with the accessed metric information for data structures stored in the memory at the time of the data request; and
wherein determining that the identified data needed for satisfaction of the query is not represented by a single data structure in the memory at the time of the data request comprises determining that the identified one or more metrics needed for satisfaction of the query are not included in a single data structure in the memory at the time of the data request.
13. The system of claim 9 , wherein the data structures stored in memory at a time of the data request comprise cubes, each of the cubes being a pre-processed abstraction of data stored in a database system and being stored in the memory prior to receipt of the data request.
14. The system of claim 9 , wherein accessing, from electronic storage, information defining data available in data structures stored in memory at the time of the data request comprises accessing, from the memory, the data structures stored in memory at the time of the data request.
15. The system of claim 9 , wherein receiving the data request that is associated with the query comprises receiving a query.
16. The system of claim 9 , wherein receiving the data request that is associated with the query comprises receiving a report generation request that requires execution of a query to gather data required to generate the report.