MISO (multistore-online-tuning) system

1. A system, comprising:

a processor;

a memory;

first and second data stores, each store having a set of materialized views of the base data and the views comprise a multistore physical design;

an execution layer coupled to the data stores;

a query optimizer coupled to the execution layer; and

a tuner coupled to the query optimizer and the execution layer, wherein the tuner determines a placement of the materialized views across the stores to improve workload performance upon considering each store's view storage budget and a transfer budget when moving views across the stores, wherein at each reorganization phase, the tuner generates a new multistore design M new = V h new ,V d new that minimizes a total workload cost TotalCost(W,M new ) by maximizing a benefit of a view for M new for a representative workload W, wherein the benefit of a view v∈V for a query q is a change in cost of q evaluated with and without view v present in the multistore design and benefit(q,v)=cost(q,M∪v)−cost(q,M), where M∪v means that v is added to both stores in M, wherein the tuner includes code for:

grouping views in the current designs into interacting sets;

sparsifying a set based on interacting views within a set and if the interacting views are strongly positive, considering the views together as a single candidate;

choosing a representative view as a candidate and discard the rest and generating a set of candidate views that contain views to be considered independently when computing a new multistore design;

solving two multidimensional knapsack problems in sequence, wherein the dimensions of each knapsack include a storage budget and transfer budget constraints.

2. The system of claim 1 , wherein the first and second data stores comprise a big data store and a relational database management system (RDBMS).

3. The system of claim 2 , wherein the big data store is Hive (HV) and the RDBMS is a parallel data warehouse (DW).

4. The system of claim 1 , comprising code for utilizing spare capacity in the DW and tuning a physical design of the stores for a workload of queries.

5. The system of claim 1 , wherein each store has a set of materialized views of base data.

6. The system of claim 1 , comprising code for:

receiving by-products of query processing in the multistore system, wherein the by-products include views or materializations of intermediate data;

placing the views or materializations across the stores based on recently observed queries as indicative of a future query workload;

determining a benefit score for each view based on a predicted future query workload, wherein each store has an allotted view storage budget, and there is a view transfer budget for transferring views between the stores; and

tuning a physical design of the multistore system.

7. The system of claim 1 , comprising code for transferring views between the stores to fit a final view placement within all of budgets and minimize a cost of the future query workload.

8. The system of claim 1 , comprising code for computing a solution for a second store.

9. The system of claim 1 , comprising code for determining a solution for the second store with a cost below a view storage budget for the high performance store and below remaining view transfer budgets not consumed by the solution for the high performance store.

10. The system of claim 1 , comprising code for deciding which view should be placed in which store and which views should be discarded.

11. The system of claim 1 , comprising code for deciding the most beneficial set of views for a given storage budget.

12. The system of claim 1 , comprising code for sing deciding the most beneficial set of views to transfer across stores.

13. The system of claim 1 , comprising code for selecting the most beneficial views for a RDBMS.

14. The system of claim 1 , comprising code for determining which views for the big data store is chosen next.

15. The system of claim 1 , comprising code for handling interactions between views.

16. The system of claim 1 , comprising code for using a dynamic programming solution to pack both stores.