Optimized database appliance
A system from processing database queries allows for cost and locale based distribution for execution of database queries. The database queries are executed on execution engines that provide flexible configuration and overlapping functionality. The system reduces various costs, including elapsed time, required to perform database queries. The system provides processing of a database query using a database catalog comprising database table locality information, record locality information and execution engine information. A query optimizer receives the query and accesses the database catalog to create a query execution plan comprising locality-based database operations. A central database operation processor providing a first execution engine executes the query execution plan by performing at least a portion of the locality-based database operations and distributing at least a portion of the locality-based database operations as a subplan. A second database operation processor providing a second execution engine executes the subplan received from the central database operation processor. At least one of the database operations can be executed on either the first execution engine or the second execution engine. A storage unit stores at least a portion of database tables and records. A data communications network connects the central database processor to the second database processor.
1. A system for processing a query on a database comprising:
a plurality of database operation processor components, with at least two of the database operation processor components having different operation processing capabilities, and such that more than one database operation processor component is capable of executing a selected database operation;
a database catalog comprising database table information, database operation processor component locale information, and database operation processor characteristic information;
a query optimizer, running on a first database operation processor component, for receiving the query and accessing the database catalog to develop a query execution plan, the query execution plan defining a list of database operations to carry out the query, the query execution plan also defining a respective database operation processor component locale for each such database operation;
wherein the first database operation processor component not only executes portions of the query execution plan designated to be handled itself, but also forwards at least a selected portion of the query execution plan to a second database operation processor component having different operation processing capabilities than the first database operation processor component; and
wherein the second database operation processor component executes portions of the query execution plan designated for itself, and also forwards other selected portions of the query execution plan to a third database operation component for execution.
2. The system of claim 1 wherein the plurality of the database operation processor components further comprise:
a central database operation processor comprising a first execution engine to provide a first one of the database operation components;
a second database operation processor comprising a second execution engine to provide a second one of the database operation components;
a storage unit storing at least a portion of database tables and records, connected to at least one of the central database operation processor and the second database operation processor; and
a data communications network connecting the central database operation processor to the second database operation processor.
3. The system of claim 2 wherein the central database operation processor is configured as a symmetric multi-processing system and the second database operation processor is configured as a component of a massively parallel processing system.
4. The system of claim 2 wherein the first execution engine or the second execution engine processes data records received from the storage unit by performing one or more operations on the data records without materializing the data record into a memory.
5. The system of claim 2 wherein the first execution engine or the second execution engine processes data records received from the data communications network by performing one or more operations on the data records without materializing the data record into a memory.
6. The system of claim 2 wherein either the first execution engine or the second execution engine dynamically reconfigures an assigned execution subplan to redirect one or more operations to the third execution engine, so that the query execution plan is re-optimized while still being executed.
7. The system of claim 1 wherein the first database operation processor component, the second database operation processor component, and the third database operation processor component have different processing capabilities selected from a group of processing capabilities consisting of execution engine memory, execution engine processor speed, cache configuration, disk speed, disk controller speed, disk controller bandwidth, network interface bandwidth or network interface speed.
8. The system of claim 1 wherein the first database operation processor component or the second database operation processor component reconfigures an executing subplan.
9. The system of claim 1 wherein the first database operation processor component comprises a general purpose central processing unit.
10. The system of claim 1 wherein the second database operation processor component comprises an application specific integrated circuit.
11. The system of claim 1 wherein the third database operation processor component comprises a field programmable gate array.