Data aggregation module supporting dynamic query responsive aggregation during the servicing of database query requests provided by one or more client machines
Improved method of and apparatus for aggregating data elements in multidimensional databases (MDDB). In the preferred embodiment, the apparatus is realized in the form of a high-performance stand-alone (i.e. external) aggregation server which can be plugged-into conventional MOLAP systems to achieve significant improvements in system performance. In accordance with the principles of the present invention, the stand-alone aggregation server contains a scalable MDDB and a high-performance aggregation engine that are integrated into the modular architecture of the aggregation server. The stand-alone aggregation server of the present invention can uniformly distribute data elements among a plurality of processors, for balanced loading and processing, and therefore is highly scalable. The stand-alone aggregation server of the present invention can be used to realize (i) an improved MDDB for supporting on-line analytical processing (OLAP) operations, (ii) an improved Internet URL Directory for supporting on-line information searching operations by Web-enabled client machines, as well as (iii) diverse types of MDDB-based systems for supporting real-time control of processes in response to complex states of information reflected in the MDDB.
1 . A data aggregation module for interfacing with a relational database management subsystem (RDBMS) having a relational data store including one or more fact tables for storing at least the base data, and a meta-data store for storing a dictionary containing dimension data, and servicing database query requests provided by one or more client machines, said data aggregation module comprising:
a multi-dimensional database (MDB) having a multi-dimensional data storage structure including a plurality of data storage cells, wherein each said data storage cell is indexed with multiple dimensions and stores either a base data value or an aggregated multi-dimensional data value;
a base data loader for loading said directory data and said base data to said MDD aggregation module;
a first data communication interface operably connected between said base loader and said RDBMS, for transmitting directory data and base data from said RDBMS to said aggregation module;
an aggregation engine for receiving base data from said base data loader, and calculating aggregated multi-dimensional data from the base data according to a multi-dimensional data aggregation process;
a MDDB handler for storing aggregated multi-dimensional data in said MDDB and retrieving aggregated multi-dimensional data from said MDDB;
a second data communication interface operably connected to said data aggregation engine and said one or more client machines, for receiving database query requests from said client machines;
a database query handler operably connected to said second data communication interface, for handling said database query requests;
a controller for managing the operation of said base data loader, said aggregation engine, said MDDB handler, and said database query handler;
wherein after base data loading operations, the base data is initially aggregated along at least one of the dimensions extracted from said dictionary and aggregated multi-dimensional data results are stored in said MDDB; and
wherein during query servicing operations, each database query request is received over said second data communication interface, (ii) the set of dimensions associated with said database query request is extracted, (iii) said dimensions are provided as multi-dimensional coordinates to said MDDB handler so that (i) said MDDB handler can retrieve aggregated multi-dimensional data from data storage cell(s) in said MDDB specified by said multi-dimensional coordinates, and/or (ii) said aggregation engine can dynamically generate, on demand, aggregated multi-dimensional data for storage at data storage cell(s) in said MDDB specified by said multi-dimensional coordinates, and (iv) returning the retrieved aggregated multi-dimensional data back to said database query handler, over said second data communication interface, for transmission to said client machine for servicing the database query request.
2 . The data aggregation module of claim 1 , wherein during data loading operations,
said base data loader extracts dimension data from the dictionary in said meta-data store, and forwards the dimension data over said first data communication interface to said aggregation engine and configures said MDDB using said dimension data so that each data storage cell therein is specified by a set of multi-dimensional coordinates.
3 . The data aggregation module of claim 2 , wherein during aggregation operations,
said base data loader extracts base data from said table(s), and forwards the base data to said aggregation engine over said first data communication interface; and
said aggregation engine calculates aggregated multi-dimensional data from the base data according to said multi-dimensional data aggregation process, and stores the aggregated multi-dimensional data within data storage cell(s) in said MDDB specified by a set of multi-dimensional coordinates.
4 . The data aggregation module of claim 1 , wherein said first data communication interface is a standard interface selected from the group selected from OLDB, OLE-DB, ODBC, SQL, API, and JDBC.
5 . The data aggregation module of claim 1 , wherein said second data communication interface is a standard interface selected from the group selected from OLDB, OLE-DB, ODBC, SQL, API, and JDBC.
6 . The data aggregation module of claim 1 , wherein said query interface is implemented in a module within said RDBMS.
7 . The data aggregation module of claim 1 , wherein said base data loader further includes an adapter (interface) for mapping the data types of said dictionary into the data types used in said MDD aggregation module, or for mapping a standard data type used to represent said dictionary into the data types used in said MDD aggregation module.
8 . The data aggregation module of claim 1 , wherein said database query handler further includes means for transforming each database query requests so as to optimize the database query requests for efficient database query request handling.
9 . The data aggregation module of claim 1 , wherein said multi-dimensional aggregation process supported by said aggregation engine further operates as follows:
(a) if the aggregated multi-dimensional data required to answer a given database query requests is already pre-calculated and stored within said MDDB, then the pre-aggregated multi-dimensional data is retrieved by said MDDB handler and returned to said client machine via said database query handler; and
(b) if the required multi-dimensional data is not already pre-aggregated and stored within said MDDB, then the required aggregated multi-dimensional data is calculated on demand by said aggregation engine, and the aggregated multi-dimensional data result is automatically transferred to said client machine, via said database query handler, while being simultaneously stored within said MDDB by said MDDB handler.
10 . The data aggregation module of claim 1 , wherein said first and second data communication interfaces are standard interfaces enabling cartridge-style interfacing with said RDBMS and said one or more client machines.
11 . The data aggregation module of claim 1 , wherein said client machines comprises one or more OLAP servers.