Data aggregation module supporting dynamic query responsive aggregation during the servicing of database query requests provided by one or more client machines
View Patent ↗Improved method of and apparatus for aggregating data elements in multidimensional databases (MDDB). In the preferred embodiment, the apparatus is realized in the form of a high-performance stand-alone (i.e. external) aggregation server which can be plugged-into conventional MOLAP systems to achieve significant improvements in system performance. In accordance with the principles of the present invention, the stand-alone aggregation server contains a scalable MDDB and a high-performance aggregation engine that are integrated into the modular architecture of the aggregation server. The stand-alone aggregation server of the present invention can uniformly distribute data elements among a plurality of processors, for balanced loading and processing, and therefore is highly scalable.
1. A method comprising:
by a computer system comprising computer hardware:
receiving a first multidimensional cube of data, wherein the first multidimensional cube of data is sparse;
initiating a first aggregation process;
conducting the first aggregation process:
receiving a query after the first aggregation process has been initiated and while the first aggregation process is being conducted;
determining a sequence for storing the first multidimensional cube of data in a multidimensional database; and
at least partly in response to the query, performing a query directed, on-the-fly roll-up process while the first aggregation process is still being conducted,
extracting, based at least in part on the determined sequence, a first slice of data for storage in the multidimensional database from the first multidimensional cube of data,
wherein the first slice of data is a second multidimensional cube of data that is a subset of the first multidimensional cube of data.
2. The method of claim 1 , wherein a plurality of non-null data points are stored in a first record in the multidimensional database indexed in an ascending order along an aggregation direction.
3. The method of claim 2 , further comprising extracting, based at least in part on the determined sequence, a second slice of data for storage from the first multidimensional cube of data.
4. The method of claim 3 , wherein non-null data points within the second slice are stored in a second record in the multidimensional database.
5. The method of claim 4 , wherein a length of the first record and second record are different and based at least in part on the number of non-null data points that were stored.
6. A computing system comprising:
a processor comprising hardware; and
a computer readable medium storing machine-executable instructions including one or more modules configured for execution by the processor in order to cause the computing system to:
receive a first multidimensional cube of data, wherein the first multidimensional cube of data is sparse;
initiate a first on-the-fly aggregation process;
conduct the first on-the-fly aggregation process:
determine a sequence for storing the first multidimensional cube of data in a multidimensional database;
receive a query after the first on-the-fly aggregation process has been initiated and while the first a on-the-fly aggregation process is being conducted;
determine a sequence for storing the first multidimensional cube of data in a multidimensional database;
at least partly in response to the received query, perform a query directed roll-up process while the first on-the-fly aggregation process is still being conducted; and
extract, based at least in part on the determined sequence, a first slice of data from the first multidimensional cube of data, wherein the first slice of data is a second multidimensional cube of data that is a subset of the first multidimensional cube of data; and
store the extracted first slice of data in the multidimensional database.
7. The system of claim 6 , wherein a plurality of non-null data points are stored in a first record in the multidimensional database indexed in an ascending order along an aggregation direction.
8. The system of claim 7 , wherein the instructions are further configured to cause the computing system to extract, based at least in part on the determined sequence, a second slice of data for storage from the first multidimensional cube of data.
9. The system of claim 8 , wherein non-null data points within the second slice are stored in a second record in the multidimensional database.
10. The system of claim 9 , wherein a length of the first record and second record are different and based at least in part on the number of non-null data points that were stored.
11. A non-transitory computer-readable medium encoded with instructions thereon, wherein the instructions are readable by a computing system in order to cause the computing system to perform operations comprising:
receiving a first multidimensional cube of data, wherein the first multidimensional cube of data is sparse;
initiating a first aggregation process;
conducting the first on-the-fly aggregation process:
determining a sequence for storing the first multidimensional cube of data in a multidimensional database;
receiving a query after the first on-the-fly aggregation process has been initiated and while the first aggregation process is being conducted;
at least partly in response to the received query, performing a query directed roll-up process while the first on-the-fly aggregation process is still being conducted;
extracting, based at least in part on the determined sequence, a first slice of data from the first multidimensional cube of data,
wherein the first slice of data is a second multidimensional cube of data that is a subset of the first multidimensional cube of data; and
storing the extracted first slice of data in the multidimensional database.
12. The non-transitory computer-readable medium of claim 11 , wherein a plurality of non-null data points are stored in a first record in the multidimensional database indexed in an ascending order along an aggregation direction.
13. The non-transitory computer-readable medium of claim 12 , wherein the instructions are further encoded to cause the computing system to perform operations comprising extracting, based at least in part on the determined sequence, a second slice of data for storage from the first multidimensional cube of data.
14. The non-transitory computer-readable medium of claim 13 , wherein non-null data points within the second slice are stored in a second record in the multidimensional database.
15. The non-transitory computer-readable medium of claim 14 , wherein a length of the first record and second record are different and based at least in part on the number of non-null data points that were stored.