Systems and methods for hierarchical aggregation of multi-dimensional data sources
Embodiments relate to systems and methods for aggregating data from data sources according to a hierarchical mapping generated from dimensions of the data sources. A set of applications such as online analytical processing (OLAP) applications can access the combined data of a set of multi-dimensional data sources via a transform engine. The set of data sources can be configured with diverse dimensions and associated data, which in general do not reflect a strictly hierarchical structure. In embodiments, the transform engine can combine or aggregate the set of data sources using common dimensions or data points, and build an index into a transform table reflecting the hierarchical level of dimension from each data source in a combined hierarchical mapping. An OLAP or other application can therefore perform searches, sorts, and/or other operations on the combined hierarchical mapping based on the resulting ordering of data, even when the original multi-dimensional data sources do not contain an explicit common hierarchy.
1. A method comprising:
receiving, by a transform engine executed by a processor, a data request from at least one application, wherein the transform engine is in communication with a set of multi-dimensional data sources;
identifying, by the transform engine, multiple multi-dimensional data sources from the set of multi-dimensional data sources that contain data that corresponds to the data request;
extracting, by the transform engine, the data corresponding to the data request from the identified multi-dimensional data sources;
identifying, by the transform engine, multiple dimensions of the extracted data, wherein hierarchical relationships exist between ones of the identified dimensions within each of the identified multi-dimensional data sources;
examining, by the transform engine, a lookup table defining presumed relationships between the identified dimensions to generate a new hierarchical relationship between ones of the identified dimensions from separate ones of the identified multi-dimensional data sources, wherein the separate ones of the identified multi-dimensional data sources do not explicitly or implicitly include a definition of the new hierarchical relationship;
generating, by the transform engine, a hierarchical structure for the identified dimensions based on the existing hierarchical relationships and the new hierarchical relationship, wherein the hierarchical structure maps each one of the identified dimensions to one of multiple levels within the existing hierarchical relationships and the new hierarchical relationship, wherein the hierarchical structure comprises a set of ordered nodes, wherein each node in the set of ordered nodes corresponds to a dimension from the identified dimensions of the extracted data and a level from the levels in the hierarchical structure; and
aggregating, by the transform engine, the extracted data into the hierarchical structure, wherein aggregating the extracted data comprises generating a transform table that stores, for each node in the set of ordered nodes, the dimension and the level of the node.
2. The method of claim 1 , wherein the at least one application comprises multiple applications.
3. The method of claim 1 , wherein the at least one application comprises an online analytical processing (OLAP) application.
4. The method of claim 1 , wherein the set of multi-dimensional data sources comprises a set of online analytical processing (OLAP) data sources.
5. The method of claim 1 , wherein the set of multi-dimensional data sources comprises a set of structured three-dimensional data sources.
6. A system comprising:
an interface to receive a data request from at least one application, wherein the transform engine is in communication with a set of multi-dimensional data sources;
a processor to execute a transform engine, in communication with the interface, the transform engine to:
identify multiple multi-dimensional data sources from the set of multi-dimensional data sources that contain data that corresponds to the data request,
extract the data corresponding to the data request from the identified multi-dimensional data sources,
identify multiple dimensions of the extracted data, wherein hierarchical relationships exist between ones of the identified dimensions within each of the identified multi-dimensional data sources,
examine a lookup table defining presumed relationships between the identified dimensions to generate a new hierarchical relationship between ones of the identified dimensions from separate ones of the identified multi-dimensional data sources, wherein the separate ones of the identified multi-dimensional data sources do not explicitly or implicitly include a definition of the new hierarchical relationship,
generate a hierarchical structure for the identified dimensions based on the existing hierarchical relationships and the new hierarchical relationship, wherein the hierarchical structure maps each one of the identified dimensions to one of multiple levels within the existing hierarchical relationships and the new hierarchical relationship, wherein the hierarchical structure comprises a set of ordered nodes, wherein each node in the set of ordered nodes corresponds to a dimension from the identified dimensions of the extracted data and a level from the levels in the hierarchical structure, and
aggregate the extracted data into the hierarchical structure, wherein aggregating the extracted data comprises generating a transform table that stores, for each node in the set of ordered nodes, the dimension and the level of the node.
7. The system of claim 6 , wherein the at least one application comprises multiple applications.
8. The system of claim 6 , wherein the at least one application comprises an online analytical processing (OLAP) application.
9. The system of claim 6 , wherein the set of multi-dimensional data sources comprises a set of online analytical processing (OLAP) data sources.
10. The system of claim 6 , wherein the set of multi-dimensional data sources comprises a set of structured three-dimensional data sources.
11. A non-transitory computer readable medium comprising executable program code which, when executed by a processor, causes the processor to perform a method comprising:
receiving, by a transform engine executed by the processor, a data request from at least one application, wherein the transform engine is in communication with a set of multi-dimensional data sources;
identifying, by the transform engine, multiple multi-dimensional data sources from the set of multi-dimensional data sources that contain data that corresponds to the data request;
extracting, by the transform engine, the data corresponding to the data request from the identified multi-dimensional data sources;
identifying, by the transform engine, multiple dimensions of the extracted data, wherein hierarchical relationships exist between ones of the identified dimensions within each of the identified multi-dimensional data sources;
examining, by the transform engine, a lookup table defining presumed relationships between the identified dimensions, to generate a new hierarchical relationship between ones of the identified dimensions from separate ones of the identified multi-dimensional data sources, wherein the separate ones of the identified multi-dimensional data sources do not explicitly or implicitly include a definition of the new hierarchical relationship;
generating, by the transform engine, a hierarchical structure for the identified dimensions based on the existing hierarchical relationships and the new hierarchical relationship, wherein the hierarchical structure maps each one of the identified dimensions to one of multiple levels within the existing hierarchical relationships and the new hierarchical relationship, wherein the hierarchical structure comprises a set of ordered nodes, wherein each node in the set of ordered nodes corresponds to a dimension from the identified dimensions of the extracted data and a level from the levels in the hierarchical structure; and
aggregating, by the transform engine, the extracted data into the hierarchical structure, wherein aggregating the extracted data comprises generating a transform table that stores, for each node in the set of ordered nodes, the dimension and the level of the node.
12. The non-transitory computer readable medium of claim 11 , wherein the at least one application comprises an online analytical processing (OLAP) application.
13. The non-transitory computer readable medium of claim 11 , wherein the set of multi-dimensional data sources comprises a set of online analytical processing (OLAP) data sources.