Data warehouse model validation
View Patent ↗This invention relates to a system, method and computer program product for a data warehouse model validation system, said data warehouse model validation system having an ETL model and a corresponding data warehouse model, said data ETL system comprising: an element group locator for locating an element group across the ETL model and the data warehouse model, whereby the element group comprises ETL elements and related data warehouse elements; an inconsistency determiner for determining inconsistencies between the ETL elements and data warehouse elements, whereby one or more elements are missing from the data warehouse model or one or more elements in the data warehouse model do not correspond to expected elements or features of elements; and an inconsistency recorder for recording any located missing elements or unexpected elements from the located element group.
1. A data warehouse model validation system comprising a processor and a memory, the memory storing program instructions which, when executed by the processor, cause the processor to:
locate, by an element group locator, at least two element groups across an extract transform load (ETL) model and a corresponding data warehouse model, wherein the at least two element groups include ETL elements and related data warehouse elements;
determine, by an inconsistency determiner, inconsistencies, for the located at least two element groups, between an ETL system having the ETL elements and a data warehouse having the related data warehouse elements, wherein the inconsistencies include an ETL element in a staging area of the ETL system being missing from the data warehouse, an ETL element in a source system being missing from the staging area or the data warehouse, or a related data warehouse element being missing from the staging area or a source system;
record, by an inconsistency recorder, a missing ETL element or a related data warehouse element from the located at least two element groups; and
prioritize, by a prioritization engine, the located at least two element groups by counting a number of differences between each of the located at least two element groups.
2. The data warehouse model validation system according to claim 1 , further comprising program instructions which, when executed by the processor, cause the processor to validate, by a validator, the data warehouse model for the located at least two located element groups when there is nothing unexpected in or nothing missing from the located at least two element groups.
3. The data warehouse model validation system according to claim 1 , further comprising program instructions which, when executed by the processor, cause the processor to recommend, by a recommender, an improvement to fix the missing the ETL elements or the related data warehouse elements or the ETL model when there are missing the ETL elements or the related data warehouse elements in the located at least two element groups.
4. The data warehouse model validation system according to claim 1 , further comprising program instructions which, when executed by the processor, cause the processor to effect, by a fixing engine, improvements to the missing the ETL elements or the related data warehouse elements or the ETL model when there are missing the ETL elements or the related data warehouse elements in the located at least two element groups.
5. The data warehouse model validation system according to claim 1 , wherein the number of expected and unexpected the ETL elements or the related data warehouse elements in one of the located at least two element groups are counted as differences.
6. The data warehouse model validation system according to claim 1 , wherein the number of differences includes differences between sub-elements of the ETL elements or the related data warehouse elements.
7. The data warehouse model validation system according to claim 1 , wherein the ETL model comprises:
a source system data models; and
a ETL staging data models.
8. The data warehouse model validation system according to claim 1 , wherein the ETL elements or the related data warehouse elements comprise data model elements and operational data.
9. A computer program product for a data warehouse model validation system having an extract transform load (ETL) model and a corresponding data warehouse model, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable by a processor to cause the processor to:
locate two or more element groups across the ETL model and the data warehouse model, wherein the located two or more element groups include ETL elements and mapped data warehouse elements;
determine inconsistencies between the ETL elements and mapped data warehouse elements within the located two or more element groups, wherein the inconsistencies include: one or more the ETL elements or mapped data warehouse elements are missing from the data warehouse model, or one or more the ETL elements or mapped data warehouse elements in the data warehouse model do not correspond to the ETL elements or mapped data warehouse elements or features of the ETL elements or mapped data warehouse elements;
record located missing the ETL elements or mapped data warehouse elements or the ETL elements or mapped data warehouse elements from the located two or more located element groups; and
prioritize, by a prioritization engine, the located two or more element groups by counting a number of differences between each of the located two or more two element groups.