System and method for data quality assessment
Methods and systems for providing data assessment across related datasets, including identifying exceptional values in datasets and assessing upstream and or downstream datasets that utilize the exceptional values. Data assessment rules use exceptional values found in a dataset and data lineage information to identify impacted data in upstream or downstream datasets.
1 . A method for assessing data stored in at least two datasets, comprising:
assessing one or more first attributes in one or more first records stored in a first dataset;
identifying a first attribute with an exceptional value;
storing a first entry in an exception table comprising a first object identifier value identifying the first dataset, a first record identifier identifying the first record containing the first attribute with an exceptional value, a first attribute identifier identifying the first attribute with an exceptional value, and a first exception identifier associated with the first entry;
identifying, in at least one second dataset, a second record containing a second attribute that utilizes the first attribute with the exceptional value;
storing a second entry in the exception table comprising a second record identifier identifying the second record; and
linking the second entry to the first entry using the first exception identifier.
2 . The method of claim 1 , wherein the step of identifying, in at least one second dataset, a second record further comprises searching the second dataset for a record containing at least one of the first record identifier, the first object identifier, and the first attribute identifier.
3 . The method of claim 2 , wherein the step of identifying, in at least one second dataset, a second record, further comprises identifying a match between the first record identifier in the first entry in the exception table to a second record identifier in the second record.
4 . The method of claim 1 , wherein the step of identifying, in at least one second dataset, a second record further comprises identifying a transform function that utilizes the first attribute with the exceptional value to create the second attribute contained in the at least one second record.
5 . The method of claim 1 , further comprising identifying, in at least one third database, a third record that utilizes at least one of the first attribute with the exceptional value and the second attribute.
6 . The method of claim 1 , wherein the step of storing a second entry in the exception table further comprises storing a second attribute identifier identifying the second attribute and a second object identifier identifying the second dataset.
7 . The method of claim 1 , further comprising the step of communicating the second record identifier to an entity able to adjust a downstream process utilizing the second record.