System and method for data quality monitors
Systems and methods are presented for data quality monitoring. Data quality monitors may be created and configured to identify objects with specified data quality issues and/or property values. Objects identified by a data quality monitor can be presented to users for confirmation and resolution. Properties used by the data quality monitor to match objects may also be displayed to users.
1. A computer-implemented method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising:
receiving, via a user interface, selection, from a plurality of object types including entity and event, a first object type and a related object type different than the first object type;
receiving, via the user interface, selection of a criterion selected from a plurality of criterion that are selectable via the user interface, wherein the selected criterion is configured to identify data objects of the first object type associated with a data object of the related object type and having a potential problem;
identifying, based on evaluation of at least the selected criterion with reference to a plurality of data objects stored in one or more data stores:
a first data object of the first object type, the first data object having a plurality of properties, wherein the first data object is identified based on satisfaction of the selected criterion by one or more properties of the first data object; and
a second data object of the related object type having a link to the first data object; and
causing presentation, in the user interface, of a graphical representation of the first and second data objects as visually linked.
2. The computer-implemented method of claim 1 , wherein the potential problem comprises at least one of: possible duplicate objects, missing properties, multi-valued properties, unparsed properties, disallowed enumerations, numeric range violations, or date range violations.
3. The computer-implemented method of claim 1 , wherein the criterion comprises a value for a property.
4. A non-transitory computer storage medium storing computer executable instructions that when executed by a computer hardware processor perform operations comprising:
receiving, via a user interface, selection, from a plurality of object types including entity and event, a first object type and a related object type different than the first object type;
receiving, via the user interface, selection of a criterion selected from a plurality of criterion that are selectable via the user interface, wherein the selected criterion is configured to identify data objects of the first object type associated with a data object of the related object type and having a potential problem;
identifying, based on evaluation of at least the selected criterion with reference to a plurality of data objects stored in one or more data stores:
a first data object of the first object type, the first data object having a plurality of properties, wherein the first data object is identified based on satisfaction of the selected criterion by one or more properties of the first data object; and
a second data object of the related object type having a link to the first data object; and
causing presentation, in the user interface, of a graphical representation of the first and second data objects as visually linked.
5. The non-transitory computer storage medium of claim 4 , wherein determining the second data object further comprises identifying a first property of the first data object that matches a second property of the second data object.
6. The non-transitory computer storage medium of claim 5 , wherein the potential problem comprises a possible duplicate object, and the operations further comprise:
receiving a user selection to resolve the potential problem;
generating a merged data object including a property from at least one of the first or second data objects;
removing at least one of the first or second data objects from a data store; and
storing the merged data object in the data store.
7. The non-transitory computer storage medium of claim 4 , wherein the operations further comprise:
generating a data quality monitor including the first object type and the criterion; and
storing the data quality monitor in a data store, wherein the data quality monitor is executable to identify potential data problems.
8. A computer system comprising:
one or more hardware computer processors programmed, via executable code instructions, to:
receiving, via a user interface, selection, from a plurality of object types including entity and event, a first object type and related object type different than the first object type;
receiving, via the user interface, selection of a criterion selected from a plurality of criterion that are selectable via the user interface, wherein the selected criterion is configured to identify data objects of the first object type associated with a data object of the related object type and having a potential problem;
identifying, based on evaluation of at least the selected criterion with reference to a plurality of data objects stored in one or more data stores;
a first data object of the first object type, the first data object having a plurality of properties, wherein the first data object is identified based on satisfaction of the selected criterion by one or more properties of the first data object; and
a second data object of the related object type having a link to the first data object; and
causing presentation, in the user interface, of a graphical, representation of the first and second data objects as visually linked.
9. The computer system of claim 8 , wherein identifying the first data object of the first object type with the potential problem further comprises matching the criterion with a property of the first data object.
10. The computer system of claim 8 , wherein determining the second data object further comprises identifying a first property of the first data object that matches a second property of the second data object.
11. The computer system of claim 8 , wherein the potential problem comprises a possible duplicate object, and the one or more hardware computer processors are further programmed to:
receive a user selection to resolve the potential problem; and
generate a merged data object including a property from at least one of the first or second data objects.
12. The computer system of claim 11 , wherein the one or more hardware computer processors are further programmed to:
remove at least one of the first or second data objects from a data store.
13. The computer system of claim 11 , wherein the one or more hardware computer processors are further programmed to:
store the merged data object in a data store.
14. The computer system of claim 10 , wherein the one or more hardware computer processors are further programmed to:
generate a data quality monitor including the first object type and the criterion; and
store the data quality monitor in a data store, wherein the data quality monitor is executable to identify potential problems.