Detecting inconsistent data records
A computer-implemented method for detecting a set of inconsistent data records in a database including multiple records, comprises selecting a data quality rule representing a functional dependency for the database, transforming the data quality rule into at least one rule vector with hashed components, selecting a set of attributes of the database, transforming at least one record of the database selected on the basis of the selected attributes into a record vector with hashed components, computing a dot product of the rule and record vectors to generate a measure representing violation of the data quality rule by the record.
1. A computer-implemented method for detecting a set of inconsistent data records in a database including multiple records, comprising:
selecting a data quality rule representing a functional dependency for the database;
transforming the data quality rule into at least one rule vector with hashed components,
wherein transforming the data quality rule includes generating at least a pair of vectors,
wherein the at least one rule vector is determined by concatenating left and right hand side components of the data quality rule,
wherein the data quality rule is a conditional functional dependency (CFD) representing a functional dependency of the database, and wherein the CFD is a constant or variable CFD including rule attributes which are constants or variables, and wherein a pair of records of the database is transformed into the record vector;
selecting a set of attributes of the database;
transforming at least one record of the database selected on the basis of the selected attributes into a record vector with hashed components wherein the hashed components of the at least one rule vector and the record vector prepare the vectors for computing a secured dot product;
computing a dot product of the rule and record vectors to generate a measure representing violation of the data quality rule by the record, wherein the measure representing violation of the data quality rule by the record is provided only to the owner of the database; and
detecting the set of inconsistent data records in the database using the generated measure.
2. The computer-implemented method as claimed in claim 1 , wherein hashed components of the vectors are fixed-size hashcodes.
3. The computer-implemented method as claimed in claim 1 , wherein the data quality rule is a conditional functional dependency (CFD) representing a functional dependency of the database which is extended with a pattern tableau specifying conditions under which the functional dependency holds for records of the database.
4. The computer-implemented method as claimed in claim 1 , wherein the data quality rule is a conditional functional dependency (CFD) representing a functional dependency of the database which is extended with a pattern tableau specifying conditions under which the functional dependency holds for records of the database, and wherein the CFD is a constant CFD including rule attributes which are constants.
5. The computer-implemented method as claimed in claim 1 , wherein the data quality rule is a conditional functional dependency (CFD) representing a functional dependency of the database which is extended with a pattern tableau specifying conditions under which the functional dependency holds for records of the database, and wherein the CFD is a variable CFD including rule attributes which are variable.
6. The computer-implemented method as claimed in claim 1 , wherein the data quality rule is a conditional functional dependency (CFD) representing a functional dependency of the database which is extended with a pattern tableau specifying conditions under which the functional dependency holds for records of the database, and wherein the record includes an attribute matching a corresponding determinant attribute for the CFD.
7. The computer-implemented method as claimed in claim 1 , wherein the data quality rule is a conditional functional dependency (CFD) representing a functional dependency of the database which is extended with a pattern tableau specifying conditions under which the functional dependency holds for records of the database, and wherein the record includes an attribute matching a corresponding determinant attribute for the CFD, and wherein a violation occurs if there is a disagreement between a dependent attribute of the record and the corresponding attribute of the CFD.
8. The computer-implemented method as claimed in claim 1 , wherein the privacy of a data quality rule is preserved in the case where it is not violated by the records.
9. The computer-implemented method as claimed in claim 1 , wherein transforming the data quality rule includes generating a pair of vectors for a rule representing components for the left and right hand sides of the rule, and wherein the pair of vectors for a data quality rule which is a variable conditional functional dependency are concatenated to form a single vector.
10. A computer program embedded on a non-transitory tangible computer readable storage medium, the computer program including machine readable instructions that, when executed by a processor, implement a method for detecting a set of inconsistent data records in a database including multiple records, comprising:
selecting a data quality rule representing a functional dependency for the database;
transforming the data quality rule into at least one rule vector with hashed components,
wherein transforming the data quality rule includes generating at least a pair of vectors,
wherein the at least one rule vector is determined by concatenating left and right hand side components of the data quality rule,
wherein the data quality rule is a conditional functional dependency (CFD) representing a functional dependency of the database, and wherein the CFD is a constant or variable CFD including rule attributes which are constants or variables, and wherein a pair of records of the database is transformed into the record vector;
selecting a set of attributes of the database;
transforming at least one record of the database selected on the basis of the selected attributes into a record vector with hashed components, wherein the hashed components of the at least one rule vector and the record vector prepare the vectors for computing a secured dot product;
computing a dot product of the rule and record vectors to generate a measure representing violation of the data quality rule by the record, wherein the measure representing violation of the data quality rule by the record is provided only to the owner of the database; and
detecting the set of inconsistent data records in the database using the generated measure.
11. The computer program embedded on the non-transitory tangible computer readable storage medium as claimed in claim 10 , the computer program including machine readable instructions that, when executed by a processor, implement a method for detecting a set of inconsistent data records in a database including multiple records, wherein:
wherein transforming the data quality rule includes generating a pair of vectors for a rule representing components for the left and right hand sides of the rule, and wherein the pair of vectors for a data quality rule which is a variable conditional functional dependency are concatenated to form a single vector.