Holistic database record repair
A computer implemented method for repairing records of a database, comprises determining a first set of records of the database which violate a functional dependency of the database, determining a second set of records of the database comprising duplicate records, computing a cost metric representing a measure for the cost of mutually dependently modifying records in the first and second sets, modifying records in the first and second sets on the basis of the cost metric to provide a modified database instance.
1. A computer implemented method for repairing records of a database, comprising:
determining a first set of records of the database that violate a functional dependency from a set of functional dependencies;
determining a second set of records of the database using a duplication mechanism wherein the second set of records are duplicate records;
appending a duplicate identifier to each record in the second set of records, wherein the duplicate identifier is identical for each record in the second set of records;
updating the set of functional dependencies to union a functional dependency based on the duplicate identifier;
determining a set of equivalence classes for records of the first set of records and the second set of records consisting of multiple record-attribute pairs;
computing a cost metric representing a measure for the cost of modifying records in the first and second sets;
merging a pair of equivalence classes of the first set of records and the second set of records into a new class to resolve a functional dependency violation and to perform a duplication of duplicate records;
computing a merge cost metric of the merged pair of equivalence classes using the cost metric of each respective class; and
modifying records in the first and second sets on the basis of the merge cost metric to provide a modified database instance.
2. A method as claimed in claim 1 , further comprising determining a set of equivalence classes for records of the first and second sets consisting of multiple record-attribute pairs, wherein attribute values for records in respective ones of the equivalence classes are the same in the modified database instance.
3. A method as claimed in claim 1 , further comprising:
refreshing the first set of records of the database which violate a functional dependency of the database; and
refreshing the second set of records of the database comprising duplicate records as a result of the step of merging.
4. A computer program embedded on a non-transitory tangible computer readable storage medium, the computer program including machine readable instructions that, when executed by a processor, implement a method for updating a database comprising:
determining a first set of records of the database which violate a functional dependency of the database from a set of functional dependencies;
determining a second set of records of the database using a duplication mechanism wherein the second set of records are duplicate records
appending a duplicate identifier to each record in the second set of records, wherein the duplicate identifier is identical for each record in the second set of records;
updating the set of functional dependencies to union a functional dependency based on the duplicate identifier;
determining a set of equivalence classes for records of the first set of records and the second set of records consisting of multiple record-attribute pairs and instructions that, when executed by the processor, implement a method for updating a database further comprising determining a cost metric representing a measure for the cost of modifying records in the first and second sets;
merging a pair of equivalence classes of the first set of records and second set of records into a new class to resolve a functional dependency violation and to perform a deduplication of duplicate records;
computing a merge cost metric of the merged pair of equivalence classes using the cost metric of each respective class; and
modifying records in the first and second sets on the basis of the cost metric to provide a modified database instance.
5. The computer program embedded on a non-transitory tangible computer readable storage medium as claimed in claim 4 , further comprising instructions that, when executed by the processor, implement a method for updating a database further comprising:
determining duplicate records using a duplication detector to group duplicate records into respective clusters, wherein records within respective ones of the clusters represent the same entity.
6. The computer program embedded on a non-transitory tangible computer readable storage medium as claimed in claim 4 , further comprising instructions that, when executed by the processor, implement a method for updating a database further comprising determining a set of equivalence classes for records of the first and second sets consisting of multiple record-attribute pairs, and instructions that, when executed by the processor, implement a method for updating a database wherein attribute values for records in respective ones of the equivalence classes are the same in the modified database instance.
7. The computer program embedded on a non-transitory tangible computer readable storage medium as claimed in claim 4 , further comprising instructions that, when executed by the processor,
refresh the first set of records of the database which violate a functional dependency of the database; and
refresh the second set of records of the database comprising duplicate records as a result of the step of merging.