Method of bibliographic field normalization
A method of normalizing a bibliographic field of a structured field relational database is disclosed. The method comprises weighting potential candidate records according to the value in the corresponding field in the records, together with other related fields in the candidate record and other related records in the database. Each of the candidate records is successively evaluated and compared against an acceptable threshold. If the weight exceeds the threshold, the candidate record is returned from the query. Otherwise, a new entry in the database is created. Optionally, before creating such a new entry, the highest weighted candidate record may be compared against a minimally acceptable threshold and if the weight exceeds such lower threshold, the candidate is returned from the query.
1. A method of normalizing a first field in a structured relational database, wherein the structured relational database has a plurality of records, the method being performed by a computer and comprising:
a) identifying a candidate record from a set of records taken from the plurality of records, each of the records in the set of records comprising at least the first field; and
b) for one or more other records in the set of records,
i) generating a similarity weight corresponding to a similarity of a candidate value in the first field of the candidate record to a corresponding value in the first field of the other record, the similarity weight reflecting one or more of:
similarities of the candidate value and the corresponding value;
similarities of values in one or more other fields of the candidate record to respective values in corresponding one or more other fields of the other record; and
similarities of values in one or more fields of records that are related to the candidate record according to one or more relationships in the structured relational database to respective values in corresponding one or more fields in records that are related to the other record according to a same one or more relationships; and
ii) comparing the similarity weight against an acceptable threshold value and when the similarity weight is at least equal to the acceptable threshold value, reconciling the candidate value in the candidate record identified in step (a) and the corresponding value in the same field of the other record; and
c) upon each reconciling of the one or more candidate values, iteratively repeating step
b) for each of:
any values in one or more other fields of the candidate record having a similarity to the respective value in the corresponding one or more other fields as previously determined in step b)i); and
any values in one or more fields of records that are related to the candidate record according to one or more relationships in the structured relational database having a similarity to respective values in corresponding one or more fields in records that are related to the other record according to the same one or more relationships as previously determined in step b)i).
2. The method as defined in claim 1 , wherein the first field is a bibliographic field of the structured relational database.
3. The method as defined in claim 1 , wherein the structured relational database contains patent records.
4. A non-transitory computer-readable storage medium having computer-executable instructions that, when executed by a computer, cause the computer to perform a method of normalizing a first field in a structured relational database, wherein the structured relational database has a plurality of records, the method comprising:
a) identifying a candidate record from a set of records taken from the plurality of records, each of the records in the set of records comprising at least the first field; and
b) for one or more other records in the set of records,
i) generating a similarity weight corresponding to a similarity of a candidate value in the first field of the candidate record to a corresponding value in the first field of the other record, the similarity weight reflecting one or more of:
similarities of the candidate value and the corresponding value;
similarities of values in one or more other fields of the candidate record to respective values in corresponding one or more other fields of the other record; and
similarities of values in one or more fields of records that are related to the candidate record according to one or more relationships in the structured relational database to respective values in corresponding one or more fields in records that are related to the other record according to a same one or more relationships; and
ii) comparing the similarity weight against an acceptable threshold value and when the similarity weight is at least equal to the acceptable threshold value, reconciling the candidate value in the candidate record identified in step (a) and the corresponding value in the same field of the other record; and
c) upon each reconciling of the one or more candidate values, iteratively repeating step
b) for each of:
any values in one or more other fields of the candidate record having a similarity to the respective value in the corresponding one or more other fields as previously determined in step b)i); and
any values one or more fields of records that are related to the candidate record according to one or more relationships in the structured relational database having a similarity to respective values in corresponding one or more fields in records that are related to the other record according to the same one or more relationships as previously determined in step b)i).
5. The computer-readable storage medium as defined in claim 4 , wherein the first field is a bibliographic field of the structured relational database.
6. The computer-readable storage medium as defined in claim 4 , wherein the structured relational database contains patent records.