Propagation of annotation metadata to overlapping annotations of synonymous type
Aspects of the invention include systems and methods for the propagation of annotation metadata to overlapping annotations of a synonymous type. A non-limiting example computer-implemented method includes performing a comparison of a set of annotations to detect a subset of annotations that are candidates of being synonymous based on a first analysis. Whether a first annotation of the subset of annotations is synonymous with a second annotation of the subset of annotations is determined based on a second analysis. Distinct annotation metadata of the first annotation are cross-propogated with annotation metadata of the second annotation based on the second analysis.
1. A computer-implemented method comprising:
converting, by a processor, a document to plain text format;
performing, by the processor, a comparison of a set of annotations, from the converted document in plain text format, to detect a subset of annotations that are candidates of being synonymous based on a first analysis;
determining, by the processor, a first annotation of the subset of annotations is an aggregate of one or more tokens;
dividing, by the processor, the first annotation into individual tokens; and
determining, by the processor, whether each individual token is respectively synonymous with the second annotation of the subset of annotations based on a second analysis; and
cross-propagating, by the processor, distinct annotation metadata of the first annotation with annotation metadata of the second annotation based on the second analysis.
2. The computer-implemented method of claim 1 , wherein the first analysis is a pairwise comparison.
3. The computer-implemented method of claim 1 , wherein the second analysis is a word vector comparison.
4. The computer-implemented method of claim 1 , wherein each annotation of the set of annotations is generated by a respective annotator.
5. The computer-implemented method of claim 4 , wherein each respective annotator is based on a respective model.
6. The computer-implemented method of claim 1 , wherein the set of annotations are derived from a text span of unstructured data.
7. A system comprising:
a memory having computer readable instructions; and
one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:
converting a document to plain text format;
performing a comparison of a set of annotations, from the converted document in plain text format, to detect a subset of annotations that are candidates of being synonymous based on a first analysis;
determining a first annotation of the subset of annotations is an aggregate of one or more tokens;
dividing the first annotation into individual tokens; and
determining whether each individual token is respectively synonymous with a second annotation of the subset of annotations based on a second analysis; and
cross-propagating distinct annotation metadata of the first annotation with annotation metadata of second annotation based on the second analysis.
8. The system of claim 7 , wherein the first analysis is a pairwise comparison.
9. The system of claim 7 , wherein the second analysis is a word vector comparison.
10. The system of claim 7 , wherein each annotation of the set of annotations is generated by a respective annotator.
11. The system of claim 10 , wherein each respective annotator is based on a respective model.
12. The system of claim 7 , wherein the set of annotations are derived from a text span of unstructured data.
13. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:
converting a document to plain text format;
performing a comparison of a set of annotations to detect a subset of annotations, from the converted document in plain text format, that are candidates of being synonymous based on a first analysis;
determining a first annotation of the subset of annotations is an aggregate of one or more tokens;
dividing the first annotation into individual tokens; and
determining whether each individual token is respectively synonymous with a second annotation of the subset of annotations based on a second analysis; and
cross-propagating distinct annotation metadata of the first annotation with annotation metadata of second annotation based on the second analysis.
14. The computer program product of claim 13 , wherein the first analysis is a pairwise comparison.
15. The computer program product of claim 13 , wherein the second analysis is a word vector comparison.
16. The computer program product of claim 13 , wherein each annotation of the set of annotations is generated by a respective annotator.
17. The computer program product of claim 16 , wherein each respective annotator is based on a respective model.