Extracting facts from unstructured text
View Patent ↗A system and method for extracting facts from unstructured text files are disclosed. Embodiments of the disclosed system and method may receive a text file as input and perform extraction and disambiguation of entities, as well as extract topics and facts. The facts are extracted by comparing against a fact template store and associating facts with events or topics. The extracted facts are stored in a data store.
1. A method comprising:
receiving, by an entity extraction computer, an electronic document having unstructured text, wherein the electronic document is a text file;
extracting, by the entity extraction computer, an entity identifier from the unstructured text in the electronic document;
extracting, by a topic extraction computer, a topic identifier from the unstructured text in the electronic document;
extracting, by a fact extraction computer, a fact identifier from the unstructured text in the electronic document by comparing text string structures in the unstructured text to a fact template database, wherein the fact template database having stored therein a fact template model identifying keywords pertaining to specific fact identifiers and corresponding keyword weights; and
associating, by a fact relatedness estimator computer, the entity identifier with the topic identifier and the fact identifier to determine a confidence score indicative of a degree of accuracy of extraction of the fact identifier, wherein the confidence score is based at least in part on a spatial distance between a part of the unstructured text in the electronic document from where the fact identifier was extracted and a part of the unstructured text from where at least one of the topic identifier or the entity identifier was extracted.
2. The method of claim 1 , wherein the distance in the unstructured text is calculated using tokenization.
3. The method of claim 1 , wherein the confidence score is further based at least in part on comparing co-occurring entity identifiers in the electronic document.
4. The method of claim 1 wherein the fact template model includes metadata.
5. The method of claim 4 wherein the metadata includes a count of a number of times a sentence structure corresponding to the fact template model is repeated across a plurality of electronic documents.
6. The method of claim 4 wherein the confidence score is stored in the metadata.
7. A system comprising:
one or more server computers having one or more processors executing computer readable instructions for a plurality of computer modules including:
an entity extraction module which receives an electronic document having unstructured text and extracts an entity identifier from the unstructured text in the electronic document, wherein the electronic document is a text file;
a topic extraction module which extracts a topic identifier from the unstructured text in the electronic document;
a fact extraction module which extracts a fact identifier from the unstructured text in the electronic document by comparing text string structures in the unstructured text to a fact template database, wherein the fact template database having stored therein a fact template model identifying keywords pertaining to specific fact identifiers and corresponding keyword weights; and
a fact relatedness estimator module which associates the entity identifier with the topic identifier and the fact identifier to determine a confidence score indicative of a degree of accuracy of extraction of the fact identifier, wherein the confidence score is based at least in part on a spatial distance between a part of the unstructured text in the electronic document from where the fact identifier was extracted and a part of the unstructured text from where at least one of the topic identifier or the entity identifier was extracted.
8. The system of claim 7 , wherein the distance in the unstructured text is calculated using tokenization.
9. The system of claim 7 , wherein the confidence score is further based at least in part on comparing co-occurring entity identifiers in the electronic document.
10. The system of claim 7 wherein the fact template model includes metadata.
11. The system of claim 10 wherein the metadata includes a count of a number of times a sentence structure corresponding to the fact template model is repeated across a plurality of electronic documents.
12. The system of claim 10 wherein the confidence score is stored in the metadata.
13. A non-transitory computer readable medium having stored thereon computer executable instructions instructive of a method comprising:
receiving, by an entity extraction computer, an electronic document having unstructured text, wherein the electronic document is a text file;
extracting, by the entity extraction computer, an entity identifier from the unstructured text in the electronic document;
extracting, by a topic extraction computer, a topic identifier from the unstructured text in the electronic document;
extracting, by a fact extraction computer, a fact identifier from the unstructured text in the electronic document by comparing text string structures in the unstructured text to a fact template database, the fact template database having stored therein a fact template model identifying keywords pertaining to specific fact identifiers and corresponding keyword weights; and
associating, by a fact relatedness estimator computer, the entity identifier with the topic identifier and the fact identifier to determine a confidence score indicative of a degree of accuracy of extraction of the fact identifier, wherein the confidence score is based at least in part on a spatial distance between a part of the unstructured text in the electronic document from where the fact identifier was extracted and a part of the unstructured text from where at least one of the topic identifier or the entity identifier was extracted.
14. The non-transitory computer readable medium of claim 13 , wherein the distance in the unstructured text is calculated using tokenization.
15. The non-transitory computer readable medium of claim 13 , wherein the confidence score is further based at least in part on comparing co-occurring entity identifiers in the electronic document.
16. The non-transitory computer readable medium of claim 13 wherein the fact template model includes metadata.
17. The non-transitory computer readable medium of claim 16 wherein the metadata includes a count of a number of times a sentence structure corresponding to the fact template model is repeated across a plurality of electronic documents.