Method of assessing the validity of factual claims
The method of assessing the validity of factual claims determines if a purported factual claim, such as a statement made on social media, in the news, etc., is “true”, “false” or “unknown” through comparison with corresponding ranked factual claims stored in a comparison database. The comparison database is assembled by storing a plurality of sets of digital content items, where each of the sets of digital content items has a unique topic. Factual claims within each set are ranked by the number of agreements between individual items within each set. A query item to be assessed is input, and at least one purported factual claim to be assessed is extracted. This purported factual claim is then compared against a predetermined number of highest-ranking factual claims within the corresponding set. A validity score is then generated based on the number of agreements and the number of disagreements from the comparison.
1. A method of assessing the validity of factual claims, comprising the steps of:
assembling a comparison database, wherein the step of assembling the comparison database comprises:
storing a plurality of sets of digital content items in computer readable memory associated with the comparison database, wherein each of the sets of digital content items has a unique topic associated therewith;
extracting at least one factual claim from each of the digital content items within each of the sets of digital content items;
comparing the at least one factual claim of each of the digital content items within each of the sets of digital content items with one or more of the factual claims corresponding to other ones of the digital content items within the same set of digital content items to determine a number of agreements for each of the factual claims within each of the sets of digital content items; and
ranking the factual claims within each of the sets of digital content items by the number of agreements associated therewith;
extracting at least one purported factual claim to be assessed from a query item;
determining at least one query topic associated with the at least one purported factual claim;
comparing the at least one purported factual claim with a predetermined number of highest-ranking factual claims within at least one of the sets of digital content items corresponding to the at least one query topic to determine a number of query agreements and a number of query disagreements associated with the at least one purported factual claim;
assigning a validity score to the at least one purported factual claim based on the number of query agreements and the number of query disagreements; and
displaying the validity score to a user.
2. The method of assessing the validity of factual claims as recited in claim 1 , wherein the number of agreements for each of the factual claims is based on similarity between the factual claims.
3. The method of assessing the validity of factual claims as recited in claim 1 , further comprising the steps of:
adding at least one new digital content item to at least one of the sets of digital content items;
extracting at least one factual claim from the at least one new digital content item within each of the corresponding sets of digital content items;
comparing the at least one factual claim of the at least one new digital content item within each of the corresponding sets of digital content items with one or more of the factual claims corresponding to other ones of the digital content items within the same set of digital content items to determine a number of agreements for each of the factual claims within each of the corresponding sets of digital content items; and
re-ranking the factual claims within each of the corresponding sets of digital content items by the number of agreements associated therewith.
4. The method of assessing the validity of factual claims as recited in claim 3 , wherein the number of agreements for each of the factual claims is based on similarity between the factual claims.
5. The method of assessing the validity of factual claims as recited in claim 1 , further comprising the steps of:
assigning a textual ranking of true to the at least one purported factual claim when the validity score is within a first predetermined range, assigning a textual ranking of false to the at least one purported factual claim when the validity score is within a second predetermined range, and assigning a textual ranking of unknown to the at least one purported factual claim when the validity score is within a third predetermined range; and
displaying the textual ranking to the user.
6. The method of assessing the validity of factual claims as recited in claim 1 , wherein the step of assembling the comparison database further comprises:
storing a plurality of temporary sets of digital content items in computer readable memory associated with a temporary database, wherein each of the temporary sets of digital content has a unique temporary topic associated therewith;
updating the plurality of temporary sets of digital content items by adding newly-collected digital content items; and
merging each of the temporary sets of digital content items with corresponding ones of the plurality of sets of digital content items in the comparison database when a predetermined number of digital content items is stored in the respective one of the temporary sets.
7. A method of assessing consensus between documents, comprising the steps of:
locating online documents and storing a location associated with each of the located online documents in a first index;
downloading data content associated with each of the locations stored in the first index to generate a set of first data elements;
parsing each of the first data elements and generating metadata associated therewith to generate a set of second data elements;
applying information extraction on each of the second data elements to identify subject, predicate, and object spans associated therewith;
storing the set of second data elements in a second index;
cross-referencing each of the second data elements with other ones of the second data elements based on the identified subject, predicate, and object spans using textual entailment;
generating a consensus ranking for each of the second data elements based on levels of entailment and contradiction with the other ones of the second data elements; and
transferring the set of second data elements to a third index after a pre-set threshold number of consensus rankings have been generated.
8. The method of assessing consensus between documents as recited in claim 7 , wherein the step of generating metadata associated with each of the first data elements is a process selected from the group consisting of named entity recognition, named entity disambiguation, entity linking, coreference resolution, text summarization, vector embeddings, n-gram representations, sentiment analysis, hate speech analysis, and combinations thereof.
9. The method of assessing consensus between documents as recited in claim 7 , further comprising the step of generating a publisher index containing a set of publisher names associated with each of the second data elements.
10. The method of assessing consensus between documents as recited in claim 9 , further comprising the step of generating a publisher credibility score associated with each of the publisher names stored in the publisher index.