NLP-based systems and methods for providing quotations
View Patent ↗Techniques for providing quotations obtained from text documents using natural language processing techniques are described. Some embodiments provide a content recommendation system (“CRS”) configured to provide quotations by extracting quotations from a corpus text documents, and providing access to the extracted quotations in response to search requests received from users. The CRS may extract quotations by using natural language processing-based techniques to identify one or more entities, such as people, places, objects, concepts, or the like, that are referenced by the extracted quotations. The CRS may then store the extracted quotations along with identified entities, such as quotation speakers and subjects, for later access via search requests.
1. A method in a content recommendation system, the method comprising:
under control of a computing system,
extracting quotations from a corpus of text documents;
identifying one or more entities that are referenced by each of the extracted quotations, each of the identified entities being electronically represented by the content recommendation system;
indexing, by the computing system, the extracted quotations, wherein indexing the extracted quotations includes storing a speaker-verb-quote triple in an inverted index managed by the content recommendation system;
determining one or more of the extracted quotations that match a received quotation search request; and
providing the determined one or more quotations.
2. The method of claim 1 wherein extracting quotations includes dividing one of the text documents into linguistic units including at least one of: a sentence, a phrase, a clause, and/or a paragraph.
3. The method of claim 1 wherein extracting quotations includes performing parts-of-speech tagging on sentences of one of the text documents.
4. The method of claim 1 wherein extracting quotations includes performing lexical analysis to identify phrases in one of the text documents.
5. The method of claim 1 wherein extracting quotations includes determining grammatical roles including subjects, verbs, and objects of sentences of one of the text documents.
6. The method of claim 1 wherein extracting quotations includes detecting punctuation indicating a beginning or end of a quotation.
7. The method of claim 1 wherein extracting quotations includes detecting a quotation verb by determining whether a verb in a sentence of one of the text documents is one of a predetermined list of quotation verbs.
8. The method of claim 1 wherein identifying one or more entities includes linking together multiple mentions of a same entity across one of the text documents, the linking including resolving pronoun coreference, resolving references and/or abbreviations, and/or resolving definite noun or pronoun anaphora.
9. The method of claim 1 wherein identifying one or more entities includes assigning a type or facet to each of the one or more entities.
10. The method of claim 1 wherein identifying one or more entities includes recognizing and disambiguating a document reference to an entity, such that the reference is linked to a representation of the entity stored in an entity repository.
11. The method of claim 1 wherein extracting quotations includes attributing each of the extracted quotations to a corresponding speaker entity.
12. The method of claim 1 wherein extracting quotations includes determining a speaker, verb, quote triple for each of the extracted quotations.
13. The method of claim 1 wherein extracting quotations includes identifying a speaker for a quotation by identifying a subject of a sentence, and optionally including identifying one or more modifiers associated with the subject.
14. The method of claim 1 wherein extracting quotations includes identifying a verb for a quotation by identifying a verb of a sentence, and optionally including identifying prepositional modifiers of the verb.
15. The method of claim 1 wherein indexing the extracted quotations includes storing at least one of: a reference to a subject entity; a facet associated with a subject entity; subject modifiers; an action; an action modifier; and/or one or more references to object entities and/or facets that are referenced within the quote.
16. The method of claim 1 further comprising receiving a quotation search request that includes a query.
17. The method of claim 16 wherein the query specifies a speaker by including at least one of: an entity reference, a facet reference, and/or any speaker.
18. The method of claim 17 wherein the query specifies one or more modifiers of the speaker.
19. The method of claim 16 wherein the query specifies a subject by including at least one of: an entity reference, a facet reference, one or more keyterms, and/or any subject.
20. The method of claim 16 wherein the query includes Boolean combinations of multiple speakers or subjects.
21. The method of claim 1 wherein providing the determined quotations includes presenting at least one of: a quotation; an attribution; a context including text surrounding a quotation; or document metadata including title, publication date, and/or author.
22. The method of claim 1 wherein providing the determined quotations includes ranking the quotations based on one or more factors including: publication date, number of documents including the quotation, and/or credibility of quotation source.
23. A computing system configured to recommend content, comprising:
a memory;
a module stored on the memory that is configured, when executed, to:
extract quotations from a corpus of text documents;
identify one or more entities that are referenced by each of the extracted quotations, each of the identified entities being electronically represented by the content recommendation system;
index the extracted quotations, wherein indexing the extracted quotations includes storing a speaker-verb-quote triple in an inverted index managed by the content recommendation system;
determine one or more of the extracted quotations that match a received quotation search request; and
provide the determined one or more quotations.
24. The computing system of claim 23 wherein the module includes software instructions for execution in the memory of the computing system.
25. The computing system of claim 23 wherein the module is a content recommendation system.
26. The computing system of claim 23 wherein the module is configured to recommend content items to at least one of a desktop computing system, a personal digital assistant, a smart phone, a laptop computer, a mobile application, and/or a third-party application.
27. A non-transitory computer-readable medium including:
contents that, when executed, cause a computing system to recommend content, by performing a method comprising:
extracting quotations from a corpus of text documents;
identifying one or more entities that are referenced by each of the extracted quotations, each of the determined entities being electronically represented by the content recommendation system;
indexing the extracted quotations, wherein indexing the extracted quotations includes storing a speaker-verb-quote triple in an inverted index managed by the content recommendation system;
determining one or more of the extracted quotations that match a received quotation search request; and
providing the determined one or more quotations.
28. The computer-readable medium of claim 27 wherein the computer-readable medium is at least one of a memory in a computing device or a data transmission medium transmitting a generated signal containing the contents.
29. The computer-readable medium of claim 27 wherein the contents are instructions that when executed cause the computing system to perform the method.