System and methods for quantitative assessment of information in natural language contents
View Patent ↗A method is disclosed for quantitatively assessing information in natural language contents related to an object name. The method includes identifying a sentence in a document, determining a subject and a predicate in the sentence, and retrieving an object-specific data set related to the object name. The object-specific data set includes property names and association-strength values. Each property name is associated with an association-strength value. The method also includes identifying a first property name in the property names that matches the subject, assigning a first association-strength value associated with the first property name to the subject, identifying a second property name in the property names that matches the predicate, assigning a second association-strength value associated with the second property name to the predicate, and multiplying the first association-strength value and the second association-strength value to produce a sentence information index.
1. A method for quantitatively assessing information in natural language contents, comprising:
receiving an object name by a computer processing system;
receiving, by the computer processing system, a document containing text in a natural language;
identifying a sentence in the document by the computer processing system;
dividing the sentence into a subject and a predicate by the computer processing system, wherein the predicate is defined as the portion of a sentence other than the subject;
retrieving an object-specific data set related to the object name from a computer storage system, wherein the object-specific data set includes a plurality of property names and association-strength values, each property name being associated with an association-strength value;
identifying a first property name in the plurality of property names that matches the subject;
assigning a first association-strength value associated with the first property name to the subject;
identifying a second property name in the plurality of property names that matches the predicate;
assigning a second association-strength value associated with the second property name to the predicate; and
multiplying the first association-strength value and the second association-strength value by the computer processing system to produce a sentence information index as a quantitative measure for the amount of information contained in the sentence about the object name.
2. The method of claim 1 , further comprising:
computing a document information index using the sentence information index, wherein the document information index is a quantitative measure for the amount of information contained in the document about the object name.
3. The method of claim 2 , wherein the step of computing the document information index comprises:
computing sentence information indices of a plurality of sentences in the document, wherein the sentence information indices are specific to the object name; and
computing the document information index as a function of the sentence information indices.
4. The method of claim 3 , wherein the step of computing the document information index comprises:
summing up the sentence information indices to produce the document information index.
5. The method of claim 2 , further comprising:
receiving a query related to the object name by the computer processing system; computing document information indices for a plurality of documents, wherein the document information indices each is a quantitative measure for the amount of information contained in the document about the object name; and
ranking the plurality of documents by their respective document information indices to produce a search result in response to a query.
6. The method of claim 2 , further comprising:
receiving an object name to classify documents by the computer processing system; computing document information indices for a plurality of documents, wherein the document information indices each is a quantitative measure for the amount of information contained in the document about the object name;
determining a threshold value for a document category related to the object name;
selecting one or more documents from the plurality of documents by comparing their respective document information indices to the threshold value; and
assigning the one or more documents to the document category.
7. The method of claim 6 , further comprising:
determining a threshold value for a document category related to the object name;
selecting one or more documents from the plurality of documents if the document information indices of the one or more documents are above or equal to the threshold value;
assigning the one or more documents to the document category.
8. The method of claim 1 , further comprising:
assigning a zero value to the sentence information index if a property name matching the subject or the predicate is not identified in the plurality of property names in the object-specific data set.
9. The method of claim 1 , wherein the subject or the predicate in the sentence comprises a single word, a string of words, a phrase, or an embedded clause.
10. The method of claim 1 , wherein the subject and the predicate in the sentence are determined by the computer processing system using a syntactic parser.
11. The method of claim 1 , wherein the object name includes a word, a list of words, a phrase, a character string, a query, a sub-component of a query, a topic, or the name of a document category.
12. The method of claim 1 , wherein the step of identifying a sentence in the document comprises:
identifying a complex sentence in the document by the computer processing system;
decomposing the complex sentence into one or more simple sentences, wherein a subject and a predicate are identified in each of the simple sentences by the computer processing system;
calculating sentence information indices for the one or more simple sentences; and
computing a sentence information index for the complex sentence using the sentence information indices for the one or more simple sentences.
13. A method for finding text documents relevant to a search query, comprising:
receiving a search query related to an object name by a computer processing system;
receiving, by the computer processing system, a plurality of documents each containing text in a natural language;
retrieving an object-specific data set related to the object name from a computer storage system, wherein the object-specific data set includes a plurality of property names and association-strength values, each property name being associated with an association-strength value;
computing a document information index for each of the plurality of documents, wherein the document information index is a quantitative measure for the amount of information contained in the document about the object name; and
ranking the plurality of documents by their respective document information indices to produce a search result in response to the search query,
wherein the step of computing a document information index comprises:
identifying a sentence in the document by the computer processing system;
dividing the sentence into a subject and a predicate by the computer processing system, wherein the predicate is defined as the portion of a sentence other than the subject;
identifying a first property name in the plurality of property names in the object-specific data set that matches the subject;
assigning a first association-strength value associated with the first property name to the subject;
identifying a second property name in the plurality of property names in the object-specific data set that matches the predicate;
assigning a second association-strength value associated with the second property name to the predicate;
multiplying the first association-strength value and the second association-strength value to produce a sentence information index; and
calculating the document information index using the sentence information index.
14. The method of claim 13 , wherein the document information index is calculated by summing up sentence information indices calculated for sentences identified in the document.
15. A method for classifying text documents in a natural language, comprising:
receiving, by the computer processing system, an object name for document classification;
receiving, by the computer processing system, a plurality of documents each containing text in a natural language;
retrieving an object-specific data set related to the object name from a computer storage system, wherein the object-specific data set includes a plurality of property names and association-strength values, each property name being associated with an association-strength value;
computing a document information index for each of the plurality of documents, wherein the document information index is a quantitative measure for the amount of information contained in the document about the object name;
selecting one or more documents relevant to the object name from the plurality of documents using their respective document information indices; and
assigning the one or more documents to a document category related to the object name,
wherein the step of computing a document information index comprises:
identifying a sentence in the document by the computer processing system;
dividing the sentence into a subject and a predicate by the computer processing system, wherein the predicate is defined as the portion of a sentence other than the subject;
identifying a first property name in the plurality of property names in the object-specific data set that matches the subject;
assigning a first association-strength value associated with the first property name to the subject;
identifying a second property name in the plurality of property names in the object-specific data set that matches the predicate;
assigning a second association-strength value associated with the second property name to the predicate;
multiplying the first association-strength value and the second association-strength value to produce a sentence information index; and
calculating the document information index using the sentence information index.
16. The method of claim 15 , further comprising:
determining a threshold value for a document category related to the object name;
selecting one or more documents from the plurality of documents if the document information indices of the one or more documents are above or equal to the threshold value;
assigning the one or more documents to the document category.
17. A method for quantitatively assessing information in a text content about an object name, comprising:
receiving an object name by a computer processing system;
retrieving an object-specific data set related to the object name from a computer storage system, wherein the object-specific data set includes a plurality of property names and association-strength values, each property name being associated with an association-strength value;
receiving, by the computer processing system, a text content in a natural language;
identifying a sentence in the text content;
identifying a plurality of component strings in the sentence by the computer processing system;
searching for a property name in the plurality of property names that matches at least one of the component strings;
assigning an association-strength value associated with the property name to the one of the component strings;
averaging, by the computer processing system, the association-strength values assigned to the component strings in the sentence to produce a sentence information index; and
calculating an information index for the text content using the sentence information index, wherein the information index is a quantitative measure for the amount of information contained in the text content about the object name.
18. A system for quantitatively assessing information in natural language contents, comprising:
a computer processing system configured to receive an object name and a document containing text in a natural language; and
a computer storage system in communication with the computer processing system and configured to store an object-specific data set related to the object name from wherein the object-specific data set includes a plurality of property names and association-strength values, each property name being associated with an association-strength value,
wherein the computer processing system is configured to identify a sentence in the document, divide the sentence into a subject and a predicate, wherein the predicate is defined as the portion of a sentence other than the subject, identify a first property name in the plurality of property names that matches the subject, assign a first association-strength value associated with the first property name to the subject, identify a second property name in the plurality of property names that matches the predicate, assign a second association-strength value associated with the second property name to the predicate; and multiply the first association-strength value and the second association-strength value to produce a sentence information index as a quantitative measure for the amount of information contained in the sentence about the object name.
19. The system of claim 18 , wherein the computer processing system is further configured to compute a document information index using the sentence information index, wherein the document information index is a quantitative measure for the amount of information contained in the document about the object name.
20. The system of claim 19 , wherein the computer processing system is further configured to sum up the sentence information indices computed for sentences in the document to produce the document information index.