Method and apparatus for concept-based classification of natural language discourse
Pinnacle concepts are not amenable to detection by the use of keywords. A unit of natural language discourse (UNLD) “refers” to a pinnacle concept “C” when that UNLD uses linguistic expressions in such a way that “C” is regarded as expressed, used or invoked by an ordinary reader of “L.” A reference can have a “reference level” value that is proportional to: the “strength” with which the pinnacle concept is referenced, the probability that a pinnacle concept is referenced or both strength and probability. Pinnacle concepts can be divided into Quantifiers and non-Quantifiers. A Quantifier can modify the reference level assigned to a non-Quantifier. A concept “C,” that is determined to be referenced by a UNLD “x,” after application of its Quantifiers, is said to be asserted by “x.” Concept-based classification is the identification of whether a pinnacle concept “C” is asserted by a UNLD. Concept-based classification can be used for concept-based search.
1. A method for determining whether a first pinnacle concept is referenced by a first unit of natural language discourse, comprising:
parsing the first unit of natural language discourse into a first parse structure that represents each sub-unit, of the first unit of natural language discourse, by a node;
adding at least one concept-value pair, each of which indicates a reference to a same non-Quantifier concept, to at least one node of the first parse structure, wherein each reference is determined by identifying an occurrence of a first linguistic feature from a first set of linguistic features and the first set of linguistic features is approximately complete with respect to the non-Quantifier concept;
adding at least one concept-value pair, each of which indicates a reference to a same Quantifier concept, to at least one node of the first parse structure, wherein each reference is determined by identifying an occurrence of a second linguistic feature from a second set of linguistic features and the second set of linguistic features is approximately complete with respect to the Quantifier concept;
propagating the at least one concept-value pair for the Quantifier concept;
identifying a first node of the first parse structure that has at least one concept-value pair for the non-Quantifier concept and at least one concept-value pair for the Quantifier concept;
determining a first value to be scaled from the least one concept-value pair for the non-Quantifier concept;
determining a first scaling value from the least one concept-value pair for the Quantifier concept;
scaling the first value to be scaled with the first scaling value to produce a first scaled value; and
propagating the at least one concept-value pair for the non-Quantifier concept.
2. A method for determining whether a first pinnacle concept is referenced by a first unit of natural language discourse, comprising:
parsing, performed at least in part with a configuration of electronic hardware, the first unit of natural language discourse into a first parse structure that represents each sub-unit, of the first unit of natural language discourse, by a node;
adding, performed at least in part with a configuration of electronic hardware, at least one concept-value pair, each of which indicates a reference to a same non-Quantifier concept, to at least one node of the first parse structure, wherein each reference is determined by identifying an occurrence of a first linguistic feature from a first set of linguistic features and the first set of linguistic features is approximately complete with respect to the non-Quantifier concept;
adding, performed at least in part with a configuration of electronic hardware, at least one concept-value pair, each of which indicates a reference to a same Quantifier concept, to at least one node of the first parse structure, wherein each reference is determined by identifying an occurrence of a second linguistic feature from a second set of linguistic features and the second set of linguistic features is approximately complete with respect to the Quantifier concept;
propagating, performed at least in part with a configuration of electronic hardware, the at least one concept-value pair for the Quantifier concept;
identifying, performed at least in part with a configuration of electronic hardware, a first node of the first parse structure that has at least one concept-value pair for the non-Quantifier concept and at least one concept-value pair for the Quantifier concept;
determining, performed at least in part with a configuration of electronic hardware, a first value to be scaled from the least one concept-value pair for the non-Quantifier concept;
determining, performed at least in part with a configuration of electronic hardware, a first scaling value from the least one concept-value pair for the Quantifier concept;
scaling, performed at least in part with a configuration of electronic hardware, the first value to be scaled with the first scaling value to produce a first scaled value; and
propagating, performed at least in part with a configuration of electronic hardware, the at least one concept-value pair for the non-Quantifier concept.
3. The method of claim 2 , wherein the first set of linguistic features is determined utilizing machine learning.
4. The method of claim 2 , wherein the first set of linguistic features is determined utilizing at least one language-reference tool to find similar additional linguistic features from at least one known linguistic feature.
5. The method of claim 2 , wherein the non-Quantifier Concept is selected from the group consisting of Bad, Good, Need and Purchase.
6. The method of claim 2 , wherein the Quantifier concept is selected from the group consisting of Intensify, Increase, Diminish, Decrease, Negation and Solution.
7. The method of claim 2 , wherein the first unit of natural language discourse is a sentence.
8. The method of claim 2 , wherein a sub-unit of the first unit of natural language discourse is a lexical unit.
9. The method of claim 2 , wherein the first linguistic feature is comprised of at least one lexical unit.
10. The method of claim 2 , wherein the step of propagating the at least one concept-value pair for the non-Quantifier concept further comprises:
keeping separate the propagation of each added concept-value pair that indicates a reference to the non-Quantifier concept.
11. The method of claim 2 , wherein the step of propagating the at least one concept-value pair for the non-Quantifier concept further comprises:
propagating a concept-value pair, that indicates a reference to the non-Quantifier concept, from a second node to a third node if there is an edge from the second node to the third node.
12. The method of claim 2 , wherein the step of propagating the at least one concept-value pair for the non-Quantifier concept further comprises:
attenuating a value portion, of a concept-value pair that indicates a reference to the non-Quantifier concept, when the concept-value pair propagates from a second node to a third node.
13. The method of claim 12 , wherein the step of attenuating further comprises:
multiplying the value portion by an attenuation coefficient.
14. The method of claim 2 , wherein the step of propagating the at least one concept-value pair for the Quantifier concept further comprises:
keeping separate the propagation of each added concept-value pair that indicates a reference to the Quantifier concept.
15. The method of claim 2 , wherein the step of propagating the at least one concept-value pair for the Quantifier concept further comprises:
propagating a concept-value pair, that indicates a reference to the non-Quantifier concept, from a second node to a third node if an edge, from the second node to the third node, is of a type selected from the group consisting of edges where one node is a verb and the other node is an agent of the verb, edges where one node is a verb and the other node is a patient of the verb and edges where one node is a modifier and the other node is an object of the modifier.
16. The method of claim 2 , wherein the step of propagating the at least one concept-value pair for the Quantifier concept further corn prises:
holding a value portion, of a concept-value pair that indicates a reference to the non-Quantifier concept, constant when the concept-value pair propagates from a second node to a third node.
17. The method of claim 2 , further comprising the following step:
taking a maximum of a set of values resulting from extracting a value portion of each concept-value pair, at a second node, for the non-Quantifier concept;
18. The method of claim 2 , further comprising the following step:
taking a sum of a set of values resulting from extracting a value portion of each concept-value pair, at the first node, for the Quantifier concept;
19. The method of claim 2 , wherein the step of scaling further comprises:
inverting the non-Quantifier concept when the first scaling value indicates that a negation is to be performed.
20. The method of claim 2 , wherein the non-Quantifier concept is a Characteristic concept.