Enabling chatbots by detecting and supporting argumentation
Systems, devices, and methods of the present invention detect argumentation in text. In an example, an application executing on a computing device accesses text comprising fragments. The application creates a discourse tree from the text. The discourse tree includes nodes, each nonterminal node representing a rhetorical relationship between two of the fragments and each terminal node of the nodes of the discourse tree is associated with one of the fragments. The application matches each fragment that has a verb to a verb signature, thereby creating a communicative discourse tree. The application determines whether the communicative discourse tree represents text that includes argumentation by applying a classification model trained to detect argumentation to the communicative discourse tree.
1. A computer-implemented method for determining a presence of argumentation in text, the method comprising:
receiving, from user device, text comprising fragments;
creating a discourse tree from the text, wherein the discourse tree comprises a plurality of nodes, each nonterminal node representing a rhetorical relationship between two of the fragments and each terminal node of the nodes of the discourse tree is associated with one of the fragments;
matching each fragment that has a verb to a verb signature, thereby creating a communicative discourse tree;
determining whether the communicative discourse tree represents text that comprises argumentation by applying a classification model trained to detect a level of argumentation to the communicative discourse tree;
responsive to determining that the level of argumentation is above a threshold, identifying the text as comprising argumentation;
adjusting a response based on the indentified argumentation; and providing the adjusted response to the user device.
2. The method of claim 1 , wherein each fragment comprises a verb, and wherein the matching comprises:
accessing, for each fragment, a plurality of verb signatures, wherein each verb signature comprises a verb of the respective fragment and a sequence of thematic roles, wherein thematic roles describe a relationship between the verb and related words;
determining, for each verb signature of the plurality of verb signatures, a plurality of thematic roles of the respective signature that match a role of a word in the fragment;
selecting a particular verb signature from the plurality of verb signatures based on the particular verb signature comprising a highest number of matches; and
associating the particular verb signature with the fragment.
3. The method of claim 2 , wherein each verb signature of the plurality of verb signatures comprises one of (i) an adverb, (ii) a noun phrase, or (iii) a noun, and wherein associating the particular verb signature with the fragment further comprises:
identifying each of a plurality of thematic roles in the particular verb signature; and
matching, for each of the plurality of thematic roles in the particular verb signature, a corresponding word in the fragment to the thematic role.
4. The method of claim 1 , wherein the classification model is a support vector machine with tree kernel learning.
5. The method of claim 1 , wherein the classification model uses nearest neighbor learning of maximal common sub-trees.
6. The method of claim 2 , wherein the associating further comprises:
identifying each of the plurality of thematic roles in the particular verb signature; and
matching, for each of the plurality of thematic roles in the particular verb signature, a corresponding word in the fragment to the respective thematic role.
7. The method of claim 1 , wherein the classification model:
determines similarities between the communicative discourse tree and one or more training communicative discourse trees from a training set of communicative discourse trees;
selects an additional communicative discourse tree from the one or more training communicative discourse trees, based on the additional communicative discourse tree having a highest number of similarities with the communicative discourse tree;
identifies whether the additional communicative discourse tree is from a positive set or a negative set, wherein the positive set is associated with a first text containing argumentation and the negative set is associated with a second text containing no argumentation; and
determines, based on the identifying, whether the text contains an argumentation or no argumentation.
8. The method of claim 7 , wherein the similarities comprise common rhetoric relations or common communicative action labels between the communicative discourse tree and the one or more training communicative discourse trees from the training set of comminicative discourse trees.
9. The method of claim 7 , wherein the similarities comprise common structures of nonterminal nodes and terminal nodes between the communicative discourse tree and the one or more training communicative discourse trees from the training set of communicative discourse trees.
10. A method of building a dataset of argumentation features, the method comprising:
accessing text comprising fragments;
creating a discourse tree from the text, wherein the discourse tree comprises a plurality of nodes, each nonterminal node representing a rhetorical relationship between two of the fragments and each terminal node of the nodes of the discourse tree is associated with one of the fragments;
matching each fragment that has a verb to a verb signature, thereby creating a communicative discourse tree;
obtaining a numerical score by applying a classification model to the communicative discourse tree;
identifying, based on the numerical score, whether the communicative discourse tree is similar to a positive set or a negative set, wherein the positive set comprises a first plurality of communicative discourse trees representing a first text containing argumentation and the negative set comprises a second plurality of communicative discourse trees representing a second text without argumentation; and
adding the communicative discourse tree to either the positive training set or the negative training set based on the identifying.
11. The method of claim 10 , wherein the classification model is trained by iteratively:
providing one of a set of training pairs to the classification model, wherein each training pair comprises a communicative discourse tree and an expected level of argumentation;
receiving, from the classification model, a determined level of argumentation;
calculating a loss function by calculating a difference between the determined level of argumentation and the expected level of argumentation; and
adjusting internal parameters of the classification model to minimize the loss function.
12. A system comprising:
a non-transitory computer-readable medium storing computer-executable program instructions; and
a processing device communicatively coupled to the non-transitory computer-readable medium for executing the computer-executable program instructions, wherein executing the computer-executable program instructions configures the processing device to perform operations comprising:
receiving, from a user device, text comprising fragments;
creating a discourse tree from the text, wherein the discourse tree comprises a plurality of nodes, each nonterminal node representing a rhetorical relationship between two of the fragments and each terminal node of the nodes of the discourse tree is associated with one of the fragments;
matching each fragment that has a verb to a verb signature of a set of verb signatures, thereby creating a communicative discourse tree; and
determining whether the communicative discourse tree represents text that comprises argumentation by applying a classification model trained to detect a level of argumentation to the communicative discourse tree;
responsive to determining that the level of argumentation is above a threshold, identifying the text as comprising argumentation; and
adjusting a response based on the indentified argumentation; and providing the adjusted response to the user device.
13. The system of claim 12 , wherein the matching comprises:
accessing a plurality of verb signatures, wherein each verb signature comprises a verb of a fragment of the fragments and a sequence of thematic roles, wherein thematic roles describe a relationship between a verb and related words;
determining, for each verb signature of the plurality of verb signatures, a plurality of thematic roles of the respective signature that match a role of a word in the fragment;
selecting a particular verb signature from the plurality of verb signatures based on the particular verb signature comprising a highest number of matches; and
associating the particular verb signature with the fragment.
14. The system of claim 13 , wherein each verb signature of the set of verb signatures comprises one of (i) an adverb, (ii) a noun phrase, or (iii) a noun, and wherein associating the particular verb signature with the fragment further comprises:
identifying each of a plurality of thematic roles in the particular verb signature; and
matching, for each of the plurality of thematic roles in the particular verb signature, a corresponding word in the fragment to the thematic role.
15. The system of claim 13 , wherein the associating further comprises:
identifying each of the plurality of thematic roles in the particular verb signature; and
matching, for each of the plurality of thematic roles in the particular verb signature, a corresponding word in the fragment to the respective thematic role.
16. The system of claim 12 , wherein the classification model is a support vector machine with tree kernel learning.
17. The system of claim 12 , wherein the classification model uses nearest neighbor learning of maximal common sub-trees.
18. The system of claim 12 , wherein the determining further comprises:
determining a similarity between the communicative discourse tree and one or more training communicative discourse trees from a training set of communicative discourse trees;
selecting an additional communicative discourse tree from the one or more communicative discourse trees based on the additional communicative discourse tree having a highest number of similarities with the communicative discourse tree;
identifying whether the additional communicative discourse tree is from a positive set or a negative set, wherein the positive set is associated with a first text containing argumentation and the negative set is associated with a second text containing no argumentation; and
determining, based on the identifying, whether the text contains an argumentation or no argumentation.
19. The system of claim 18 , wherein the similarity is based on common rhetoric relations or common communicative action labels between the communicative discourse tree and the one or more training communicative discourse trees from the training set of communicative discourse trees.
20. The system of claim 17 , wherein the similarity is based on common structures of nonterminal nodes and terminal nodes between the communicative discourse tree and the one or more training communicative discourse trees from the training set of communicative discourse trees.