Automated detection of reasoning in arguments
Automated detection of reasoning in arguments. A training set is generated by: obtaining multiple arguments, each comprising one or more sentences provided as digital text; automatically estimating a probability that each of the arguments includes reasoning, wherein the estimating comprises applying a contextual language model to each of the arguments; automatically labeling as positive examples those of the arguments which have a relatively high probability to include reasoning; and automatically labeling as negative examples those of the arguments which have a relatively low probability to include reasoning. Based on the generated training set, a machine learning classifier is automatically trained to estimate a probability that a new argument includes reasoning. The trained machine learning classifier is applied to the new argument, to estimate a probability that the new argument includes reasoning.
1 . A method comprising:
training a machine learning classifier to estimate a probability that a new argument includes reasoning, wherein the training is based on positive and negative examples each being an argument that is estimated, by a machine learning-based contextual language model, to have a relatively high probability to include reasoning or a relatively low probability to include reasoning, respectively; and
applying the trained machine learning classifier to the new argument, to estimate the probability that the new argument includes reasoning.
2 . The method of claim 1 , wherein at least some of the arguments of the positive and negative examples lack any conjunction that linguistically precedes a reasoning.
3 . The method of claim 1 , wherein at least some of the arguments of the positive and negative examples include an equivocal conjunction that can either linguistically precede a reasoning or have a meaning unrelated to reasoning.
4 . The method of claim 1 , wherein the machine learning-based contextual language model is a masked language model.
5 . The method of claim 4 , wherein the estimating of the probability to include reasoning comprises:
generating multiple variations of each of the arguments of the positive and negative examples by inserting a placeholder between every pair of consecutive words in each of the arguments of the positive and negative examples, such that each of the variations includes one placeholder;
using the masked language model to predict terms that can linguistically replace each of the placeholders, and to provide a probability for each replacement;
identifying, among the predicted terms, conjunctions that exist in a reasoning lexicon, wherein the reasoning lexicon comprises conjunctions that linguistically precede reasoning; and
assigning a single probability to each of the arguments of the positive and negative examples, based on the probabilities for replacement by the identified terms,
wherein each of the arguments of the positive and negative examples is labeled as being positive or negative based on the single probabilities of each of the arguments.
6 . The method of claim 5 , wherein the generating of the multiple variations is further by:
replacing every equivocal conjunction, that can either linguistically precede a reasoning or have a meaning unrelated to reasoning, with a placeholder.
7 . The method according to claim 1 , wherein the training and the applying are performed by at least one hardware processor.
8 . A system comprising:
(a) at least one hardware processor; and
(b) a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by said at least one hardware processor to:
train a machine learning classifier to estimate a probability that a new argument includes reasoning, wherein the training is based on positive and negative examples each being an argument that is estimated, by a machine learning-based contextual language model, to have a relatively high probability to include reasoning or a relatively low probability to include reasoning, respectively; and
apply the trained machine learning classifier to the new argument, to estimate the probability that the new argument includes reasoning.
9 . The system of claim 8 , wherein at least some of the arguments of the positive and negative examples lack any conjunction that linguistically precedes a reasoning.
10 . The system of claim 8 , wherein at least some of the arguments of the positive and negative examples include an equivocal conjunction that can either linguistically precede a reasoning or have a meaning unrelated to reasoning.
11 . The system of claim 8 , wherein the machine learning-based contextual language model is a masked language model.
12 . The system of claim 11 , wherein the estimating of the probability to include reasoning comprises:
generating multiple variations of each of the arguments of the positive and negative examples by inserting a placeholder between every pair of consecutive words in each of the arguments of the positive and negative examples, such that each of the variations includes one placeholder;
using the masked language model to predict terms that can linguistically replace each of the placeholders, and to provide a probability for each replacement;
identifying, among the predicted terms, conjunctions that exist in a reasoning lexicon, wherein the reasoning lexicon comprises conjunctions that linguistically precede reasoning; and
assigning a single probability to each of the arguments of the positive and negative examples, based on the probabilities for replacement by the identified terms,
wherein each of the arguments of the positive and negative examples is labeled as being positive or negative based on the single probabilities of each of the arguments.
13 . The system of claim 12 , wherein the generating of the multiple variations is further by:
replacing every equivocal conjunction, that can either linguistically precede a reasoning or have a meaning unrelated to reasoning, with a placeholder.
14 . A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by at least one hardware processor to:
train a machine learning classifier to estimate a probability that a new argument includes reasoning, wherein the training is based on positive and negative examples each being an argument that is estimated, by a machine learning-based contextual language model, to have a relatively high probability to include reasoning or a relatively low probability to include reasoning, respectively; and
apply the trained machine learning classifier to the new argument, to estimate the probability that the new argument includes reasoning.
15 . The computer program product of claim 14 , wherein at least some of the arguments of the positive and negative examples lack any conjunction that linguistically precedes a reasoning.
16 . The computer program product of claim 14 , wherein at least some of the arguments of the positive and negative examples include an equivocal conjunction that can either linguistically precede a reasoning or have a meaning unrelated to reasoning.
17 . The computer program product of claim 14 , wherein the contextual language model is a masked language model.
18 . The computer program product of claim 17 , wherein the estimating of the probability to include reasoning comprises:
generating multiple variations of each of the arguments of the positive and negative examples by inserting a placeholder between every pair of consecutive words in each of the arguments of the positive and negative examples, such that each of the variations includes one placeholder;
using the masked language model to predict terms that can linguistically replace each of the placeholders, and to provide a probability for each replacement;
identifying, among the predicted terms, conjunctions that exist in a reasoning lexicon, wherein the reasoning lexicon comprises conjunctions that linguistically precede reasoning; and
assigning a single probability to each of the arguments of the positive and negative examples, based on the probabilities for replacement by the identified terms,
wherein each of the arguments of the positive and negative examples is labeled as being positive or negative based on the single probabilities of each of the arguments.
19 . The computer program product of claim 18 , wherein the generating of the multiple variations is further by:
replacing every equivocal conjunction, that can either linguistically precede a reasoning or have a meaning unrelated to reasoning, with a placeholder.