Machine-learning based irrelevant sentence classifier
There is a need for more effective, efficient, and accurate computer text comprehension. This need is addressed by applying unique text processing techniques to identify and remove irrelevant sentences from a narrative. The text processing techniques include a machine-learning based model that is trained using automatically generated training data that is tailored to a particular circumstance. A method for machine narrative comprehension includes receiving a narrative data object comprising one or more sentences; determining, using a machine-learning based irrelevant classifier model, a relevance of at least one of the one or more sentences; responsive to a determination that at least one sentence is irrelevant, generating a pertinent summary by removing the at least one sentence from the narrative; and generating, based at least in part on the pertinent summary, an output indicia data object for the narrative data object.
1 . A computer-implemented method comprising:
receiving, by one or more processors, a narrative data object comprising one or more sentences;
inputting, by the one or more processors, a sentence, without contextual data from the narrative data object, of the one or more sentences to a binary classification model to determine a binary classification that identifies a general relevance of the sentence, wherein the binary classification comprises either a relevant classification or an irrelevant classification and the binary classification model is trained by:
(i) receiving (a) a training input comprising a historical narrative data object with a plurality of historical narrative sentences and (b) a training output comprising one or more summary sentences for the historical narrative data object,
(ii) determining a first labeled historical narrative sentence from the training input by assigning a relevant training label to a first historical narrative sentence of the plurality of historical narrative sentences in response to a first determination that a first maximum relevance score of a first plurality of relevance scores between the first historical narrative sentence and the one or more summary sentences exceeds a score threshold of zero,
(iii) determining a second labeled historical narrative sentence from the training input by assigning an irrelevant training label to a second historical narrative sentence of the plurality of historical narrative sentences in response to a second determination that a second maximum relevance score of a second plurality of relevance scores between the second historical narrative sentence and the one or more summary sentences fails to exceed the score threshold of zero, and
(iv) training the binary classification model using the first labeled historical narrative sentence and the second labeled historical narrative sentence from the training input;
responsive to a determination that the binary classification is the irrelevant classification, generating, by the one or more processors, a pertinent summary by removing the sentence from the narrative data object; and
outputting, by the one or more processors and based at least in part on the pertinent summary, an output indicia data object for the narrative data object.
2 . The computer-implemented method of claim 1 , wherein the output indicia data object comprises an intent prediction for the narrative data object, and wherein outputting, based at least in part on the pertinent summary, the output indicia data object for the narrative data object comprises:
generating, using a machine-learning based intent prediction model, the intent prediction based at least in part on the pertinent summary.
3 . The computer-implemented method of claim 1 , wherein the output indicia data object comprises a classification prediction for the narrative data object, and wherein outputting, based at least in part on the pertinent summary, the output indicia data object for the narrative data object comprises:
generating, using a machine-learning based prediction model, the classification prediction based at least in part on the pertinent summary.
4 . The computer-implemented method of claim 1 , wherein the output indicia data object comprises a paraphrased contextual summary for the narrative data object, and wherein outputting, based at least in part on the pertinent summary, the output indicia data object for the narrative data object comprises:
generating, using a machine-learning based paraphraser model, the paraphrased contextual summary based at least in part on the pertinent summary.
5 . The computer-implemented method of claim 1 , wherein the binary classification model is trained using supervisory training techniques.
6 . The computer-implemented method of claim 1 , wherein the first maximum relevance score comprises a first Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score and the second maximum relevance score comprises a second ROUGE score.
7 . The computer-implemented method of claim 6 , wherein the relevant training label is determined responsive to the first ROUGE score exceeding the score threshold, and the irrelevant training label is determined responsive to the second ROUGE score failing to exceed the score threshold.
8 . The computer-implemented method of claim 1 , wherein the narrative data object comprises a multi-party interaction transcript corresponding to a multi-party interaction between at least two participants and the pertinent summary comprises a summary sentence of the multi-party interaction.
9 . A system comprising:
one or more processors; and
one or more memories storing processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving a narrative data object comprising one or more sentences;
inputting a sentence, without contextual data from the narrative data object, of the one or more sentences to a binary classification model to determine a binary classification that identifies a general relevance of the sentence, wherein the binary classification comprises either a relevant classification or an irrelevant classification and the binary classification model is trained by:
(i) receiving (a) a training input comprising a historical narrative data object with a plurality of historical narrative sentences and (b) a training output comprising one or more summary sentences for the historical narrative data object,
(ii) determining a first labeled historical narrative sentence from the training input by assigning a relevant training label to a first historical narrative sentence of the plurality of historical narrative sentences in response to a first determination that a first maximum relevance score of a first plurality of relevance scores between the first historical narrative sentence and the one or more summary sentences exceeds a score threshold of zero,
(iii) determining a second labeled historical narrative sentence from the training input by assigning an irrelevant training label to a second historical narrative sentence of the plurality of historical narrative sentences in response to a second determination that a second maximum relevance score of a second plurality of relevance scores between the second historical narrative sentence and the one or more summary sentences fails to exceed the score threshold of zero, and
(iv) training the binary classification model using the first labeled historical narrative sentence and the second labeled historical narrative sentence from the training input;
responsive to a determination that the binary classification is the irrelevant classification, generating a pertinent summary by removing the sentence from the narrative data object; and
outputting, based at least in part on the pertinent summary, an output indicia data object for the narrative data object.
10 . The system of claim 9 , wherein the output indicia data object comprises an intent prediction for the narrative data object, and outputting, based at least in part on the pertinent summary, the output indicia data object for the narrative data object comprises:
generating, using a machine-learning based intent prediction model, the intent prediction for the narrative data object.
11 . The system of claim 10 , wherein the system comprises an automated robotic assistant.
12 . The system of claim 11 , wherein the narrative data object is an assistance query, and the operations further comprise:
generating a query resolution for the assistance query based at least in part on the intent prediction, and
initiating an action based at least in part on the query resolution.
13 . One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a narrative data object comprising one or more sentences;
inputting a sentence, without contextual data from the narrative data object, of the one or more sentences to a binary classification model to determine a binary classification that identifies a general relevance of the sentence, wherein the binary classification comprises either a relevant classification or an irrelevant classification and the binary classification model is trained by:
(i) receiving (a) a training input comprising a historical narrative data object with a plurality of historical narrative sentences and (b) a training output comprising one or more summary sentences for the historical narrative data object,
(ii) determining a first labeled historical narrative sentence from the training input by assigning a relevant training label to a first historical narrative sentence of the plurality of historical narrative sentences in response to a first determination that a first maximum relevance score of a first plurality of relevance scores between the first historical narrative sentence and the one or more summary sentences exceeds a score threshold of zero,
(iii) determining a second labeled historical narrative sentence from the training input by assigning an irrelevant training label to a second historical narrative sentence of the plurality of historical narrative sentences in response to a second determination that a second maximum relevance score of a second plurality of relevance scores between the second historical narrative sentence and the one or more summary sentences fails to exceed the score threshold of zero, and
(iv) training the binary classification model using the first labeled historical narrative sentence and the second labeled historical narrative sentence from the training input;
responsive to a determination that the binary classification is the irrelevant classification, generating a pertinent summary by removing the sentence from the narrative data object; and
outputting, based at least in part on the pertinent summary, an output indicia data object for the narrative data object.