Model performance through text-to-text transformation via distant supervision from target and auxiliary tasks

1 . A computer-implemented method of improving a machine learning training dataset by performing text-to-text transformation, the computer-implemented method comprising:

performing a text-to-text transformation operation on an original input text of a specific task to generate a plurality of transformed text, wherein

the original input text corresponds to a multiple sentence question,

the text-to-text transformation operation includes compressing the multiple sentence question by removing one or more portions of the multiple sentence question to generate the plurality of transformed text,

each of the one or more portions of the multiple sentence question includes words, phrases, or a sentence of the multiple sentence question, and

the plurality of transformed text corresponds to a summary of the multiple sentence question using fewer words than in the original input text;

applying a task-specific performance metric to measure an operation of the specific task using each of the plurality of transformed text;

pairing, based on the applying of the task-specific performance metric, each of the plurality of transformed text with the task-specific performance metric; and

updating a training dataset to include each pairing of the plurality of transformed text with the task-specific performance metric.

2 . The computer-implemented method of claim 1 , wherein each pairing of the plurality of transformed text is provided to the training dataset after determining whether a quality criterion is satisfied.

3 . The computer-implemented method of claim 1 , wherein

the specific task comprises at least one target task provided in the original input text; and

the computer-implemented method further comprises

performing a distant supervision of the text-to-text transformation operation based on a set of related auxiliary tasks having labeled data in addition to data of the specific task.

4 . The computer-implemented method of claim 3 , further comprising:

performing a new text transformation operation using the updated training dataset to generate a new plurality of transformed text;

applying the task-specific performance metric to measure the operation of the specific task using each of the new plurality of transformed text;

pairing, based on the applying of the task-specific performance metric, each of the new plurality of transformed text with the task-specific performance metric; and

updating the training dataset to include each pairing of the new plurality of transformed text with the task-specific performance metric.

5 . The computer-implemented method of claim 3 , wherein:

the labeled data of the set of related auxiliary tasks includes a set of question and answer (QA) pairs; and

the text-to-text transformation operation further comprises:

generating a plurality of question compression candidates from the set of QA pairs; and

selecting one or more of the plurality of question compression candidates based on an answer ranking of the set of QA pairs.

6 . The computer-implemented method of claim 5 , wherein the plurality of question compression candidates is based on a number of words in the original input text.

7 . The computer-implemented method of claim 1 , wherein the updating of the training dataset is performed iteratively after performing a successively new text transformation operation, the applying of the task-specific performance metric, and pairing each successively new transformed text until a terminating criterion is satisfied.

8 . The computer-implemented method of claim 7 , wherein the pairing of each successively new transformed text and the task-specific performance metric is provided to the training dataset after determining whether a quality criterion is satisfied.

9 . The computer-implemented method of claim 1 , wherein the specific task comprises answering a specific question embedded in the multiple sentence question.

10 . The computer-implemented method of claim 9 , wherein the text-to-text transformation operation further comprises providing an answer to the specific question embedded in one or more relevant portions of the multiple sentence question.

11 . The computer-implemented method of claim 10 , wherein the providing of the answer further comprises:

determining one or more portions of the specific question that are at least one of irrelevant to providing the answer to the specific question, or redundant to at least a part of relevant portions of the specific question; and

discarding the one or more portions of the specific question.

12 . A computer-implemented method of improving a machine learning training dataset by performing text-to-text transformation, the computer-implemented method comprising:

receiving training examples of an original input text of a specific task;

performing a text-to-text transformation operation of the original input text into a plurality of transformed text, wherein

the original input text corresponds to a multiple sentence question,

the text-to-text transformation operation includes compressing the multiple sentence question by removing one or more portions of the multiple sentence question to generate the plurality of transformed text,

each of the one or more portions of the multiple sentence question includes words, phrases, or a sentence of the multiple sentence question,

the plurality of transformed text corresponds to a summary of the multiple sentence question using fewer words than in the original input text, and

the specific task comprises at least one target task provided in the original input text;

performing a distant supervision of the text-to-text transformation operation based on a set of related auxiliary tasks having labeled data in addition to data of the at least one target task to generate training examples of a specific transformed text;

training a machine learning model to perform Natural Language Processing (NLP) based upon training data comprising the training examples of the original input text and the training examples of the specific transformed text;

receiving test data comprising a new original input text for the text-to-text transformation; and

transforming the new original input text into a new transformed text using the trained machine learning model.

13 . The computer-implemented method of claim 12 , wherein the generated training examples of the specific transformed text comprises summarizing the multiple sentence question.

14 . The computer-implemented method of claim 12 , wherein the generated training examples of the specific transformed text comprises summarizing and shortening the multiple sentence question.

15 . A computing device configured to improve a machine learning training dataset by performing text-to-text transformation, the computing device comprising:

a processor;

a memory coupled to the processor, the memory storing instructions to cause the processor to:

perform a text-to-text transformation operation on an original input text of a specific task to generate a plurality of transformed text, wherein

the original input text corresponds to a multiple sentence question,

the text-to-text transformation operation includes compression of the multiple sentence question by removal of one or more portions of the multiple sentence question to generate the plurality of transformed text,

each of the one or more portions of the multiple sentence question includes words, phrases, or a sentence of the multiple sentence question, and

the plurality of transformed text corresponds to a summary of the multiple sentence question using fewer words than in the original input text;

apply a task-specific performance metric to measure an operation of the specific task using each of the plurality of transformed text;

pair, based on the application of the task-specific performance metric, each of the plurality of transformed text with the task-specific performance metric; and

update a training dataset to include each pairing of the plurality of transformed text with the task-specific performance metric.

16 . The computing device of claim 15 , wherein

the specific task comprises at least one target task provided in the original input text, and

the instructions further cause the processor to

execution of execute a distant supervision of the text-to-text transformation operation based on a set of related auxiliary tasks having labeled data in addition to data of the specific task.

17 . The computing device of claim 16 , wherein:

the labeled data of the related auxiliary tasks includes a set of question and answer (QA) pairs; and

the instructions further cause the processor to:

generate a plurality of question compression candidates from the set of QA pairs; and

select one or more of the plurality of question compression candidates based on an answer ranking of the set of QA pairs.

18 . The computing device of claim 17 , wherein the plurality of question compression candidates is based on a number of words in the original input text.

19 . The computing device of claim 15 , wherein the instructions further cause the processor to:

perform a new text transformation operation using the updated training dataset to generate a new plurality of transformed text;

apply the task-specific performance metric to measure the operation of the specific task using each of the new plurality of transformed text;

pair, based on the application of the task-specific performance metric, each of the new plurality of transformed text with the task-specific performance metric; and

update the training dataset to include each pairing of the new plurality of transformed text with the task-specific performance metric.