Proficiency and native language-adapted grammatical error correction
In an embodiment, the disclosed technologies are capable of receiving, by a digital model, data representing a first text sequence in a first language; using the digital model, modifying the first text sequence to result in creating and digitally storing a second text sequence in the first language; and outputting, by the digital model, the second text sequence in the first language. The modifying may include any one or more of: deleting text from the first text sequence, adding text to the first text sequence, modifying text of the first text sequence, reordering text of the first text sequence, adding a digital markup to the first text sequence. The digital model may have been fine-tuned, after having been machine-learned, using a subset of values of model parameters associated with an encoding layer or an embedding layer or both the encoding layer and the embedding layer.
1. A method comprising:
receiving under digital program control, by a digital model, electronic digital data representing a first text sequence in a first language, wherein the digital model comprises a fluency-adjusted grammatical error correction model;
generating, by the fluency-adjusted grammatical error correction model, a second text sequence in the first language which is grammatically corrected and fluency adjusted, wherein the fluency-adjusted grammatical error correction model is trained by:
training the fluency-adjusted grammatical error correction model using domain-independent training data that comprises a set of uncorrected text items and for each uncorrected text item, a corresponding corrected text item, wherein the domain-independent training data comprises text sequences relating to a plurality of different topics and writing styles received from native and non-native speakers of various native backgrounds, the fluency-adjusted grammatical error correction model comprising a plurality of artificial neural network layers and model parameters associated with artificial neural network layers;
training the fluency-adjusted grammatical error correction model using in-domain training data, wherein the training the fluency-adjusted grammatical error correction model using in-domain training data adjusts only a subset of values of the model parameters associated with an encoding layer or an embedding layer or both the encoding layer and the embedding layer, the in-domain training data comprising a set of text sequences and, for each text sequence, a set of corresponding features, the set of corresponding features comprising a proficiency label and a native language label; and
outputting, by the digital model, the second text sequence in the first language.
2. The method of claim 1 , further comprising creating the set of text sequences, the set of uncorrected text items, and corresponding corrected text items in the first language.
3. The method of claim 1 , further comprising creating the proficiency label using a stored digital value of a Common European Framework of Reference for Languages (CEFR) proficiency level value.
4. The method of claim 1 , further comprising creating the native language label using a stored digital value that identifies a native language associated with a spoken text sequence of the set of text sequences.
5. The method of claim 1 , further comprising using, as the first language, a language comprising words usable for human-to-human communication.
6. The method of claim 1 , further comprising receiving, from a graphical user interface, text input comprising the first text sequence, and outputting, to the graphical user interface, text output comprising the second text sequence.
7. The method of claim 6 , further comprising creating the first text sequence by segmenting the text input into at least two sub-word units.
8. A method for training a fluency-adjusted grammatical error correction model, the method comprising:
training the fluency-adjusted grammatical error correction model using a first dataset from text sequences relating to a plurality of different topics and writing styles received from native and non-native speakers of various native backgrounds, the first dataset comprising domain-independent training data that comprises a set of uncorrected text items and for each uncorrected text item, a corresponding corrected text item, the fluency-adjusted grammatical error correction model comprising a plurality of artificial neural network layers and model parameters associated with the plurality of artificial neural network layers; and
training on the fluency-adjusted grammatical error correction model using a second dataset, the second dataset comprising in-domain training data that comprises a set of text sequences and, for each text sequence, a set of corresponding features, the set of corresponding features comprising a proficiency label and a native language label, wherein the training on the fluency-adjusted grammatical error correction model using a second dataset adjusts only a subset of values of the model parameters associated with an encoding layer or an embedding layer or both the encoding layer and the embedding layer.
9. The method of claim 8 , further comprising creating, for each text sequence, a text sequence of the set of text sequences by segmenting text of the text sequence into at least two sub-word units.
10. The method of claim 8 , further comprising creating the fluency-adjusted grammatical error correction model using a recurrent neural network.
11. The method of claim 8 , further comprising creating the fluency-adjusted grammatical error correction model using an encoder-decoder neural network with an attention mechanism and at least one long term short term memory (LSTM) unit.
12. The method of claim 8 , wherein at least one type of error is present in the text sequences and location of the error within the text sequences for the in-domain training data.
13. The method of claim 8 , further comprising fine-tuning the fluency-adjusted grammatical error correction model using a transfer learning method for neural networks.
14. A method, comprising:
receiving, by a digital model, an input text sequence in a first language, wherein the digital model comprises a fluency-adjusted grammatical error correction model;
generating, by the fluency-adjusted grammatical error correction model, an output text sequence in the first language which is grammatically corrected and fluency adjusted, wherein the fluency-adjusted grammatical error correction model is trained by:
training the fluency-adjusted grammatical error correction model using a first data set that comprises a set of uncorrected text sequences and for each uncorrected text sequence, a corresponding corrected text sequence, wherein the first data set comprises text sequences relating to a plurality of different topics and writing styles received from native and non-native speakers of various native backgrounds, the set of uncorrected text sequences comprising at least one word produced by a person whose first native language is different from the first language; and
training the fluency-adjusted grammatical error correction model using an in-domain training data set, wherein the training the fluency-adjusted grammatical error correction model adjusts values of model parameters associated with only a subset of the digital model after being trained using the first data set, the in-domain training data set comprising a set of text sequences and, for each text sequence, a set of corresponding features comprising a proficiency label and a native language label wherein at least one type of error is present in a first text sequence and location of the error within the first text sequence for the in-domain training data set;
generating, by the digital model, the output text sequence in the first language, the output text sequence comprising the input text sequence modified for grammatical correction and fluency adjustment of the first text sequence based on a particular native language and proficiency level; and
outputting, by the digital model, the output text sequence in the first language.
15. The method of claim 14 , further comprising fine tuning only an encoding layer or only an embedding layer or only both the encoding layer and the embedding layer of the digital model.
16. The method of claim 14 , further comprising using, as the proficiency label, a digital value that corresponds to a Common European Framework of Reference for Languages (CEFR) proficiency level.
17. The method of claim 14 , further comprising using, as the native language label, a digital value that corresponds to a second native language of a speaker associated with a text sequence of the set of text sequences.
18. The method of claim 14 , further comprising using, as the first language, a language that comprises words usable for human-to-human communication.
19. The method of claim 14 , further comprising receiving, from a graphical user interface, text input comprising the input text sequence, and outputting, to the graphical user interface, text output comprising the output text sequence.
20. The method of claim 19 , further comprising creating the input text sequence by segmenting the text input into at least two sub-word units.