Low-resource multilingual machine learning framework
Systems and methods for performing machine learning on multilingual text data.
1. A method for multilingual machine learning on text data, comprising:
receiving English-language labeled training data comprising at least one indication of human conduct that violates a policy, ethical standard, and/or law;
applying a pre-trained cross-lingual model to the received labeled training data;
applying zero-shot machine learning to the received labeled training data to generate supervised learning task specific layers;
producing a zero-shot machine learning model comprising applying the pre-trained cross-lingual model to input data to generate an output and applying the supervised learning task specific layers to the output to generate a classification; and
applying the zero-shot machine learning model to unlabeled unseen text data to perform functions including the classification of unseen text data to detect at least one identification of a potential violation condition.
2. The method of claim 1 , wherein the text data corresponds to one or more electronic communications between persons.
3. The method of claim 2 , wherein the text classification includes identifying the unseen text data as corresponding to conduct of the persons.
4. The method of claim 1 , wherein the cross-lingual model is pre-trained based on one or more spoken languages as represented in labeled training data.
5. The method of claim 1 , wherein the text data comprises at least one of a sentence, paragraph, or document.
6. A method for multilingual machine learning on text data, comprising:
receiving labeled and/or unlabeled English training data comprising at least one indication of human conduct that violates a policy, ethical standard, and/or law;
receiving labeled and/or unlabeled non-English training data;
applying a pre-trained cross-lingual model to the received training data to generate supervised learning task specific layers;
generating a semi-supervised multi-lingual model comprising applying the pre-trained cross-lingual model to input data to generate an output and applying the supervised learning task specific layers to the output to generate a classification; and
applying the semi supervised multi-lingual model to unseen multi-lingual text data to perform functions including the classification of unseen text data to detect at least one identification of a potential violation condition.
7. The method of claim 6 , wherein the text data corresponds to one or more electronic communications between persons.
8. The method of claim 6 , wherein the unseen multi-lingual text data comprises at least one of a sentence, paragraph, or document that includes multiple languages.
9. A method for multilingual machine learning on text data, comprising:
receiving labeled data, unlabeled data, multi-lingual data, and human annotated data;
applying a selection criteria;
based at least in part on the selection criteria, providing text data to one or more annotators to perform one or more functions that include at least one of assigning a label to the text data, evaluating semantic similarity, or translating the text data;
generating a multi-lingual model based at least in part on the functions performed by the one or more annotators; and
applying the multi-lingual model to unseen text data to produce a pre-trained cross-lingual model.
10. The method of claim 9 , further comprising using an annotator allocation resource cost and/or budget to determine the one or more functions performed by the one or more annotators.
11. The method of claim 10 , wherein a number and/or type of the one or more annotators is based at least in part on the resource cost and/or budget.
12. The method of claim 9 , wherein a type of the one or more annotators comprises an English-speaking subject matter expert or a multi-lingual subject matter expert.
13. The method of claim 9 , wherein the selection criteria comprises at least one of classifier confusion, data space coverage, colloquial language, domain-specific language, or idiomatic language.
14. A system for multilingual machine learning on text data, comprising:
a plurality of cross-lingual encoders coupled to a corresponding plurality of neural networks; and
an attention-based ensemble encoder component configured to receive an output of the plurality of neural networks, wherein the system is configured to classify the outputs of the plurality of neural networks by detecting at least one identification of a potential violation condition.
15. The system of claim 14 , further comprising a component configured for performing concatenation, received from the attention-based ensemble encoder.
16. The system of claim 14 , further comprising a fully connected neural network configured to produce a cross-lingual sentence representation.
17. The system of claim 14 , wherein an output of each of the plurality of cross-lingual encoders is input into a corresponding one of the corresponding plurality of neural networks.