Machine learning model for identifying offensive, computer-generated natural-language text or speech

1. A method, comprising:

obtaining, with one or more processors, a training set of n-grams labeled as offensive;

causing, with one or more processors, a machine learning model to be trained based on the training set of n-grams, wherein the machine learning model, when trained, is configured to classify natural language text as offensive or non-offensive;

obtaining, with one or more processors, input natural language text expressing a computer-generated utterance;

classifying, with one or more processors, after causing training, the computer-generated utterance as offensive or non-offensive using the machine learning model; and

causing, with one or more processors, an output to be provided to a recipient, the output being based on whether the machine learning model classifies the computer-generated utterance as offensive or non-offensive.

2. The method of claim 1 , wherein:

at least some of the n-grams in the training set comprise five or more words;

each of the at least some of the n-grams in the training set have a plurality of labels indicating demographic attributes of a population for which a respective n-gram is designated as offensive;

at least some of the n-grams are labeled as offensive for some populations but not other populations;

each n-gram in the training set comprises natural language text obtained from a corpus of unstructured natural language text documents;

the machine learning model comprises a supervised machine learning model; and

the computer-generated utterance comprises an utterance generated as an output of an utterance generation engine, the utterance generation engine comprising one or more of: a chatbot, a machine translation system, a text summarization model, or a speech-to-text (“STT”) processing system.

3. The method of claim 1 , wherein causing the machine learning model to be trained comprises:

training the machine learning model based on a first sub-set of the training set of n-grams; and

cross-validating the trained machine-learning model based on a second sub-set of the training set of n-grams, the second sub-set being at least partially different from the first sub-set.

4. The method of claim 1 , wherein obtaining the training set of n-grams comprises:

parsing a plurality of n-grams from a corpus of unstructured natural language text;

determining an offensiveness score for each n-gram based on natural-language processing semantic analysis of responses to usage of the n-gram in the corpus, the offensiveness score indicating a likelihood that an audience reaction to the n-gram will be negative; and

mapping offensiveness scores or values based on the offensiveness scores to corresponding n-grams as labels.

5. The method of claim 1 , wherein classifying the comprises:

determining, using the machine learning model, an offensiveness score for the computer-generated utterance, the offensiveness score indicating a likelihood that an audience reaction to the computer-generated utterance will be negative;

determining whether the offensiveness score satisfies an offensiveness score threshold criterion, wherein:

the machine learning model is configured to classify the computer-generated utterance as offensive in response to determining that the offensiveness score satisfies the offensiveness score threshold criterion, and

the machine learning model is configured to classify the computer-generated utterance as non-offensive in response to determining that the offensiveness score does not satisfy the offensiveness score threshold criterion.

6. The method of claim 1 , further comprising:

classifying the computer-generated utterance as offensive;

selecting or generating, in response to classifying the computer-generated utterance as offensive, a different computer-generated utterance determined to be semantically similar to the computer-generated utterance, wherein the output is based on the different computer-generated utterance.

7. The method of claim 1 , wherein the computer-generated utterance comprises a phrase absent from the training set of n-grams.

8. The method of claim 1 , further comprising:

determining that the machine learning model classifies the computer-generated utterance as offensive;

generating a plurality of computer-generated utterances that are each determined to be semantically similar to the computer-generated utterance;

determining a ranking of the plurality of computer-generated utterances according to semantic similarity; and

selecting a highest ranking one of the ranking of the plurality of computer-generated utterances that is not classified as offensive by the machine learning model.

9. The method of claim 1 , further comprising:

obtaining a plurality of demographic attributes or geographic attributes of the recipient that is to be provided the computer-generated utterance, wherein:

classifying the computer-generated utterance comprises classifying the computer-generated utterance as offensive is based on the plurality of demographic attributes or geographic attributes; and

the machine learning model is configured to classify the computer-generated utterance differently for a different recipient having one or more different demographic attributes or geographic attributes.

10. The method of claim 1 , wherein causing the machine learning model to be trained comprises training the machine learning model, the method further comprises:

re-training the machine learning model based on feedback from classifications by the machine learning model.

11. The method of claim 1 , further comprising:

obtaining features of each of at least some of the n-grams, the features comprising: demographic information of a provider of a response to usage of the corresponding n-gram indicative of offensiveness, and temporal metadata indicating a time that the response to usage of the corresponding n-gram indicative of offensiveness was generated, wherein:

training is based on the features, and

the machine learning model is configured accept the features of the recipient and classify the computer-generated utterance based on the features of the recipient.

12. The method of claim 1 , wherein:

classification of the computer-generated utterance is based on cosine angle, Minkowski distance, or Euclidean distance between a feature vector of the computer-generated utterance and a feature vector of one or more of the n-grams in the training set labeled as offensive; and

the feature vector of the computer-generated utterance and the feature vector of one or more of the n-grams have dimensions corresponding to term-frequency inverse document frequency (“TFIDF”) scores of different terms among the n-grams in the training set.

13. The method of claim 1 , further comprising:

developing the machine learning model;

deploying the machine learning model to one or more testing systems to test functionalities of the machine learning model; and

releasing the machine learning model to one or more consumer systems to be used to classify inputs as offensive or non-offensive.

14. The method of claim 1 , wherein the machine learning model is configured to classify the computer-generated utterance as offensive based on one or more features of a recipient of the computer-generated utterance, the method further comprises:

identifying one or more additional features of a different intended recipient of the computer-generated utterance; and

classifying the computer-generated utterance as non-offensive based on the one or more additional features.

15. The method of claim 1 , wherein the machine learning model comprises a software-development tool or a suite of software-development tools.

16. The method of claim 1 , further comprising:

providing the output to the machine learning model with an exogenous classification of the computer-generated utterance as offensive or non-offensive; and

causing the machine learning model to be further trained based on the output and the exogenous classification.

17. A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors effectuate operations comprising:

obtaining, with one or more processors, a training set of n-grams labeled as offensive;

causing, with one or more processors, a machine learning model to be trained based on the training set of n-grams, wherein the machine learning model, when trained, is configured to classify natural language text as offensive or non-offensive;

obtaining, with one or more processors, input natural language text expressing a computer-generated utterance;

classifying, with one or more processors, after causing training, the computer-generated utterance as offensive or non-offensive using the machine learning model; and

causing, with one or more processors, an output to be provided to a recipient, the output being based on whether the machine learning model classifies the computer-generated utterance as offensive or non-offensive.

18. The medium of claim 17 , wherein obtaining the training set of n-grams comprises:

parsing a plurality of n-grams from a corpus of unstructured natural language text;

determining an offensiveness score for each n-gram based on natural-language processing semantic analysis of responses to usage of the n-gram in the corpus, the offensiveness score indicating a likelihood that an audience reaction to the n-gram will be negative; and

mapping offensiveness scores or values based on the offensiveness scores to corresponding n-grams as labels.

19. The medium of claim 17 , wherein the instructions when executed by one or more processors further effectuate operations comprising:

determining that the machine learning model classifies the computer-generated utterance as offensive;

generating a plurality of computer-generated utterances that are each determined to be semantically similar to the computer-generated utterance;

determining a ranking of the plurality of computer-generated utterances according to semantic similarity; and

selecting a highest ranking one of the ranking of the plurality of computer-generated utterances that is not classified as offensive by the machine learning model.

20. The medium of claim 17 , wherein the instructions when executed by one or more processors further effectuate operations comprising:

obtaining a plurality of demographic attributes or geographic attributes of the recipient that is to be provided the computer-generated utterance, wherein:

classifying the computer-generated utterance comprises classifying the computer-generated utterance as offensive is based on the plurality of demographic attributes or geographic attributes; and

the machine learning model is configured to classify the computer-generated utterance differently for a different recipient having one or more different demographic attributes or geographic attributes.