Multilingual, context-aware machine learning model configured for profanity detection and mitigation
Certain aspects of the disclosure relate to profanity detection and mitigation. A method generally includes training a machine learning (ML) model using labeled training data instances by, for each training data instance: providing the tokens of the respective training data instance to an input layer of the ML model; receiving a first output for each token of the respective training data instance classifying the respective token as a profanity-containing or a non-profanity-containing token; receiving a second output for the respective training data instance classifying the respective training data instance as a profanity-containing or a non-profanity-containing instance; determining a loss value based on the first output for each token and the second output using a loss function comprising a regularization term configured to increase loss based on disagreement between the first output for each token and the second output; and modifying parameter(s) of the ML model based on the loss value.
1 . A method of training a machine learning (ML) model to detect profanity, comprising:
labeling each of a plurality of training data instances with a first label, the first label comprising a profanity-containing instance label or a non-profanity-containing instance label;
labeling each token of a plurality of tokens for each of the plurality of training data instances with a second label, the second label comprising a profanity-containing token label or a non-profanity-containing token label; and
training the ML model by, for each respective training data instance of the plurality of training data instances:
providing the plurality of tokens of the respective training data instance to an input layer of the ML model;
receiving a first output for each of the plurality of tokens of the respective training data instance classifying a respective token as a profanity-containing token or a non-profanity-containing token;
receiving a second output for the respective training data instance classifying the respective training data instance as a profanity-containing instance or a non-profanity-containing instance;
determining a loss value based on the first output for each of the plurality of tokens and the second output using a loss function comprising a regularization term configured to increase loss based on disagreement between the first output for each of the plurality of tokens and the second output; and
modifying one or more parameters of the ML model based on the loss value.
2 . The method of claim 1 , wherein labeling each token of the plurality of tokens for each of the plurality of training data instances with a second label comprises, for one or more tokens of the plurality of tokens:
determining that each of the one or more tokens are not included in a list of known profanity tokens, wherein the list of known profanity tokens comprises a plurality of profanity tokens in two or more languages; and
based on each of the one or more tokens not being included in the list of known profanity tokens, labeling each of the one or more tokens with the non-profanity-containing token label.
3 . The method of claim 1 , wherein labeling each token of the plurality of tokens for each of the plurality of training data instances with a second label comprises, for one or more tokens of the plurality of tokens:
determining that each of the one or more tokens are included in a list of known profanity tokens, wherein the list of known profanity tokens comprises a plurality of profanity tokens in two or more languages; and
based on each of the one or more tokens being included in the list of known profanity tokens, for each respective token of the one or more tokens:
determining whether the first label associated with the training data instance comprising the respective token is the profanity-containing instance label or the non-profanity-containing instance label;
when the first label associated with the training data instance comprising the respective token is the profanity-containing instance label, labeling the respective token with the profanity-containing token label; and
when the first label associated with the training data instance comprising the respective token is the non-profanity-containing instance label, labeling the respective token with the non-profanity-containing token label.
4 . The method of claim 1 , further comprising:
obtaining a first set of first training data instances in a first language; and
for each first training data instance in the first set of first training data instances, generating, via machine translation, a second training data instance to create a second set of second training data instances in a second language, wherein the plurality of training data instances comprises the first set of first training data instances and the second set of second training data instances.
5 . The method of claim 4 , wherein labeling each of a plurality of training data instances with the first label comprises:
labeling each of the first set of first training data instances with the first label; and
for each respective first training data instance, labeling the second training data instance generated based on the respective first training data instance with a same first label as the respective first training data instance.
6 . The method of claim 1 , wherein, for each training data instance of the plurality of training data instances, receiving the first output for each of the plurality of tokens of the respective training data instance comprises receiving the first output for each of the plurality of tokens as a sequence of tokens.
7 . The method of claim 1 , wherein the ML model comprises an encoder only transformer architecture.
8 . The method of claim 7 , wherein the encoder only transformer architecture comprises a decoding-enhanced bidirectional encoder representations from transformers with disentangled attention (DeBERTa) model.
9 . A method of profanity detection, comprising:
receiving a first input sentence comprising a first plurality of tokens;
processing, with a machine learning (ML) model trained for sentence-level profanity classification and token-level profanity classification using a loss function comprising a regularization term configured to increase loss based on disagreement between a sentence-level profanity classification and a token-level profanity classification generated for a same input sentence, the first input sentence comprising the first plurality of tokens and thereby generating a first classification output for the first input sentence, the first classification output comprising a profanity-containing sentence classification;
determining the first classification output for the first input sentence comprises the profanity-containing sentence classification; and
based on the determination, processing with the ML model the first input sentence comprising the first plurality of tokens and thereby generating a second classification output for each of the first plurality of tokens of the first input sentence, wherein the second classification output generated for at least one token of the first plurality of tokens comprises a profanity-containing token classification.
10 . The method of claim 9 , further comprising:
concealing the at least one token in the first input sentence; and
using the first input sentence comprising the at least one token, which has been concealed, to perform one or more tasks.
11 . The method of claim 9 , further comprising:
receiving a second input sentence comprising a second plurality of tokens;
processing, with the ML model, the second input sentence comprising the second plurality of tokens and thereby generating a third classification output for the second input sentence, the third classification output comprising a non-profanity-containing sentence classification;
determining the third classification output for the second input sentence comprises the non-profanity-containing sentence classification; and
based on the determination, using the second input sentence to perform one or more tasks.
12 . The method of claim 9 , further comprising training the ML model for the sentence-level profanity classification and the token-level profanity classification using a plurality of training data instances, wherein training the ML model comprises, for each respective training data instance of the plurality of training data instances:
providing a plurality of tokens of the respective training data instance to an input layer of the ML model;
receiving a first output for each of the plurality of tokens of the respective training data instance classifying the respective token as a profanity-containing token or a non-profanity-containing token;
receiving a second output for the respective training data instance classifying the respective training data instance as a profanity-containing instance or a non-profanity-containing instance;
determining a loss value based on the first output for each of the plurality of tokens and the second output using the loss function comprising the regularization term; and
modifying one or more parameters of the ML model based on the loss value.
13 . The method of claim 12 , further comprising, prior to training the ML model:
labeling each of the plurality of training data instances with a first label, the first label comprising a profanity-containing instance label or a non-profanity-containing instance label; and
labeling each token of a plurality of tokens for each of the plurality of training data instances with a second label, the second label comprising a profanity-containing token label or a non-profanity-containing token label.
14 . The method of claim 13 , wherein labeling each token of the plurality of tokens for each of the plurality of training data instances with a second label comprises, for one or more tokens of the plurality of tokens:
determining that each of the one or more tokens are not included in a list of known profanity tokens, wherein the list of known profanity tokens comprises a plurality of profanity tokens in two or more languages; and
based on each of the one or more tokens not being included in the list of known profanity tokens, labeling each of the one or more tokens with the non-profanity-containing token label.
15 . The method of claim 13 , wherein labeling each token of the plurality of tokens for each of the plurality of training data instances with a second label comprises, for one or more tokens of the plurality of tokens:
determining that each of the one or more tokens are included in a list of known profanity tokens, wherein the list of known profanity tokens comprises a plurality of profanity tokens in two or more languages; and
based on each of the one or more tokens being included in the list of known profanity tokens, for each respective token of the one or more tokens:
determining whether the first label associated with the training data instance comprising the respective token is the profanity-containing instance label or the non-profanity-containing instance label;
when the first label associated with the training data instance comprising the respective token is the profanity-containing instance label, labeling the respective token with the profanity-containing token label; and
when the first label associated with the training data instance comprising the respective token is the non-profanity-containing instance label, labeling the respective token with the non-profanity-containing token label.
16 . The method of claim 13 , further comprising, prior to training the ML model:
obtaining a first set of first training data instances in a first language; and
for each first training data instance in the first set of first training data instances, generating, via machine translation, a second training data instance to create a second set of second training data instances in a second language, wherein the plurality of training data instances comprises the first set of first training data instances and the second set of second training data instances.
17 . The method of claim 16 , wherein labeling each of a plurality of training data instances with the first label comprises:
labeling each of the first set of first training data instances with the first label; and
for each respective first training data instance, labeling the second training data instance generated based on the respective first training data instance with a same first label as the respective first training data instance.
18 . The method of claim 9 , wherein the ML model comprises an encoder only transformer architecture.
19 . The method of claim 18 , wherein the encoder only transformer architecture comprises a decoding-enhanced bidirectional encoder representations from transformers with disentangled attention (DeBERTa) model.
20 . A processing system, comprising:
one or more memories comprising computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions and cause the processing system to:
label each of a plurality of training data instances with a first label, the first label comprising a profanity-containing instance label or a non-profanity-containing instance label;
label each token of a plurality of tokens for each of the plurality of training data instances with a second label, the second label comprising a profanity-containing token label or a non-profanity-containing token label; and
train a machine learning (ML) model to detect profanity, wherein to train the ML model comprises to, for each respective training data instance of the plurality of training data instances:
provide the plurality of tokens of the respective training data instance to an input layer of the ML model;
receive a first output for each of the plurality of tokens of the respective training data instance classifying a respective token as a profanity-containing token or a non-profanity-containing token;
receive a second output for the respective training data instance classifying the respective training data instance as a profanity-containing instance or a non-profanity-containing instance;
determine a loss value based on the first output for each of the plurality of tokens and the second output using a loss function comprising a regularization term configured to increase loss based on disagreement between the first output for each of the plurality of tokens and the second output; and
modify one or more parameters of the ML model based on the loss value.