Context-aware artificial intelligence system for content moderation
A method can include determining, via a multi-channel text model, a text profanity score for a textual content for an item. The method further can include determining, via a vision model, a vision profanity score for an image content for the item. In addition, the method can include determining whether the text profanity score exceeds a text blocking score. Moreover, the method can include determining whether the vision profanity score exceeds a vision blocking score. The method further can include upon determining that the text profanity score exceeds the text blocking score or that the vision profanity score exceeds the vision blocking score, setting a blocking label for the item in an item database as blocked. Other embodiments are disclosed.
1 . A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computing instructions configured to, when run on the one or more processors, cause the one or more processors to perform:
determining, via a multi-channel text model, a combined text profanity score for textual content of an online promotion associated with an item, wherein the multi-channel text model comprises:
a first channel text model that generates one or more text embeddings for the textual content and generates a first text profanity score for the textual content from the one or more text embeddings;
a second channel text model that generates a respective text profanity degree for each of a plurality of profanity categories, wherein the respective text profanity degree is indicative of the textual content being classified into a respective profanity category, wherein the second channel text model is different from the first channel text model; and
a meta-learner model that generates the combined text profanity score for the textual content by combining the first text profanity score with each of the respective text profanity degrees;
determining, via a vision model, a vision profanity score for image content of the online promotion, wherein the vision model generates one or more vision tensors for the image content and determines the vision profanity score from the vision tensors;
determining whether the combined text profanity score exceeds a text blocking score;
determining whether the vision profanity score exceeds a vision blocking score; and
upon determining that the combined text profanity score exceeds the text blocking score or that the vision profanity score exceeds the vision blocking score, setting a blocking label for the item in an item database as blocked, wherein the item is blocked or unblocked in the online promotion based on the blocking label.
2 . The system in claim 1 , wherein the computing instructions are further configured to cause the one or more processors to perform one or more of:
training the multi-channel text model to generate the combined text profanity score for the textual content of the online promotion based on one or more labeled comment databases, comprising:
training the first channel text model to generate the first text profanity score for the textual content based on the one or more labeled comment databases; and
training the second channel text model to generate the respective text profanity degree for the textual content for each of profanity categories based on the one or more labeled comment databases, wherein:
the combined text profanity score is determined based on the first text profanity score and the respective text profanity degree for each of the profanity categories; or
training the vision model to generate the vision profanity score for the image content based on one or more domain-specific labeled image databases.
3 . The system in claim 2 , wherein:
training the multi-channel text model further comprises, before training the first channel text model and before training the second channel text model, removing one or more unwanted elements from training text data in the one or more labeled comment databases.
4 . The system in claim 2 , wherein:
training the vision model further comprises one or more of:
augmenting training image data in the one or more domain-specific labeled image databases, comprising:
transforming the training image data into varied image data; and
storing the varied image data in the one or more domain-specific labeled image databases;
stratifying the training image data in the one or more domain-specific labeled image databases based on one or more of item types or image labels for the training image data; or
training the vision model using multi-fold cross validation.
5 . The system in claim 1 , wherein the first channel text model comprises:
a first text embedding generator that generates a first text embedding for the textual content of the online promotion;
a second text embedding generator configured to generate a second text embedding for the textual content of the online promotion; and
one or more lightweight regressor models that generates the first text profanity score for the textual content of the online promotion based on:
the first text embedding, as generated by the first text embedding generator; and
the second text embedding, as generated by the second text embedding generator.
6 . The system in claim 5 , wherein the one or more lightweight regressor models are further configured to:
cause the one or more processors to perform at least two different regression algorithms, each of the at least two different regression algorithms generates a respective text score for the textual content of the online promotion; and
determine the first text profanity score for the textual content of the online promotion further based on the respective text score, as generated by each of the at least two different regression algorithms.
7 . The system in claim 5 , wherein:
the one or more lightweight regressor models are each trained using multi-fold cross validation.
8 . The system in claim 5 , wherein one or more of:
the first text embedding generator is further configured to cause the one or more processors to perform a context-aware embedding algorithm; or
the second text embedding generator is further configured to cause the one or more processors to perform a statistics-based embedding algorithm.
9 . The system in claim 1 , wherein the vision model comprises an ensemble learning model comprising:
a first vision model that generates a first tensor;
a second vision model that generates a second tensor; and
a linear layer that generates the vision profanity score based on the first tensor and the second tensor.
10 . The system in claim 1 , wherein the online promotion comprises a social media promotion comprising the textual content and the image content, wherein the item is content of the social media promotion that is promoted by the social media promotion.
11 . The system in claim 1 , wherein the item is associated with metadata in the item database, wherein the blocking label is set in the metadata.
12 . A method being implemented via execution of computing instructions configured to run at one or more processors and stored at one or more non-transitory computer-readable media, the method comprising:
determining, via a multi-channel text model, a combined text profanity score for textual content of an online promotion associated with an item, wherein the multi-channel text model comprises:
a first channel text model that generates one or more text embeddings for the textual content and generates a first text profanity score for the textual content from the one or more text embeddings;
a second channel text model that generates a respective text profanity degree for each of a plurality of profanity categories, wherein the respective text profanity degree is indicative of the textual content being classified into a respective profanity category, wherein the second channel text model is different from the first channel text model; and
a meta-learner model that generates the combined text profanity score for the textual content by combining the first text profanity score with each of the respective text profanity degrees;
determining, via a vision model, a vision profanity score for image content of the online promotion, wherein the vision model generates one or more vision tensors for the image content and determines the vision profanity score from the vision tensors;
determining whether the combined text profanity score exceeds a text blocking score;
determining whether the vision profanity score exceeds a vision blocking score; and
upon determining that the combined text profanity score exceeds the text blocking score or that the vision profanity score exceeds the vision blocking score, setting a blocking label for the item in an item database as blocked, wherein the item is blocked or unblocked in the online promotion based on the blocking label.
13 . The method in claim 12 , further comprising one or more of:
training the multi-channel text model to generate the combined text profanity score for the textual content of the online promotion based on one or more labeled comment databases, comprising:
training the first channel text model to generate the first text profanity score for the textual content based on the one or more labeled comment databases; and
training the second channel text model to generate the respective text profanity degree for the textual content for each of profanity categories based on the one or more labeled comment databases, wherein:
the combined text profanity score is determined based on the first text profanity score and the respective text profanity degree for each of the profanity categories; or
training the vision model to generate the vision profanity score for the image content based on one or more domain-specific labeled image databases.
14 . The method in claim 13 , wherein:
training the multi-channel text model further comprises, before training the first channel text model and before training the second channel text model, removing one or more unwanted elements from training text data in the one or more labeled comment databases.
15 . The method in claim 13 , wherein:
training the vision model further comprises one or more of:
augmenting training image data in the one or more domain-specific labeled image databases, comprising:
transforming the training image data into varied image data; and
storing the varied image data in the one or more domain-specific labeled image databases;
stratifying the training image data in the one or more domain-specific labeled image databases based on one or more of item types or image labels for the training image data; or
training the vision model using multi-fold cross validation.
16 . The method in claim 12 , wherein the first channel text model comprises:
a first text embedding generator that generates a first text embedding for the textual content of the online promotion;
a second text embedding generator that generates a second text embedding for the textual content of the online promotion; and
one or more lightweight regressor models that generates the first text profanity score for the textual content of the online promotion based on:
the first text embedding, as generated by the first text embedding generator; and
the second text embedding, as generated by the second text embedding generator.
17 . The method in claim 16 , wherein the one or more lightweight regressor models are further configured to:
cause the one or more processors to perform at least two different regression algorithms, each of the at least two different regression algorithms generates a respective text score for the textual content of the online promotion; and
determine the first text profanity score for the textual content of the online promotion further based on the respective text score, as generated by each of the at least two different regression algorithms.
18 . The method in claim 16 , wherein:
the one or more lightweight regressor models are each trained using multi-fold cross validation.
19 . The method in claim 16 , wherein one or more of:
the first text embedding generator is further configured to cause the one or more processors to perform a context-aware embedding algorithm; or
the second text embedding generator is further configured to cause the one or more processors to perform a statistics-based embedding algorithm.
20 . The method in claim 12 , wherein the vision model comprises an ensemble learning model comprising:
a first vision model that generates a first tensor;
a second vision model that generates a second tensor; and
a linear layer that generates the vision profanity score based on the first tensor and the second tensor.