Training and using a model for content moderation of multimodal media
A computerized method trains and uses a multimodal fusion transformer (MFT) model for content moderation. Language modality data and vision modality data associated with a multimodal media source is received. Language embeddings are generated from the language modality data and vision embeddings are generated from the vision modality data. Both kinds of embeddings are generated using operations and/or processes that are specific to the associated modalities. The language embeddings and vision embeddings are combined into combined embeddings and the MFT model is used with those combined embeddings to generate a language semantic output token, a vision semantic output token, and a combined semantic output token. Contrastive loss data is generated using the three semantic output tokens and the MFT model is adjusted using that contrastive loss data. After the MFT model is trained sufficiently, it is configured to perform content moderation operations using semantic output tokens.
1 . A system comprising:
a processor; and
a memory comprising programming instructions for execution by the processor, wherein the programming instructions, upon execution by the processor, cause the system to perform the following operations:
receiving multimodal social media content associated with a social media platform, the multimodal social media content combining language modality data with vision modality data, each of the language modality data and the vision modality data being innocuous alone, wherein a combined meaning of the language modality data and the vision modality data is harmful within a multimodal context;
generating language embeddings from the language modality data using a first embedding operation;
generating vision embeddings from the vision modality data using a second embedding operation;
combining the language embeddings and the vision embeddings into combined embeddings; and
training a multimodal fusion transformer (MFT) model to perform multimodal content moderation, within the multimodal context, for the social media platform using training data that includes the combined embeddings, wherein training the MFT model includes using the MFT model to generate one or more language semantic output tokens, one or more vision semantic output tokens, and one or more combined semantic output tokens from the combined embeddings, determining contrastive loss data based on the one or more language semantic output tokens, the one or more vision semantic output tokens, and the one or more combined semantic output tokens, and adjusting the MFT model based on the contrastive loss data.
2 . The system of claim 1 , wherein determining the contrastive loss data includes:
calculating a first cosine similarity value between the one or more language semantic output tokens and the one or more combined semantic output tokens;
calculating a second cosine similarity value between the one or more vision semantic output tokens and the one or more combined semantic output tokens; and
combining the first cosine similarity value with the second cosine similarity value to form the contrastive loss data.
3 . A method comprising:
receiving multimodal social media content associated with a social media platform, the multimodal social media content combining language modality data with vision modality data, each of the language modality data and the vision modality data being innocuous alone, wherein a combined meaning of the language modality data and the vision modality data is harmful within a multimodal context;
generating language embeddings from the language modality data using a first embedding operation;
generating vision embeddings from the vision modality data using a second embedding operation;
combining the language embeddings and the vision embeddings into combined embeddings; and
training a multimodal fusion transformer (MFT) model to perform multimodal content moderation, within the multimodal context, for the social media platform using training data that includes the combined embeddings, wherein training the MFT model includes using the MFT model to generate one or more language semantic output tokens, one or more vision semantic output tokens, and one or more combined semantic output tokens from the combined embeddings, determining contrastive loss data based on the one or more language semantic output tokens, the one or more vision semantic output tokens, and the one or more combined semantic output tokens, and adjusting the MFT model based on the contrastive loss data.
4 . The method of claim 3 , wherein generating the contrastive loss data includes:
calculating a first cosine similarity value between the one or more language semantic output tokens and the one or more combined semantic output tokens;
calculating a second cosine similarity value between the one or more vision semantic output tokens and the one or more combined semantic output tokens; and
combining the first cosine similarity value with the second cosine similarity value to form the contrastive loss data.
5 . A non-transitory computer storage medium having programming instructions stored thereon that, upon execution by a processor of a system, cause the system to perform the following operations:
receiving multimodal social media content associated with a social media platform, the multimodal social media content combining language modality data with vision modality data, each of the language modality data and the vision modality data being innocuous alone, wherein a combined meaning of the language modality data and the vision modality data is harmful within a multimodal context;
generating language embeddings from the language modality data using a first embedding operation;
generating vision embeddings from the vision modality data using a second embedding operation;
combining the language embeddings and the vision embeddings into combined embeddings; and
training a multimodal fusion transformer (MFT) model to perform multimodal content moderation, within the multimodal context, for the social media platform using training data that includes the combined embeddings, wherein training the MFT model includes using the MFT model to generate one or more language semantic output tokens, one or more vision semantic output tokens, and one or more combined semantic output tokens from the combined embeddings, determining contrastive loss data based on the one or more language semantic output tokens, the one or more vision semantic output tokens, and the one or more combined semantic output tokens, and adjusting the MFT model based on the contrastive loss data.
6 . The system of claim 1 , wherein the combined embeddings include one or more language classification tokens (CLS) tokens generated as an aggregate representation of the one or more language semantic output tokens, one or more vision CLS tokens generated as an aggregate representation of the one or more vision semantic output tokens, and one or more combined CLS tokens generated as an aggregate representation of the one or more language semantic output tokens and the one or more vision CLS tokens.
7 . The system of claim 1 , wherein the contrastive loss data includes domain-specific contrastive loss data associated with a content moderation domain, and wherein the domain- specific contrastive loss data is used to train the MFT model is trained to generate domain- specific semantic output tokens associated with the content moderation domain.
8 . The system of claim 1 , wherein the contrastive loss data includes Masked Language Modeling (MLM) loss data.
9 . The system of claim 8 , wherein the MLM loss data is used to train the MFT model to determine masked or missing text tokens based on surrounding text tokens in the training data.
10 . The system of claim 8 , wherein the MLM loss data is generated by masking a text token of the language embeddings.
11 . The system of claim 1 , wherein the contrastive loss data includes Masked region of interest (RoI) loss data.
12 . The system of claim 11 , wherein the Masked RoI loss data is used to train the MFT model to predict visual information in a bounding box based on the one or more language semantic output tokens.
13 . The system of claim 11 , wherein the Masked RoI loss data is generated by masking a region of interest (RoI) bounding box of the vision embeddings to produce masked vision embeddings.
14 . The system of claim 13 , wherein the masked vision embeddings include a feature map embedding associated with the RoI bounding box and an object category associated with the RoI bounding box.
15 . The system of claim 1 , wherein the contrastive loss data includes Image-Text Matching (ITM) loss data.
16 . The system of claim 15 , wherein the ITM loss data is used to train the MFT model to determine whether text and image data from the training data match semantically.
17 . The system of claim 15 , wherein the ITM loss data is computed by the MFT model upon determining that the language modality data and the vision modality data originate from a common source.