Graph-based labeling of heterogenous digital content items
Technologies for graph-based labeling of digital content items include, in some embodiments, for digital content items received from user systems by an application system, generating and storing a content graph. The content graph can include labeled nodes that correspond to digital content items that have labels, unlabeled nodes that correspond to digital content items that do not have labels, and edges that indicate relationships between content items. Edge data for an edge between an unlabeled node and an adjacent node can be retrieved from the content graph. Responsive to a set of inputs that includes the retrieved edge data and embedding data associated with the unlabeled node, a machine learning model trained on labeled nodes and edges of the content graph can assign a label to the unlabeled node.
1 . A method comprising:
for a set of digital content items received from user systems by an application system, generating and storing a content graph, wherein:
the application system is capable of distributing digital content to the user systems over a network based on a social graph and a set of spam labels; and
the content graph comprises labeled nodes that correspond to a subset of the set of digital content items that have spam labels, unlabeled nodes that correspond to a subset of the set of digital content items that do not have spam labels, and edges nodes of the content graph;
querying the content graph to retrieve edge data for an edge between an unlabeled node and an adjacent node, wherein:
the unlabeled node corresponds to a digital content item that does not have a spam label;
the adjacent node corresponds to a different digital content item; and
the edge data indicates a relationship between the unlabeled node and the adjacent node; and
responsive to a set of inputs comprising the retrieved edge data and embedding data associated with the unlabeled node, by a machine learning model, assigning a label of the set of spam labels to the unlabeled node, wherein:
the embedding data comprises output of a cross-modality embedding model;
the output of the cross-modality embedding model is connected to input of the machine learning model;
the machine learning model comprises a graph transformer-based neural network having connected modality-specific layers;
the graph transformer-based neural network is trained to determine inter-content item relationships based on the labeled nodes and the edges of the content graph; and
the label is assigned to the unlabeled node by the connected modality-specific layers of the trained graph transformer-based neural network performing graph-based propagation of label predictions from labeled nodes to unlabeled nodes of different modalities based on edge relationships between the nodes in the content graph.
2 . The method of claim 1 , further comprising querying the content graph to retrieve modality data for the unlabeled node and different modality data for the adjacent node, and, responsive to the set of inputs including the modality data for the unlabeled node and the different modality data for the adjacent node, by the machine learning model, determining the label for the unlabeled node.
3 . The method of claim 1 , further comprising querying the content graph to retrieve content type data for the unlabeled node and different content type data for the adjacent node, and, responsive to the set of inputs including the content type data and the different content type data, by the machine learning model, determining the label for the unlabeled node.
4 . The method of claim 1 , further comprising, by the cross-modality embedding model trained on digital content items of different modalities, generating a cross-modality embedding for the digital content item, and, responsive to the set of inputs including the cross-modality embedding, by the machine learning model, determining the label for the unlabeled node.
5 . The method of claim 1 , further comprising responsive to activity data of an entity that is associated with the digital content item and represented in a social graph, by an entity activity model trained on entity activity data extracted from the application system, generating an entity activity-content item embedding, and, responsive to the set of inputs including the entity activity-content item embedding, by the machine learning model, determining the label for the unlabeled node.
6 . The method of claim 1 , further comprising machine learning the embedding data associated with the unlabeled node by an embedding layer of the machine learning model, wherein the embedding layer of the machine learning model comprises the cross-modality embedding model, and responsive to the set of inputs including the machine learned embedding data, by the machine learning model, determining the label for the unlabeled node.
7 . A system comprising:
a processor; and
memory coupled to the processor, wherein the memory comprises instructions that when executed by the processor cause the processor to:
for a set of digital content items received from user systems by an application system, generate and store a content graph, wherein:
the application system is capable of distributing digital content to the user systems over a network based on a social graph and a set of spam labels; and
the content graph comprises labeled nodes that correspond to a subset of the set of digital content items that have spam labels, unlabeled nodes that correspond to a subset of the set of digital content items that do not have spam labels, and edges between nodes of the content graph;
query the content graph to retrieve edge data for an edge between an unlabeled node and an adjacent node, wherein:
the unlabeled node corresponds to a digital content item that does not have a spam label;
the adjacent node corresponds to a different digital content item; and
the edge data indicates a relationship between the unlabeled node and the adjacent node; and
responsive to a set of inputs comprising the retrieved edge data and embedding data associated with the unlabeled node, by a machine learning, assign a label of the set of spam labels to the unlabeled node, wherein:
the embedding data comprises output of a cross-modality embedding model;
the output of the cross-modality embedding model is connected to input of the machine learning model;
the machine learning model comprises a graph transformer-based neural network having connected modality-specific layers;
the graph transformer-based neural network is trained to determine inter-content item relationships based on the labeled nodes and the edges of the content graph; and
the label is assigned to the unlabeled node by the connected modality-specific layers of the trained graph transformer-based neural network performing graph-based propagation of label predictions from labeled nodes to unlabeled nodes of different modalities based on edge relationships between the nodes in the content graph.
8 . The system of claim 7 , wherein the instructions, when executed by the processor further cause the processor to query the content graph to retrieve modality data for the unlabeled node and different modality data for the adjacent node, and, responsive to the set of inputs including the modality data for the unlabeled node and the different modality data for the adjacent node, by the machine learning model, determine the label for the unlabeled node.
9 . The system of claim 7 , wherein the instructions, when executed by the processor further cause the processor to query the content graph to retrieve content type data for the unlabeled node and different content type data for the adjacent node, and, responsive to the set of inputs including the content type data and the different content type data, by the machine learning model, determine the label for the unlabeled node.
10 . The system of claim 7 , wherein the instructions, when executed by the processor further cause the processor to, by the cross-modality embedding model trained on digital content items of different modalities, generate a cross-modality embedding for the digital content item, and, responsive to the set of inputs including the cross-modality embedding, by the machine learning model, determine the label for the unlabeled node.
11 . The system of claim 7 , wherein the instructions, when executed by the processor further cause the processor to, responsive to activity data of an entity that is associated with the digital content item and represented in a social graph, by an entity activity model trained on entity activity data extracted from the application system, generate an entity activity-content item embedding, and, responsive to the set of inputs including the entity activity-content item embedding, by the machine learning model, determine the label for the unlabeled node.
12 . The system of claim 7 , wherein the instructions, when executed by the processor further cause the processor to machine learn the embedding data associated with the unlabeled node by an embedding layer of the machine learning model, wherein the embedding layer of the machine learning model comprises the cross-modality embedding model, and responsive to the set of inputs including the machine learned embedding data, by the machine learning model, determine the label for the unlabeled node.
13 . A non-transitory machine-readable medium comprising instructions that when executed by a processor cause the processor to:
for a set of digital content items received from user systems by an application system, generate and store a content graph, wherein:
the application system is capable of distributing digital content to the user systems over a network based on a social graph and a set of spam labels; and
the content graph comprises labeled nodes that correspond to a subset of the set of digital content items that have spam labels, unlabeled nodes that correspond to a subset of the set of digital content items that do not have spam labels, and edges between nodes of the content graph;
query the content graph to retrieve edge data for an edge between an unlabeled node and an adjacent node, wherein:
the unlabeled node corresponds to a digital content item that does not have a spam label;
the adjacent node corresponds to a different digital content item; and
the edge data indicates a relationship between the unlabeled node and the adjacent node; and
responsive to a set of inputs comprising the retrieved edge data and embedding data associated with the unlabeled node, by a machine learning, assign a label of the set of spam labels to the unlabeled node, wherein:
the embedding data comprises output of a cross-modality embedding model;
the output of the cross-modality embedding model is connected to input of the machine learning model;
the machine learning model comprises a graph transformer-based neural network having connected modality-specific layers;
the graph transformer-based neural network is trained to determine inter-content item relationships based on the labeled nodes and the edges of the content graph; and
the label is assigned to the unlabeled node by the connected modality-specific layers of the trained graph transformer-based neural network performing graph-based propagation of label predictions from labeled nodes to unlabeled nodes of different modalities based on edge relationships between the nodes in the content graph.
14 . The non-transitory machine-readable medium of claim 13 , wherein the instructions, when executed by the processor further cause the processor to query the content graph to retrieve modality data for the unlabeled node and different modality data for the adjacent node, and, responsive to the set of inputs including the modality data for the unlabeled node and the different modality data for the adjacent node, by the machine learning model, determine the label for the unlabeled node.
15 . The non-transitory machine-readable medium of claim 13 , wherein the instructions, when executed by the processor further cause the processor to query the content graph to retrieve content type data for the unlabeled node and different content type data for the adjacent node, and, responsive to the set of inputs including the content type data and the different content type data, by the machine learning model, determine the label for the unlabeled node.
16 . The non-transitory machine-readable medium of claim 13 , wherein the instructions, when executed by the processor further cause the processor to, by the cross-modality embedding model trained on digital content items of different modalities, generate a cross-modality embedding for the digital content item, and, responsive to the set of inputs including the cross-modality embedding, by the machine learning model, determine the label for the unlabeled node.
17 . The non-transitory machine-readable medium of claim 13 , wherein the instructions, when executed by the processor further cause the processor to, responsive to activity data of an entity that is associated with the digital content item and represented in a social graph, by an entity activity model trained on entity activity data extracted from the application system, generate an entity activity-content item embedding, and, responsive to the set of inputs including the entity activity-content item embedding, by the machine learning model, determine the label for the unlabeled node.
18 . The non-transitory machine-readable medium of claim 13 , wherein the instructions, when executed by the processor further cause the processor to machine learn the embedding data associated with the unlabeled node by an embedding layer of the machine learning model, wherein the embedding layer of the machine learning model comprises the cross-modality embedding model, and responsive to the set of inputs including the machine learned embedding data, by the machine learning model, determine the label for the unlabeled node.
19 . The non-transitory machine-readable medium of claim 13 , wherein the labeled node corresponds to content that is a first type of a post, an article, a comment, or a reply, and the unlabeled node corresponds to a second type that is different from the first type.
20 . The non-transitory machine-readable medium of claim 13 , wherein the labeled node corresponds to content having a first modality of a text, image, video, or audio; and the unlabeled node corresponds to a second modality that is different from the first modality.