Reactive viral spam detection
Techniques herein balance the need for flexibility with the need accuracy, using a reactive approach to viral spam detection. After content (e.g., a social media platform news feed or timeline post) is created, interaction activity (e.g., content views) with the content is monitored. Based on the monitoring of the interactivity activity, it is determined whether a reactive viral spam analysis condition is satisfied for the content (e.g., because the number of content views exceeds a threshold). In response to determining that the reactive viral spam analysis condition is satisfied, a determination is made whether the content is or is not viral spam. If the content is determined to be viral spam, then it may be reported or flagged for further action (e.g., take down after manual confirmation).
1. A method for reactive viral spam detection, the method comprising:
monitoring interaction activity with a content item created and distributed by a multi-user application software system;
based on the monitoring of the interaction activity, determining that a reactive viral spam analysis condition is satisfied;
in response to determining that the reactive viral spam analysis condition is satisfied, determining whether the content item is or is not viral spam; and
wherein determining whether the content item is or is not viral spam is based on:
obtaining creator context information for a creator of the content item, the creator context information comprising comments left by the creator of the content item before the content item is created by the creator,
obtaining share cascade graph information for the content item,
obtaining post context information for the content item, the post context information representing interaction activity with the content item,
generating, by a first trained sequential context encoder, a first sequential context encoding of the creator context information,
generating, by a trained graph neural network, a graph neural network encoding of the share cascade graph information,
generating, by a second trained sequential context encoder, a second sequential context encoding of the post context information, and
classifying, by a trained machine learning classifier, the content item as viral spam or not viral spam based on at least the first sequential context encoding, the graph neural network encoding, and the second sequential context encoding.
2. The method of claim 1 , wherein the creator context information further comprises a respective set of one or more polarity scores for each comment of a sequence of comments by the creator on a set of content items.
3. The method of claim 1 , wherein the post context information comprises a respective set of one or more polarity scores for each comment of a sequence of comments on the content item.
4. The method of claim 1 , further comprising:
jointly training a graph neural network and a machine learning classifier based on a set of labeled training examples to yield the trained graph neural network and the trained machine learning classifier, wherein each labeled training example of the set of labeled training examples comprises respective share cascade graph information and a respective label.
5. The method of claim 4 , wherein the respective label of each labeled training example of the set of labeled training examples indicates whether the respective share cascade graph represents viral spam or represents viral non-spam.
6. The method of claim 1 , further comprising:
jointly training a first sequential context encoder, a second sequential context encoder, a graph neural network, and a machine learning classifier based on a set of labeled training examples to yield the first trained sequential context encoder, the second trained sequential context encoder, the trained graph neural network, and the trained machine learning classifier, wherein each labeled training example of the set of labeled training examples comprises respective creator context information, respective shared cascade graph information, respective post context information, and a respective label.
7. The method of claim 1 , wherein monitoring, by the multi-user application software system, the interaction activity with the content item comprises accumulating, by the multi-user application software system, a number of views of the content item; and
wherein determining, by the multi-user application software system, that the reactive viral spam analysis condition is satisfied based on the monitoring of the interaction activity comprises determining that the number of views of the content item exceeds a threshold number of views.
8. The method of claim 1 , wherein the share cascade graph information for the content item comprises directed acyclic graph information for the content item, the directed acyclic graph information indicating a root node and one or more child nodes of a graph, the directed acyclic graph information indicating that the root node represents the content item and each child node of the one or more child nodes represents respective shared content comprising the content item.
9. A non-transitory machine-readable storage medium storing instructions for reactive viral spam detection, the instructions, when executed by a set of one or more processors of a computer system, cause the computer system to perform operations comprising:
monitoring interaction activity with a content item created and distributed by a multi-user application software system;
based on the monitoring of the interaction activity, determining that a reactive viral spam analysis condition is satisfied;
in response to determining that the reactive viral spam analysis condition is satisfied,
determining whether the content item is or is not viral spam; and
wherein determining whether the content item is or is not viral spam is based on:
obtaining creator context information for a creator of the content item, the creator context information comprising comments left by the creator of the content item before the content item is created by the creator,
obtaining share cascade graph information for the content item,
obtaining post context information for the content item, the post context information representing interaction activity with the content item,
generating, by a first trained sequential context encoder, a first sequential context encoding of the creator context information,
generating, by a trained graph neural network model, a graph neural network encoding of the share cascade graph information,
generating, by a second trained sequential context encoder, a second sequential context encoding of the post context information, and
configuring a machine learning classifier to classify the content item as viral spam or not viral spam based on at least the first sequential context encoding, the graph neural network encoding, and the second sequential context encoding.
10. The non-transitory machine-readable storage medium of claim 9 , wherein the creator context information further comprises a respective set of one or more polarity scores for each comment of a sequence of comments by the creator on a set of content items.
11. The non-transitory machine-readable storage medium of claim 9 , wherein the post context information comprises a respective set of one or more polarity scores for each comment of a sequence of comments on the content item.
12. A system for reactive viral spam detection, the system comprising:
a set of one or more processors;
memory coupled to the set of one or more processors; and
wherein the memory comprises instructions which, when executed by the set of one or more processors, cause the system to perform operations comprising:
monitoring interaction activity with a content item created and distributed by a multi-user application software system;
based on the monitoring of the interaction activity, determining that a reactive viral spam analysis condition is satisfied;
in response to determining that the reactive viral spam analysis condition is satisfied, determining whether the content item is or is not viral spam; and
wherein determining whether the content item is or is not viral spam is based on:
obtaining creator context information for a creator of the content item, the creator context information comprising comments left by the creator of the content item before the content item is created by the creator,
obtaining share cascade graph information for the content item,
obtaining post context information for the content item, the post context information representing interaction activity with the content item,
generating, by a first trained sequential context encoder, a first sequential context encoding of the creator context information,
generating, by a trained graph neural network, a graph neural network encoding of the share cascade graph information,
generating, by a second trained sequential context encoder, a second sequential context encoding of the post context information, and
classifying, by a trained machine learning classifier, the content item as viral spam or not viral spam based on at least the first sequential context encoding, the graph neural network encoding, and the second sequential context encoding.
13. The system of claim 12 , wherein the creator context information further comprises a respective set of one or more polarity scores for each comment of a sequence of comments by the creator on a set of content items.
14. The system of claim 12 , wherein the post context information comprises a respective set of one or more polarity scores for each comment of a sequence of comments on the content item.
15. The system of claim 12 , the operations further comprising:
jointly training a graph neural network and a machine learning classifier based on a set of labeled training examples to yield the trained graph neural network and the trained machine learning classifier, wherein each labeled training example of the set of labeled training examples comprises respective share cascade graph information and a respective label.
16. The system of claim 15 , wherein the respective label of each labeled training example of the set of labeled training examples indicates whether the respective share cascade graph represents viral spam or represents viral non-spam.
17. The system of claim 12 , further comprising:
jointly training a first sequential context encoder, a second sequential context encoder, a graph neural network, and a machine learning classifier based on a set of labeled training examples to yield the first trained sequential context encoder, the second trained sequential context encoder, the trained graph neural network, and the trained machine learning classifier, wherein each labeled training example of the set of labeled training examples comprises respective creator context information, respective shared cascade graph information, respective post context information, and a respective label.
18. The system of claim 12 , wherein monitoring, by the multi-user application software system, the interaction activity with the content item comprises accumulating, by the multi-user application software system, a number of views of the content item; and
wherein determining, by the multi-user application software system, that the reactive viral spam analysis condition is satisfied based on the monitoring of the interaction activity comprises determining that the number of views of the content item exceeds a threshold number of views.
19. The system of claim 12 , wherein the share cascade graph information for the content item comprises directed acyclic graph information for the content item, the directed acyclic graph information indicating a root node and one or more child nodes of a graph, the directed acyclic graph information indicating that the root node represents the content item and each child node of the one or more child nodes represents respective shared content comprising the content item.