Training data quality for spam classification
In one embodiment, a method includes accessing posts in a social-networking system. Each of the posts is unlabeled with respect to whether the post is known to be spam. The method also includes determining a posting user who submitted the post to the social-networking system and a recipient user to whom the post is addressed. The method further includes determining a first vector representation of the posting user and a second vector representation of the recipient user based on one or more features associated with the post, the posting user, and the recipient user. The method still further includes comparing the vector representations and building a machine learning model for automatically detecting spam posts in the social-networking system using a subset of the plurality of posts as non-spam training data.
1. A method comprising:
by a computing device, accessing a training data comprising a plurality of posts in a social-networking system that have been flagged by users of the social-networking system as being spam;
by the computing device, determining, for each of the plurality of posts, a posting user who submitted the post to the social-networking system and a recipient user to whom the post is addressed;
by the computing device, determining, for each of the plurality of posts, a first feature vector representation of the posting user and a second feature vector representation of the recipient user based on one or more features associated with the post, the posting user, and the recipient user;
by the computing device, comparing, for each of the plurality of posts, the first feature vector representation of the posting user to the second feature vector representation of the recipient user;
by the computing device, determining first feature vector representations of posting users associated with a subset of posts as sufficiently similar to second feature vector representations of recipient users associated with the subset of posts according to a predefined metric;
by the computing device, in response to determining the first feature vector representations of the posting users associated with the subset of posts as sufficiently similar to the second feature vector representations of the recipient users associated with the subset of posts, identifying the subset of posts as non-spam posts, without explicit user input, from the plurality of posts that have been flagged by the users of the social-networking system as being spam;
by the computing device, filtering the training data for training a machine learning model by removing the subset of posts that are identified as non-spam posts, without explicit user input, from the plurality of posts that have been flagged by the users of the social-networking system as being spam; and
by the computing device, using the filtered training data for training the machine learning model for automatically detecting a particular post in the social-networking system as spam or not spam.
2. The method of claim 1 , wherein the recipient user is associated with a concept node in a social graph within the social-networking system.
3. The method of claim 1 , wherein the predefined metric comprises a distance threshold between vectors in a multi-dimensional mapping space.
4. The method of claim 1 , wherein the predefined metric comprises a particular pattern of interactions defined by each first and second feature vector representation.
5. The method of claim 1 , wherein training the machine learning model further comprises using a second subset of the plurality of posts as spam training data, wherein each post in the second subset are determined to be sufficiently different using a second predefined metric.
6. The method of claim 1 , wherein the first feature vector representation and the second feature vector representation of each post in the subset are determined to be sufficiently similar by mapping each feature vector representation in a d-dimensional embedding space.
7. The method of claim 1 , wherein the plurality of posts that have been flagged by the users of the social-networking system as being spam is an indication by the users that the plurality of posts are unsolicited posts.
8. The method of claim 1 , wherein generating the first and second feature vector representations for each of the plurality of posts comprises aggregating term embeddings associated with the post.
9. The method of claim 1 , wherein the plurality of posts are NFX posts.
10. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
access a training data comprising a plurality of posts in a social-networking system that have been flagged by users of the social-networking system as being spam;
determine, for each of the plurality of posts, a posting user who submitted the post to the social-networking system and a recipient user to whom the post is addressed;
determine, for each of the plurality of posts, a first feature vector representation of the posting user and a second feature vector representation of the recipient user based on one or more features associated with the post, the posting user, and the recipient user;
compare, for each of the plurality of posts, the first feature vector representation of the posting user to the second feature vector representation of the recipient user;
determine first feature vector representations of posting users associated with a subset of posts as sufficiently similar to second feature vector representations of recipient users associated with the subset of posts according to a predefined metric;
in response to determining the first feature vector representations of the posting users associated with the subset of posts as sufficiently similar to the second feature vector representations of the recipient users associated with the subset of posts, identify the subset of posts as non-spam posts, without explicit user input, from the plurality of posts that have been flagged by the users of the social-networking system as being spam;
filter the training data for training a machine learning model by removing the subset of posts that are identified as non-spam posts, without explicit user input, from the plurality of posts that have been flagged by the users of the social-networking system as being spam; and
use the filtered training data to train the machine learning model for automatically detecting spam posts a particular post in the social-networking system as spam or not spam.
11. The media of claim 10 , wherein the recipient user is associated with a concept node in a social graph within the social-networking system.
12. The media of claim 10 , wherein the predefined metric comprises a distance threshold between vectors in a multi-dimensional mapping space.
13. The media of claim 10 , wherein the predefined metric comprises a particular pattern of interactions defined by each first and second feature vector representation.
14. The media of claim 10 , wherein training the machine learning model further comprises using a second subset of the plurality of posts as spam training data, wherein each post in the second subset are determined to be sufficiently different using a second predefined metric.
15. The media of claim 10 , wherein the first feature vector representation and the second feature vector representation of each post in the subset are determined to be sufficiently similar by mapping each feature vector representation in a d-dimensional embedding space.
16. The media of claim 10 , wherein the plurality of posts that have been flagged by the users of the social-networking system as being spam is an indication by the users that the plurality of posts are unsolicited posts.
17. The media of claim 10 , wherein generating the first and second feature vector representations for each of the plurality of posts comprises aggregating term embeddings associated with the post.
18. A system comprising:
one or more processors; and
one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
access a training data comprising a plurality of posts in a social-networking system that have been flagged by users of the social-networking system as being spam;
determine, for each of the plurality of posts, a posting user who submitted the post to the social-networking system and a recipient user to whom the post is addressed;
determine, for each of the plurality of posts, a first feature vector representation of the posting user and a second feature vector representation of the recipient user based on one or more features associated with the post, the posting user, and the recipient user;
compare, for each of the plurality of posts, the first feature vector representation of the posting user to the second feature vector representation of the recipient user;
determine first feature vector representations of posting users associated with a subset of posts as sufficiently similar to second feature vector representations of recipient users associated with the subset of posts according to a predefined metric;
in response to determining the first feature vector representations of the posting users associated with the subset of posts as sufficiently similar to the second feature vector representations of the recipient users associated with the subset of posts, identify the subset of posts as non-spam posts, without explicit user input, from the plurality of posts that have been flagged by the users of the social-networking system as being spam;
filter the training data for training a machine learning model by removing the subset of posts that are identified as non-spam posts, without explicit user input, from the plurality of posts that have been flagged by the users of the social-networking system as being spam; and
use the filtered training data to train the machine learning model for automatically detecting a particular post in the social-networking system as spam or not spam.
19. The system of claim 18 , wherein training the machine learning model further comprises using a second subset of the plurality of posts as spam training data, wherein each post in the second subset are determined to be sufficiently different using a second predefined metric.
20. The system of claim 18 , wherein the first feature vector representation and the second feature vector representation of each post in the subset are determined to be sufficiently similar by mapping each feature vector representation in a d-dimensional embedding space.