Detecting abusive language using character N-gram features
View Patent ↗Methods and apparatus for detecting abusive language are disclosed. In one embodiment, a set of character N-grams is ascertained for a set of text. Feature values for a plurality of features of the set of text are determined, based, at least in part, on the set of character N-grams. A computer-generated model is applied to the feature values for the plurality of features to generate a score for the set of text, where the model includes a plurality of weights, each of the weights corresponding to one of the features. It may then be determined whether the set of text includes abusive language based, at least in part, on the score.
1. A method, comprising:
ascertaining, for a set of text posted via the Internet, an initial set of character N-grams within the set of text;
eliminating one or more duplicate character N-grams in the initial set of character N-grams to generate a set of character N-grams;
determining feature values for a plurality of features of the set of text based, at least in part, on the set of character N-grams, the plurality of features including one or more syntactic features indicative of a first identity of a first N-gram within the set of text, one or more linguistic features indicative of a second identity of a second N-gram within the set of text, and one or more semantic features indicative of a third identity of a third N-gram within the set of text;
applying a machine learning model to the feature values for the plurality of features to generate a score for the set of text, wherein the machine learning model includes a plurality of weights, wherein the plurality of weights comprises a first weight corresponding to at least one feature of the one or more syntactic features, a second weight corresponding to at least one feature of the one or more linguistic features and a third weight corresponding to at least one feature of the one or more semantic features, wherein the applying comprises (i) applying the first weight to the at least one feature of the one or more syntactic features indicative of the first identify of the first N-gram, (ii) applying the second weight to the at least one feature of the one or more linguistic features indicative of the second identify of the second N-gram, and (iii) applying the third weight to the at least one feature of the one or more semantic features indicative of the third identify of the third N-gram, wherein the applying the first weight, the applying the second weight and the applying the third weight is used to generate the score for the set of text comprising the first N-gram, the second N-gram and the third N-gram;
determining whether the set of text includes abusive language based, at least in part, on the score; and
responsive to determining that the set of text includes abusive language based, at least in part, on the score exceeding a threshold value associated with abusive language, deleting the set of text.
2. The method as recited in claim 1 , comprising:
generating the machine learning model based, at least in part, on a training set of comments, each comment in the training set of comments having a feature value for each feature of the plurality of features.
3. The method as recited in claim 2 , wherein each comment in the training set of comments includes at least some abusive language.
4. The method as recited in claim 1 , comprising:
generating at least one of a graph or a tree representing the set of text; and
determining the one or more syntactic features based, at least in part, on at least one of the graph or the tree.
5. The method as recited in claim 1 , wherein the plurality of features comprise a count indicating a number of times that one or more N-grams occur within the set of text.
6. The method as recited in claim 1 , wherein the one or more linguistic features are used to identify one or more elements of non-abusive language comprising one or more politeness words.
7. A computer program product comprising one or more non-transitory computer-readable media having computer program instructions stored therein, the computer program instructions being configured such that, when executed by one or more computing devices, the computer program instructions cause the one or more computing devices to:
ascertain, for a set of text posted via the Internet, an initial set of character N-grams within the set of text;
eliminate one or more duplicate character N-grams in the initial set of character N-grams to generate a set of character N-grams;
determine feature values for a plurality of features of the set of text based, at least in part, on the set of character N-grams, the plurality of features including at least two of: one or more syntactic features indicative of a first identity of a first N-gram within the set of text, one or more linguistic features indicative of a second identity of a second N-gram within the set of text, or one or more semantic features indicative of a third identity of a third N-gram within the set of text;
apply a machine learning model to the feature values for the plurality of features to generate a score for the set of text, wherein the machine learning model includes a plurality of weights, wherein the plurality of weights comprises at least two of a first weight corresponding to at least one feature of the one or more syntactic features, a second weight corresponding to at least one feature of the one or more linguistic features or a third weight corresponding to at least one feature of the one or more semantic features, wherein the applying comprises at least two of (i) applying the first weight to the at least one feature of the one or more syntactic features indicative of the first identify of the first N-gram, (ii) applying the second weight to the at least one feature of the one or more linguistic features indicative of the second identify of the second N-cram, or (iii) applying the third weight to the at least one feature of the one or more semantic features indicative of the third identify of the third N-cram, wherein at least two of the applying the first weight, the applying the second weight or the applying the third weight is used to generate the score for the set of text comprising the first N-gram, the second N-gram and the third N-gram;
determine whether the set of text includes abusive language based, at least in part, on the score; and
responsive to determining that the set of text includes abusive language based, at least in part, on the score exceeding a threshold value associated with abusive language, delete the set of text.
8. The computer program product as recited in claim 7 , the computer program instructions being configured to:
generate the machine learning model based, at least in part, on a training set of comments, each comment in the training set of comments having a feature value for each feature of the plurality of features.
9. The computer program product as recited in claim 8 , wherein each comment in the training set of comments includes at least some abusive language.
10. The computer program product as recited in claim 7 , the computer program instructions being configured to:
generate at least one of a graph or a tree representing the set of text; and
determine the one or more syntactic features based, at least in part, on at least one of the graph or the tree.
11. The computer program product as recited in 7 , wherein N is less than 10.
12. An apparatus, comprising:
at least one processor; and
at least one memory storing thereon computer-readable instructions configured to: ascertain, for a set of text posted via the Internet, an initial set of character N-grams within the set of text;
eliminate one or more duplicate character N-grams in the initial set of character N-grams to generate a set of character N-grams;
determine feature values for a plurality of features of the set of text based, at least in part, on the set of character N-grams, the plurality of features including at least two of: one or more syntactic features indicative of a first identity of a first N-gram within the set of text, one or more linguistic features indicative of a second identity of a second N-gram within the set of text, or one or more semantic features indicative of a third identity of a third N-gram within the set of text;
apply a machine learning model to the feature values for the plurality of features to generate a score for the set of text, wherein the machine learning model includes a plurality of weights, wherein the plurality of weights comprises at least two of a first weight corresponding to at least one feature of the one or more syntactic features, a second weight corresponding to at least one feature of the one or more linguistic features or a third weight corresponding to at least one feature of the one or more semantic features, wherein the applying comprises at least two of (i) applying the first weight to the at least one feature of the one or more syntactic features indicative of the first identify of the first N-gram, (ii) applying the second weight to the at least one feature of the one or more, linguistic features indicative of the second identify of the second N-cram, or (iii) applying the third weight to the at least one feature of the one or more semantic features indicative of the third identify of the third N-gram, wherein at least two of the applying the first weight, the applying the second weight or the applying the third weight is used to generate the score for the set of text comprising the first N-gram, the second N-gram and the third N-gram; and
determine whether the set of text includes abusive language based, at least in part, on the score.
13. The apparatus as recited in claim 12 , the computer-readable instructions being configured to:
generate the machine learning model based, at least in part, on a training set of comments, each comment in the training set of comments having a feature value for each feature of the plurality of features.
14. The apparatus as recited in claim 13 , wherein each comment in the training set of comments includes at least some abusive language.
15. The apparatus as recited in claim 12 , the computer-readable instructions being configured to:
generate at least one of a graph or a tree representing the set of text; and
determine the one or more syntactic features based, at least in part, on at least one of the graph or the tree.
16. The apparatus as recited in claim 12 , wherein the set of text is a comment posted on at least one of a blog, a message board forum, an article or a photograph.
17. The apparatus as recited in claim 12 , wherein N is less than 10.
18. The method as recited in claim 1 , the one or more syntactic features including one or more of: parts of speech tags indicating parts of speech of words within the set of text or dependency relations among words within the set of text.
19. The method as recited in claim 1 , the one or more linguistic features including one or more of: a length of the set of text in tokens, an average length of a word in the set of text, a number of punctuation marks in the set of text, a number of a particular type of punctuation mark in the set of text, a number of one letter tokens in the set of text, a number of capitalized letters in the set of text, a number of Uniform Resource Locators (URLs) in the set of text, a number of tokens in the set of text having non-alphabetical characters in a middle of the tokens, a number of discourse connectives in the set of text, a number of politeness words in the set of text, a number of modal verbs or phrases in the set of text, a number of unknown words in the set of text, or a number of blacklisted words within the set of text.
20. The apparatus as recited in claim 12 , the plurality of features including a count indicating a number of times that one or more N-grams occur within the set of text.