Training a multi-label classifier
The technology disclosed includes a system to perform multi-label support vector machine (SVM) classification of a document. The system creates document features representing frequencies or semantics of words in the document. Trained SVM classification parameters for a plurality of labels are applied to the document features for the document. The system determines positive and negative distances between SVM hyperplanes for the labels and the feature vector. Labels with positive distance to the feature vector are harvested. When the distribution of negative distances is characterized by a mean and standard deviation, the system further harvests the labels with a negative distance such that the harvested labels include the labels with a negative distance between the mean negative distance and zero and separated from the mean negative distance by a predetermined first number of standard deviations.
1. A method, comprising:
training a multi-label classifier, the training comprising:
accessing training examples for documents belonging to a plurality of label classes, wherein a number of the plurality of label classes is more than fifty (50) label classes;
creating document features representing aspects of words in each of the documents;
training a support vector machine with the document features for one-vs-the-rest classification using the plurality of label classes, the training comprising:
training a one-vs-the-rest classifier the number of times such that the one-vs-the-rest classifier is trained one time for each of the plurality of label classes, each training including:
providing the training examples belonging to the respective label class to the support vector machine running the one-vs-the-rest classifier to obtain training output labels, and
comparing the training output labels using a linear support vector machine classifier to generate the number of hyperplane determinations that separate each label class of the plurality of label classes from the rest of the plurality of label classes; and
storing parameters of the trained support vector machine, the parameters comprising the hyperplane determinations.
2. The method of claim 1 , further comprising:
deploying the trained multi-label classifier and the stored parameters of the trained support vector machine to a production environment.
3. The method of claim 1 , further comprising:
classifying a document with multiple label classes of the plurality of label classes using the trained multi-label classifier.
4. The method of claim 1 , further comprising:
classifying a document with a label class of the plurality of label classes using the trained multi-label classifier.
5. The method of claim 1 , wherein the document features comprise frequency features based on term frequency-inverse document frequency.
6. The method of claim 1 , wherein the document features comprise semantic features based on embedding in a multi-dimensional vector space using Word2Vec.
7. The method of claim 1 , wherein the document features comprise semantic features based on embedding in a multi-dimensional vector space using global vectors for word representation.
8. The method of claim 1 , further comprising:
selecting support vector machine hyper parameters across regularization, class weight, and loss function in a predetermined search range to generate an at-least-one (ALO) score within ten percent of maximum attainable over the predetermined search range.
9. The method of claim 8 , wherein the ALO score is calculated based on a ratio of a count of the documents with at least one pairwise match between inferred labels and ground truth labels to a total number of documents with at least one ground truth label.
10. The method of claim 1 , wherein a first label class of the plurality of label classes is parked domain to classify documents posted on parked domains, the method further comprising:
identifying parked domains and collecting documents posted on the parked domains, the identifying comprising:
crawling uniform resource locators (URLs) that are within a predetermined edit distance of selected URL names;
determining for at least some of the crawled URLs that URL resolution is referred to an authoritative nameserver that appears in a list of parked domain nameservers identified as dedicated to parked domains; and
collecting the documents posted on the crawled URLs that are referred to the parked domain nameservers; and
labeling the collected documents as collected from the parked domains and storing the documents and parked domain labels for use in training.
11. A system, comprising:
one or more processors; and
one or more memories having stored thereon instructions that, upon execution by the one or more processors, cause the one or more processors to:
train a multi-label classifier, the training comprising:
accessing training examples for documents belonging to a plurality of label classes, wherein a number of the plurality of label classes is more than fifty (50) label classes;
creating document features representing aspects of words in each of the documents;
training a support vector machine with the document features for one-vs-the-rest classification using the plurality of label classes, the training comprising:
training a one-vs-the-rest classifier the number of times such that the one-vs-the-rest classifier is trained one time for each of the plurality of label classes, each training including:
providing the training examples belonging to the respective label class to the support vector machine running the one-vs-the-rest classifier to obtain training output labels, and
comparing the training output labels using a linear support vector machine classifier to generate the number of hyperplane determinations that separate each label class of the plurality of label classes from the rest of the plurality of label classes; and
storing parameters of the trained support vector machine, the parameters comprising the hyperplane determinations.
12. The system of claim 11 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
deploy the trained multi-label classifier and the stored parameters of the trained support vector machine to a production environment.
13. The system of claim 11 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
classify a document with multiple label classes of the plurality of label classes using the trained multi-label classifier.
14. The system of claim 11 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
classify a document with a label class of the plurality of label classes using the trained multi-label classifier.
15. The system of claim 11 , wherein the document features comprise frequency features based on term frequency-inverse document frequency.
16. The system of claim 11 , wherein the document features comprise semantic features based on embedding in a multi-dimensional vector space using Word2Vec.
17. The system of claim 11 , wherein the document features comprise semantic features based on embedding in a multi-dimensional vector space using global vectors for word representation.
18. The system of claim 11 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
select support vector machine hyper parameters across regularization, class weight, and loss function in a predetermined search range to generate an at-least-one (ALO) score within ten percent of maximum attainable over the predetermined search range.
19. The system of claim 18 , wherein the ALO score is calculated based on a ratio of a count of the documents with at least one pairwise match between inferred labels and ground truth labels to a total number of documents with at least one ground truth label.
20. The system of claim 11 , wherein a first label class of the plurality of label classes is parked domain to classify documents posted on parked domains, and wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
identify parked domains and collecting documents posted on the parked domains, the identifying comprising:
crawling uniform resource locators (URLs) that are within a predetermined edit distance of selected URL names;
determining for at least some of the crawled URLs that URL resolution is referred to an authoritative nameserver that appears in a list of parked domain nameservers identified as dedicated to parked domains; and
collecting the documents posted on the crawled URLs that are referred to the parked domain nameservers; and
label the collected documents as collected from the parked domains and storing the documents and parked domain labels for use in training.