Multi-label document classification for documents from disjoint class sets

1 . A computer-implemented method comprising:

generating, by a machine learning service of a provider network, a plurality of label embeddings corresponding to a plurality of labels using a pre-trained model;

clustering the plurality of label embeddings into a plurality of clusters, wherein a number of clusters in the plurality of clusters is less than a number of labels in the plurality of labels, wherein a first cluster of the plurality of clusters corresponds to a first document class and a second cluster of the plurality of clusters corresponds to a second document class;

creating a multi-label classifier model that includes a neural network-based classifier for each cluster in the plurality of clusters, wherein the neural network-based classifier for each cluster includes an encoder to generate a representation of the unlabeled document and, for each label corresponding to a label embedding in the cluster, a decoder to classify the representation of the unlabeled document with the label;

training a set of parameters of each neural network-based classifier using a training document set, the training document set including a plurality of documents and, for each document, an associated one or more labels from the plurality of labels;

classifying, using the multi-label classifier model, an unlabeled document identified by a user with multiple labels of the plurality of labels; and

storing the one or more labels in a data store.

2 . The computer-implemented method of claim 1 , wherein the multi-label classifier model further includes a group classifier and further comprising:

providing, by the group classifier to the neural network-based classifier for a given cluster of the plurality of clusters, an indication of whether the unlabeled document belongs to the given cluster.

3 . The computer-implemented method of claim 1 , wherein the neural network-based classifier for each cluster includes an encoder to generate a fixed-length representation of the unlabeled document and, for each label corresponding to a label embedding in the cluster, a decoder to classify the fixed-length representation of the unlabeled document with the label.

4 . A computer-implemented method comprising:

generating a plurality of label embeddings corresponding to a plurality of labels;

clustering the plurality of label embeddings into a plurality of clusters, wherein a number of clusters in the plurality of clusters is less than a number of labels in the plurality of labels;

creating a machine learning model that includes a neural network-based classifier for each cluster in the plurality of clusters, wherein the neural network-based classifier for each cluster includes an encoder to generate a representation of the unlabeled document and, for each label corresponding to a label embedding in the cluster, a decoder to classify the representation of the unlabeled document with the label;

training a set of parameters of each neural network-based classifier in the machine learning model using a training document set, the training document set including a plurality of documents and, for each document, an associated one or more labels from the plurality of labels;

classifying, using the machine learning model, an unlabeled document identified by a user with one or more labels of the plurality of labels; and

storing the one or more labels in a data store.

5 . The computer-implemented method of claim 4 , wherein the machine learning model further includes a group classifier and further comprising:

providing, by the group classifier to the neural network-based classifier for a given cluster of the plurality of clusters, an indication of whether the unlabeled document belongs to the given cluster.

6 . The computer-implemented method of claim 4 , wherein the clustering is performed using k-means clustering.

7 . The computer-implemented method of claim 4 , further comprising selecting the plurality of clusters, wherein the selecting includes:

generating a first metric based at least in part on a distance between a first label embedding in a first cluster of the plurality of clusters to a cluster mean for the first cluster;

clustering the plurality of label embeddings into another plurality of clusters, wherein a number of clusters in the other plurality of clusters is different than the number of clusters in the plurality of clusters;

generating a second metric based at least in part on a distance between a second label embedding in a second cluster of the other plurality of clusters to a cluster mean for the second cluster; and

selecting the plurality of clusters based at least in part on a comparison of the first metric to the second metric.

8 . The computer-implemented method of claim 4 , further comprising:

clustering the plurality of label embeddings into another plurality of clusters, wherein a number of clusters in the other plurality of clusters is different than the number of clusters in the plurality of clusters;

creating another machine learning model that includes a neural network-based classifier for each cluster in the other plurality of clusters;

generating a first metric representing a classification performance of the machine learning model on a validation document set;

training a set of parameters of each neural network-based classifier in the other machine learning model using the training document set;

generating a second metric representing a classification performance of the other machine learning model on the validation document set; and

selecting the machine learning model to classify the unlabeled document based at least in part on a comparison of the first metric to the second metric.

9 . The computer-implemented method of claim 4 , wherein the plurality of label embeddings corresponding to the plurality of labels are generated using a pre-trained model.

10 . The computer-implemented method of claim 4 , further comprising:

training a set of parameters of each neural network-based classifier using a training document set, the training document set including a plurality of training documents and, for each document, an associated one or more labels from the plurality of labels.

11 . The computer-implemented method of claim 10 :

wherein a label embedding corresponding to a given label in the plurality of label embeddings includes a document embedding of at least one training document labeled with the given label, and

wherein the document embedding is generated using a pre-trained model.

12 . A system comprising:

a first one or more electronic devices of a provider network to implement a data store; and

a second one or more electronic devices of the provider network to implement a machine learning service, the machine learning service including instructions that upon execution cause the machine learning service to:

generate a plurality of label embeddings corresponding to a plurality of labels;

cluster the plurality of label embeddings into a plurality of clusters, wherein a number of clusters in the plurality of clusters is less than a number of labels in the plurality of labels;

create a machine learning model that includes a neural network-based classifier for each cluster in the plurality of clusters, wherein the neural network-based classifier for each cluster includes an encoder to generate a representation of the unlabeled document and, for each label corresponding to a label embedding in the cluster, a decoder to classify the representation of the unlabeled document with the label;

train a set of parameters of each neural network-based classifier in the machine learning model using a training document set, the training document set including a plurality of documents and, for each document, an associated one or more labels from the plurality of labels;

classify, using the machine learning model, an unlabeled document identified by a user with one or more labels of the plurality of labels; and

store the one or more labels in the data store.

13 . The system of claim 12 , wherein the machine learning model further includes a group classifier, and wherein the machine learning service includes further instructions that upon execution cause the machine learning service to:

provide, by the group classifier to the neural network-based classifier for a given cluster of the plurality of clusters, an indication of whether the unlabeled document belongs to the given cluster.

14 . The system of claim 12 , wherein the neural network-based classifier for each cluster includes an encoder to generate a representation of the unlabeled document and, for each label corresponding to a label embedding in the cluster, a decoder to classify the representation of the unlabeled document with the label.

15 . The system of claim 12 , wherein the machine learning service includes further instructions that upon execution cause the machine learning service to:

generate a first metric based at least in part on a distance between a first label embedding in a first cluster of the plurality of clusters to a cluster mean for the first cluster;

cluster the plurality of label embeddings into another plurality of clusters, wherein a number of clusters in the other plurality of clusters is different than the number of clusters in the plurality of clusters;

generate a second metric based at least in part on a distance between a second label embedding in a second cluster of the other plurality of clusters to a cluster mean for the second cluster; and

select the plurality of clusters based at least in part on a comparison of the first metric to the second metric.

16 . The system of claim 12 , wherein the machine learning service includes further instructions that upon execution cause the machine learning service to:

cluster the plurality of label embeddings into another plurality of clusters, wherein a number of clusters in the other plurality of clusters is different than the number of clusters in the plurality of clusters;

create another machine learning model that includes a neural network-based classifier for each cluster in the other plurality of clusters;

generate a first metric representing a classification performance of the machine learning model on a validation document set;

train a set of parameters of each neural network-based classifier in the other machine learning model using the training document set;

generate a second metric representing a classification performance of the other machine learning model on the validation document set; and

select the machine learning model to classify the unlabeled document based at least in part on a comparison of the first metric to the second metric.

17 . The system of claim 12 , wherein the plurality of label embeddings corresponding to the plurality of labels are generated using a pre-trained model.

18 . The system of claim 12 , wherein the machine learning service includes further instructions that upon execution cause the machine learning service to:

train a set of parameters of each neural network-based classifier using a training document set, the training document set including a plurality of training documents and, for each document, an associated one or more labels from the plurality of labels,

wherein a label embedding corresponding to a given label in the plurality of label embeddings includes a document embedding of at least one training document labeled with the given label, and

wherein the document embedding is generated using a pre-trained model.