Prototypical network algorithms for few-shot learning
Techniques for training an embedding using a limited training set are described. In some examples, the embedding is trained by generating a plurality of vectors from a random sample of the limited set of training data classes using a layer of the particular machine learning classification model, randomly selecting samples from the plurality of vectors into a set of samples, computing at least one distance for each sampled class from a center parameter for the class using the set of samples, generating a discrete probability distribution over the classes for a query point based on distances to a center parameter for each of the classes in the embedding space, calculating a loss value for the modified prototypical network, the calculation of the loss value being for a fixed geometry of the embedding space and including a measure of the difference between distributions, and back propagating.
1. A computer-implemented method comprising:
receiving and storing a set of labeled training images having a plurality of classes;
receiving a request to train a particular machine learning image classification model based upon the stored set of labeled training images, the machine learning image classification model having an input layer, one or more hidden layers, and an output layer;
training an embedding that is a representation of input data as a point in space of a given dimension to use with the particular machine learning image classification model using a network including the embedding by:
performing embeddings using a layer of the particular machine learning image classification model to generate a plurality of vectors from a random sample of the labeled training images classes,
randomly selecting samples from the plurality of vectors into a set of samples,
computing at least one distance for each sampled class from a center parameter for the class using the set of samples,
generating a discrete probability distribution over the classes for a query point based on distances to a center parameter for each of the classes in the embedding space,
calculating a loss value for the network, the calculation of the loss value being for a fixed geometry of the embedding space and including a measure of the difference between distributions, and
back propagating at least a derivative of the loss value to the embedding to adjust weights of the embedding;
after training the particular machine learning image classification model using a network, receiving a classification request to classify an image using the trained particular machine learning image classification model using the embedding, the request including an image to classify;
classifying the image using the trained particular machine learning image classification model using the embedding to create a result; and
returning the result according to the classification request.
2. The computer-implemented method of claim 1 , wherein the measure of the difference between distributions is an entropy function.
3. The computer-implemented method of claim 1 , wherein the distances are squared Euclidean distances and the fixed geometry is a sphere.
4. A computer-implemented method comprising:
receiving a request to train a machine learning classification model based upon a set of labeled training data, the set of labeled training data having a plurality of classes and the machine learning classification model having an input layer, one or more hidden layers, and an output layer;
training an embedding that is a representation of input data as a point in space of a given dimension to use by the machine learning classification model using a network including the embedding by:
generating a plurality of vectors from a random sample of the labeled training data classes using a layer of the particular machine learning classification model,
randomly selecting samples from the plurality of vectors into a set of samples,
computing at least one distance for each sampled class from a center parameter for the class using the set of samples,
generating a discrete probability distribution over the classes for a query point based on distances to a center parameter for each of the classes in the embedding space,
calculating a loss value for the network, the calculation of the loss value being for a fixed geometry of the embedding space and including a measure of the difference between distributions, and
back propagating at least a derivative of the loss value to the embedding to adjust weights of the embedding.
5. The computer-implemented method of claim 4 , further comprising:
after training the particular machine learning classification model using the network, receiving a classification request to classify data using the trained particular machine learning classification model, the request including data to classify;
classifying the data using the embedding of the trained particular machine learning classification model to produce a result; and
returning the result according to the classification request.
6. The computer-implemented method of claim 4 , further comprising:
after training the particular machine learning classification model using the network, receiving a classification request to classify data using the trained particular machine learning classification model, the request including an identification of location of data to classify;
retrieving the data at the identified location;
classifying the data using the embedding of the trained particular machine learning classification model to produce a result; and
returning the result according to the classification request.
7. The computer-implemented method of claim 4 , wherein the measure of the difference between distributions is an entropy function.
8. The computer-implemented method of claim 4 , wherein the distances are squared Euclidean distances and the fixed geometry is a sphere.
9. The computer-implemented method of claim 4 , wherein the particular machine learning classification model is a neural network.
10. The computer-implemented method of claim 9 , wherein the neural network is of a type selected at least from a convolutional neural networks (CNN), deep neural networks (DNN), recurrent neural networks (RNNs), fully connected network, or combination thereof.
11. The computer-implemented method of claim 4 , wherein the particular machine learning classification model is an image classification model.
12. The computer-implemented method of claim 4 , wherein a service of a provider network performs the training of the particular machine learning classification model.
13. The computer-implemented method of claim 12 , wherein the service of the provider network provides a graphical user interface to manage the set of labeled training data.
14. A provider network comprising:
a particular machine learning classification model implemented using a first one or more electronic devices; and
a classification service implemented by a second one or more electronic devices, the classification service including instructions that upon execution cause the classification service to:
train an embedding that is a representation of input data as a point in space of a given dimension to use by the particular machine learning classification model using a network including the embedding by:
generating a plurality of vectors from a random sample of labeled training data classes using a layer of the particular machine learning classification model,
randomly selecting samples from the plurality of vectors into a set of samples,
computing at least one distance for each sampled class from a center parameter for the class using the set of samples,
generating a discrete probability distribution over the classes for a query point based on distances to a center parameter for each of the classes in the embedding space,
calculating a loss value for the network, the calculation of the loss value being for a fixed geometry of the embedding space and including a measure of the difference between distributions, and
back propagating at least a derivative of the loss value to the embedding to adjust weights of the embedding.
15. The provider network of claim 14 , wherein after training the particular machine learning classification model using a network, the provider network is to:
receive a classification request to classify data using the trained particular machine learning classification model, the request including data to classify;
classify the data using the embedding of the trained particular machine learning classification model to produce a result; and
return the result according to the classification request.
16. The provider network of claim 15 , wherein after training the particular machine learning classification model using a network, the provider network is to:
receive a classification request to classify data using the trained particular machine learning classification model, the request including an identification of location of data to classify;
retrieve the data at the identified location;
classify the data using the embedding of the trained particular machine learning classification model to produce a result; and
return the result according to the classification request.
17. The provider network of claim 15 , wherein the distances are squared Euclidean distances and the fixed geometry is a sphere.
18. The provider network of claim 15 , wherein the particular machine learning classification model is an image classification model.
19. The provider network of claim 15 , wherein the measure of the difference between distributions is an entropy function.
20. The provider network of claim 15 , wherein the particular machine learning classification model is a neural network.