Image classification system
An image classification system is provided for determining a likely classification of an image using multiple machine learning models that share a base machine learning model. The image classification system may be a browser-based system on a user computing device that obtains multiple machine learning models over a network from a remote system once, stores the models locally in the image classification system, and uses the models multiple times without needing to subsequently request the machine learning models again from the remote system. The image classification system may therefore determine likely a classification associated with an image by running the machine learning models on a user computing device.
1. A system comprising computer-readable memory and one or more processors configured by computer-executable instructions to at least:
receive, from a user computing device, a request for a network resource configured to:
generate, using a base model, a base model output associated with an image;
select a subset of work models, from a hierarchy of work models, for generation of work model output, wherein the subset of work models comprises a first work model from a first level of the hierarchy and a second work model from a second level of the hierarchy;
subsequent to selection of the subset of work models, generate, using the subset of work models, one or more work model outputs based at least partly on the base model output, wherein a second subset of work models from the hierarchy of work models is not used to generate the one or more work model outputs; and
determine a label associated with the image, wherein the label is based at least partly on the one or more work model outputs;
transmit, to the user computing device, the network resource in response to the request; and
receive, from the user computing device, the image and image classification data representing the label associated with the image.
2. The system of claim 1 , wherein the one or more processors are further configured by the computer-executable instructions to transmit the base model and the subset of work models to the user computing device.
3. The system of claim 1 , wherein the label is associated with a confidence score, and wherein the label is displayed based at least partly on the confidence score satisfying a threshold.
4. The system of claim 1 , wherein the base model and the subset of work models comprise neural network based models executed on the user computing device.
5. The system of claim 1 , wherein the network resource is further configured to:
receive a second image selected by the user computing device for classification;
generate, using the base model and the second image, a second base model output associated with the second image;
generate, using the subset of work models and the base model output, a second plurality of work model outputs; and
determine a second label associated with the second image, wherein the second label is determined based at least partly on the second plurality of work model outputs, and wherein the second label is different than the label.
6. The system of claim 1 , wherein the network resource configured to generate the one or more work model outputs based at least partly on the base model output is configured to generate a plurality of work model outputs using the subset of work models and the base model output subsequent to selecting the subset of work models.
7. The system of claim 1 , wherein the network resource is further configured to select the first work model from a plurality of work models of the first level of the hierarchy based at least partly on at least one of: a prior request received prior to the request, or a prior work model output generated prior to the one or more work model outputs.
8. The system of claim 1 , wherein the first level of the hierarchy comprises a plurality of work model including the first work model, and wherein, only the first work model of the hierarchy of work models of the first level of the hierarchy is used to generate a work model output from the base model output.
9. The system of claim 1 , further comprising the user computing device, wherein the user computing device is configured to receive the network resource and the image and the image classification data representing the label associated with the image.
10. The system of claim 1 , further comprising a data store storing the network resource, the base model, and a plurality of work models.
11. A computer-implemented method comprising:
as implemented by a computing system comprising one or more computer processors configured to execute specific instructions:
receiving, from a user computing device, a request for a network resource configured to:
generate, using a base model, a base model output associated with an image;
select a subset of work models, from a hierarchy of work models, for generation of work model output, wherein the subset of work models comprises a first work model from a first level of the hierarchy and a second work model from a second level of the hierarchy;
subsequent to selection of the subset of work models, generate, using the subset of work models, one or more work model outputs based at least partly on the base model output, wherein a second subset of work models from the hierarchy of work models is not used to generate the one or more work model outputs; and
determine a label associated with the image, wherein the label is based at least partly on the one or more work model outputs;
transmitting, to the user computing device, the network resource in response to the request; and
receiving, from the user computing device, the image and image classification data representing the label associated with the image.
12. The computer-implemented method of claim 11 , further comprising transmitting the base model and the subset of work models to the user computing device.
13. The computer-implemented method of claim 11 , further comprising:
training the base model using a first set of training data; and
training a work model of the subset of work models using a second set of training data different from the first set of training data.
14. The computer-implemented method of claim 13 , wherein training the base model comprises training a first convolutional neural network using a first set of images.
15. The computer-implemented method of claim 14 , wherein training the work model comprises training a second convolutional neural network using a second set of images corresponding to a particular level of a multi-level hierarchy.