Image recognition using machine learning models with masked classification layers

1 . A system comprising:

a camera; and

a computer system in communication with the camera,

wherein the computer system is programmed with one or more sets of instructions that, when executed by the computer system, cause the computer system to execute a method comprising:

identifying a training data set, wherein the training data set comprises a first set of images and a first set of labels, wherein each of the first set of labels corresponds to a class of objects including at least one of the first set of images depicting one of the objects of the class;

training a machine learning model to recognize objects depicted within images, wherein the machine learning model comprises a masking component and a plurality of layers, and wherein training the machine learning model comprises:

providing at least some of the first set of images as training inputs to the machine learning model, wherein each of the training inputs comprises one of the first set of images;

receiving training outputs from the machine learning model in response to the training inputs, wherein each of the training outputs is received in response to one of the first set of images; and

adjusting at least one weight of at least one connection between a node of a first one of the plurality of layers and a node of a second one of the plurality of layers based at least in part on a difference between one of the training outputs received in response to one of the first set of images and one of the first set of labels corresponding to the class of objects;

programming the masking component of the machine learning model with identifiers of a plurality of items;

receiving at least a first image from the camera;

providing at least a portion of the first image to the machine learning model;

identifying a plurality of activations received from the machine learning model in response to the portion of the first image, wherein each of the plurality of activations corresponds to one of a plurality of classes of objects;

determining that each of a first subset of the plurality of activations corresponds to one of the plurality of items;

determining that each of a second subset of the plurality of activations does not correspond to one of the plurality of items;

adjusting each of the second subset of the plurality of activations to a value of negative infinity;

generating a probability distribution based at least in part on the first subset of the plurality of activations and the second subset of the plurality of activations; and

determining that the first image depicts an object in one of the classes of objects based at least in part on the probability distribution.

2 . The system of claim 1 , wherein the camera is provided within a materials handling facility, and

wherein each of the plurality of items is available at the materials handling facility.

3 . The system of claim 1 , wherein the machine learning model comprises a convolutional neural network.

4 . The system of claim 1 , wherein the method further comprises:

detecting a location of at least one hand depicted within the first image; and

cropping the first image about the location of the at least one hand, wherein the portion of the first image is the cropped first image.

5 . The system of claim 1 , wherein generating the probability distribution comprises:

providing each of the first subset of the plurality of activations and the second subset of the plurality of activations to a layer of the machine learning model configured to execute a SoftMax function.

6 . A method comprising:

providing at least a first image as a first input to a first component of a machine learning model, wherein the first component comprises an input layer, an output layer, and at least one middle layer between the input layer and the output layer;

receiving a first plurality of activations as first outputs in response to the first input, wherein each of the first plurality of activations is generated by the output layer of the first component;

determining, by a second component of the machine learning model, that each of a first subset of the first plurality of activations corresponds to one of a plurality of items in a set;

determining, by the second component, that each of a second subset of the first plurality of activations does not correspond to any of the plurality of items in the set;

setting, by the second component, each of the second subset of the first plurality of activations to a predetermined negative value;

generating, by a third component of the machine learning model, a probability distribution based at least in part on the first subset of the first plurality of activations and the second subset of the first plurality of activations; and

determining that the first image depicts one of the plurality of items in the set based at least in part on the probability distribution.

7 . The method of claim 6 , further comprising:

receiving, by a computer system, at least the first image from at least one camera over one or more networks, wherein the computer system is programmed with one or more instructions for executing the machine learning model,

wherein the at least one camera is provided within a materials handling facility.

8 . The method of claim 7 , wherein receiving at least the first image from the at least one camera over the one or more networks comprises:

receiving, by the computer system, a plurality of images from the at least one camera over the one or more networks;

detecting, by the computer system, locations of at least one hand depicted within each of the plurality of images; and

cropping, by the computer system, each of the plurality of images about the locations of the at least one hand,

wherein providing at least the first image as the first input to the first component of the machine learning model comprises:

providing each of the plurality of cropped images as the first input to the first component of the machine learning model,

wherein the first image is one of the plurality of cropped images.

9 . The method of claim 7 , wherein at least one storage unit having the plurality of items in the set is within a field of view of the at least one camera.

10 . The method of claim 6 , wherein the second component is a masking component, and

wherein the method further comprises:

programming the second component with identifiers of each of the plurality of items in the set.

11 . The method of claim 6 , wherein the first component is a convolutional neural network.

12 . The method of claim 11 , further comprising:

generating, by the first component, a first set of features, wherein each of the first set of features relates to at least one characteristic of at least the first image, and

wherein each of the first plurality of activations is generated based at least in part on the first set of features.

13 . The method of claim 6 , further comprising:

prior to providing at least the first image as the first input to the first component,

providing a plurality of sets of images as inputs to the first component, wherein each of the sets of images depicts an object within one of a plurality of classes;

receiving a plurality of outputs from the first component, wherein each of the plurality of outputs is received in response to one of the plurality of sets of images depicting an object within one of the plurality of classes as an input; and

training the first component based at least in part on a difference between one of the plurality of outputs received in response to one of the plurality of sets of images depicting an object in the one of the plurality of classes and a label of the one of the plurality of classes.

14 . The method of claim 6 , wherein the third component is a layer configured to execute a SoftMax function on the first plurality of activations.

15 . The method of claim 6 , wherein the predetermined negative value is negative infinity.

16 . A server comprising at least one processor and at least one data store,

wherein the server is in communication with a camera, and

wherein the server is programmed with one or more sets of instructions that, when executed by the at least one processor, cause the server to at least:

receive at least a first image from the camera;

provide at least the first image as a first input to at least one classifying layer of a machine learning model, wherein the at least one classifying layer comprises an input layer, an output layer, and at least one middle layer between the input layer and the output layer;

receive a first plurality of activations as first outputs in response to the first input, wherein each of the first plurality of activations is generated by the output layer of the at least one classifying layer;

determine, by a masking component of the machine learning model, that each of a first subset of the first plurality of activations corresponds to one of a plurality of items in a set;

determine, by the masking component, that each of a second subset of the first plurality of activations does not correspond to any of the plurality of items in the set;

set, by the masking component, each of the second subset of the first plurality of activations to a predetermined negative value;

generate, by a SoftMax layer of the machine learning model, a probability distribution based at least in part on the first subset of the first plurality of activations and the second subset of the first plurality of activations; and

identify an item depicted within at least the first image based at least in part on the probability distribution, wherein the item is one of the plurality of items in the set.

17 . The server of claim 16 , wherein the at least one classifying layer is at least one layer of a convolutional neural network.

18 . The server of claim 16 , wherein the one or more sets of instructions, when executed by the at least one processor, further cause the server to at least:

identify at least one item within a field of view of the camera, wherein the at least one item within the field of view of the camera is at least one of the plurality of items in the set.

19 . The server of claim 16 , wherein the predetermined negative value is negative infinity.

20 . The server of claim 16 , wherein the one or more sets of instructions, when executed by the at least one processor, further cause the server to at least:

provide a plurality of sets of images as inputs to the masking component, wherein each of the sets of images depicts an item within one of a plurality of classes, and wherein the plurality of sets of images are provided as inputs to the masking component prior to providing at least the first image as the first input to the at least one classifying layer;

receive a plurality of outputs from the masking component, wherein each of the plurality of outputs is received in response to one of the plurality of sets of images depicting an item within one of the plurality of classes as an input; and

train the masking component based at least in part on a difference between one of the plurality of outputs received in response to one of the plurality of sets of images depicting an item within the one of the plurality of classes and a label of the one of the plurality of classes.