Machine learning classification based on separate processing of multiple views
Methods, systems, and apparatuses, including computer programs encoded on a computer storage medium, for machine learning classification based on separate processing of multiple views. In some implementations, a system obtains image data for multiple images showing different views of an object. A machine learning model is used to generate a separate output based on each the multiple images individually. The outputs for the respective images are combined to generate a combined output. A predicted characteristic of the object is determined based on the combined output. An indication of the predicted characteristic of the object is provided.
1. A method performed by one or more computers, the method comprising:
obtaining, by the one or more computers, image data for multiple images of a three-dimensional object, wherein the multiple images respectively provide different views of the three-dimensional object and respectively show different portions of the three-dimensional object;
using, by the one or more computers, a machine learning model to generate a separate output for each of the multiple images, wherein each of the outputs is generated using a same set of machine learning model parameters and each of the outputs is produced by processing only a single, respective one of the multiple images using the machine learning model, wherein the machine learning model was trained using training data that included:
at least some training images, wherein each of the at least some training images includes a depiction of at least one side of a corresponding object; and
for each of the at least some training images, a corresponding label that represents a classification of the corresponding object based on one or more features of the corresponding object that are not visible in the training image but are visible from other another side of the corresponding object not shown in the training image;
combining, by the one or more computers, the outputs for the respective images to generate a combined output;
determining, by the one or more computers, a predicted characteristic of the three-dimensional object based on the combined output; and
providing, by the one or more computers, an indication of the predicted characteristic of the three-dimensional object.
2. The method of claim 1 , wherein determining the predicted characteristic comprises selecting a classification for the object from among a predetermined set of classifications based on the combined output.
3. The method of claim 1 , wherein the predicted characteristic is a value indicative of a property of the three-dimensional object.
4. The method of claim 1 , wherein the different views are views of different sides of the three-dimensional object.
5. The method of claim 1 , wherein the predicted characteristic comprises at least one of:
a state or condition of the three-dimensional object;
an object type for the three-dimensional object; or
a size of the three-dimensional object.
6. The method of claim 1 , wherein the machine learning model is a trained neural network that has been trained to classify an overall state of an object based on an image representing a single view of an object that shows only a portion of the object.
7. The method of claim 6 , wherein the set of machine learning model parameters are parameters of the trained neural network that were learned during training of the trained neural network.
8. The method of claim 6 , wherein the set of machine learning model parameters are values of weights of neurons of the trained neural network.
9. The method of claim 1 , wherein the output for each of the multiple images is an output vector, and wherein combining the outputs comprises summing corresponding values of the output vectors to generate a combined output vector.
10. The method of claim 6 , wherein the trained neural network comprises an output layer configured to provide values that respectively correspond to different classifications in a predetermined set of classifications, wherein the values represent a probability distribution over the predetermined set of classifications.
11. A system comprising:
one or more computers; and
one or more computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
obtaining, by the one or more computers, image data for multiple images of a three-dimensional object, wherein the multiple images respectively provide different views of the three-dimensional object and respectively show different portions of the three-dimensional object;
using, by the one or more computers, a machine learning model to generate a separate output for each of the multiple images, wherein each of the outputs is generated using a same set of machine learning model parameters and each of the outputs is produced by processing only a single, respective one of the multiple images using the machine learning model, wherein the machine learning model was trained using training data that included:
at least some training images, wherein each of the at least some training images includes a depiction of at least one side of a corresponding object; and
for each of the at least some training images, a corresponding label that represents a classification of the corresponding object based on one or more features of the corresponding object that are not visible in the training image but are visible from other another side of the corresponding object not shown in the training image;
combining, by the one or more computers, the outputs for the respective images to generate a combined output;
determining, by the one or more computers, a predicted characteristic of the three-dimensional object based on the combined output; and
providing, by the one or more computers, an indication of the predicted characteristic of the three-dimensional object.
12. The system of claim 11 , wherein determining the predicted characteristic comprises selecting a classification for the object from among a predetermined set of classifications based on the combined output.
13. The system of claim 11 , wherein the predicted characteristic is a value indicative of a property of the three-dimensional object.
14. The system of claim 11 , wherein the multiple views are views of different sides of the three-dimensional object.
15. The system of claim 11 , wherein the predicted characteristic comprises at least one of:
a state or condition of the three-dimensional object;
an object type for the three-dimensional object; or
a size of the three-dimensional object.
16. The system of claim 11 , wherein the machine learning model is a trained neural network that has been trained to classify an overall state of an object based on an image representing a single view of an object that shows only a portion of the object.
17. The system of claim 16 , wherein the output for each of the multiple images is an output vector, and wherein combining the outputs comprises summing corresponding values of the output vectors to generate a combined output vector.
18. One or more non-transitory computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:
obtaining, by the one or more computers, image data for multiple images of a three-dimensional object, wherein the multiple images respectively provide different views of the three-dimensional object and respectively show different portions of the three-dimensional object;
using, by the one or more computers, a machine learning model to generate a separate output for each of the multiple images, wherein each of the outputs is generated using a same set of machine learning model parameters and each of the outputs is produced by processing only a single, respective one of the multiple images using the machine learning model, wherein the machine learning model was trained using training data that included:
at least some training images, wherein each of the at least some training images includes a depiction of at least one side of a corresponding object; and
for each of the at least some training images, a corresponding label that represents a classification of the corresponding object based on one or more features of the corresponding object that are not visible in the training image but are visible from other another side of the corresponding object not shown in the training image;
combining, by the one or more computers, the outputs for the respective images to generate a combined output;
determining, by the one or more computers, a predicted characteristic of the three-dimensional object based on the combined output; and
providing, by the one or more computers, an indication of the predicted characteristic of the three-dimensional object.
19. The one or more non-transitory computer-readable media of claim 18 , wherein determining the predicted characteristic comprises selecting a classification for the object from among a predetermined set of classifications based on the combined output.
20. The one or more non-transitory computer-readable media of claim 18 wherein the predicted characteristic is a value indicative of a property of the three-dimensional object.