System and method for identifying object information in image or video data
Systems and methods are provided for identifying a product in an image and outputting stock keeping units of the product. The system comprises three main components: a database server, a data analytics system and a standard dashboard. The database server contains real-time inventory images as well as historical images of each product type. The data analytics system is executed by a computer processor configured to apply object detection and classification and deep learning algorithms to detect product information captured by the image. The data analytics system is also configured to determine hierarchical classification categories for the product. The standard dashboard is configured to output a report regarding the product information.
1. A system for identifying a product in an image and outputting stock keeping units of the product, comprising:
a database server executed by one or more storage devices for storing image data comprising real-time inventory images and historical images of each product type;
a data analytics system executed by one or more computers; and
a standard dashboard;
wherein the data analytics system comprises instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
receiving an input image, via application programming interface from a recording device or an external database;
generating an output characterizing an object presented within the image, via an object detection component of a first neural network in the data analytics system;
receiving the output generated by the first neural network, via a second neural network;
processing the output, via an object classification component of the second neural network in the data analytics system, to determine hierarchical classification categories comprising a first level classification category of the object, a second level classification category of the object and a third level classification category of the object; and
generating a descriptor of the object based on at least one of the first level classification category, the second level classification category and the third level classification category of the object, via the data analytics system;
and
wherein the standard dashboard is configured to output the descriptor of the object.
2. The system of claim 1 , wherein the application programming interface is microservice based.
3. The system of claim 1 , wherein the object detection component applies instance segmentation configured to extract a product at individual level and masks the product out with any other information.
4. The system of claim 3 , wherein the instance segmentation is achieved based on convolutional neural network.
5. The system of claim 1 , wherein the first neural network comprises a hybrid neural network model consisting of convolutional neural network and multilayer perceptron architecture.
6. The system of claim 5 , wherein the first neural network comprising a convolutional layer, a pooled layer and an upsample layer.
7. The system of claim 1 , wherein the second neural network comprising a convolutional layer, and a perceptron layer.
8. The system of claim 1 , wherein the first and second neural networks are trained in an end-to-end automated learning process.
9. A method for identifying a product in an image and outputting stock keeping units of the product, comprising
receiving an input image, via application programming interface from a recording device or an external database;
generating an output characterizing an object presented within the image, via an object detection component of a first neural network in a data analytics system;
receiving the output generated by the first neural network, via a second neural network;
processing the output, via an object classification component of the second neural network in the data analytics system, to determine hierarchical classification categories comprising a first level classification category of the object, a second level classification category of the object and a third level classification category of the object;
generating a descriptor of the object based on at least one of the first level classification category, the second level classification category and the third level classification category of the object, via the data analytics system; and
outputting the descriptor of the object, via a standard dashboard.
10. The method of claim 9 , wherein the application programming interface is microservice based.
11. The method of claim 9 , wherein the object detection component applies instance segmentation configured to extract a product at individual level and masks the product out with any other information.
12. The method of claim 11 , wherein the instance segmentation is achieved based on convolutional neural network.
13. The method of claim 9 , wherein the first neural network comprises a hybrid neural network model consisting of convolutional neural network and multilayer perceptron architecture.
14. The method of claim 13 , wherein the first neural network comprising a convolutional layer, a pooled layer and an upsample layer.
15. The method of claim 9 , wherein the second neural network comprising a convolutional layer, and a perceptron layer.
16. The method of claim 9 , wherein the first and second neural networks are trained in an end-to-end automated learning process.
17. A system for identifying a product in an image and outputting stock keeping units of the product, comprising:
a database server executed by one or more storage devices for storing image data comprising real-time inventory images and historical images of each product type;
a data analytics system executed by one or more computers; and
a standard dashboard;
wherein the data analytics system comprises instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
receiving an input image, via application programming interface from a recording device or an external database;
generating an output characterizing an object presented within the image, via an object detection component of a first neural network in the data analytics system;
generating a first descriptor for the object based on the output, via the first neural network in the data analytics system;
receiving the output and the first descriptor generated by the first neural network, via a second neural network;
processing the output, via an object classification component of the second neural network in the data analytics system, to determine hierarchical classification categories comprising a first level classification category of the object, a second level classification category of the object and a third level classification category of the object; and
generating a second descriptor of the object based on at least one of the first level classification category, the second level classification category and the third level classification category of the object, via the data analytics system;
and
wherein the standard dashboard is configured to output the first and second descriptor of the object.
18. The system of claim 17 , wherein the application programming interface is microservice based.
19. The system of claim 17 , wherein the object detection component applies instance segmentation configured to extract a product at individual level and masks the product out with any other information.
20. The system of claim 19 , wherein the instance segmentation is achieved based on convolutional neural network.
21. The system of claim 17 , wherein the first neural network comprises a hybrid neural network model consisting of convolutional neural network and multilayer perceptron architecture.
22. The system of claim 21 , wherein the first neural network comprising a convolutional layer, a pooled layer and an upsample layer.
23. The system of claim 17 , wherein the second neural network comprising a convolutional layer, and a perceptron layer.
24. The system of claim 17 , wherein the first and second neural networks are trained in an end-to-end automated learning process.