Systems and methods for machine vision based object recognition
The present disclosure is related to object recognition and tracking using multi-camera driven machine vision. In one aspect, a method includes capturing, via a multi-camera system, a plurality of images of a user, each of the plurality of images representing the user from a unique angle; identifying, using the plurality of images, the user; detecting, throughout a facility, an item selected by the user; creating a visual model of the item to track movement of the item throughout the facility; determining, using the visual model, whether the item is selected for purchase; and detecting that the user is leaving the facility; and processing a transaction for the item when the item is selected for purchase and when the user has left the facility.
1. A computer-implemented method comprising:
receiving one or more images captured by cameras associated with a facility;
applying a machine-learning model to the one or more images to generate an output indicative of whether an item located within the facility has been selected;
detecting a selection of the item based on the generated output;
associating the selected item with a user;
generating a three-dimensional representation of the selected item, wherein the three-dimensional representation is identified from the one or more images;
tracking in real-time a movement of the selected item throughout the facility, wherein tracking includes determining a location of the three-dimensional representation relative to the facility;
determining that the user has left the facility based on the tracked movement of the selected item; and
processing a transaction for the selected item after determining that that the user has left the facility.
2. The computer-implemented method of claim 1 , wherein generating the three-dimensional representation includes modifying the three- dimensional representation into another representation associated with a different dimensional space.
3. The computer-implemented method of claim 1 , wherein the machine-learning model includes a convolutional neural network.
4. The computer-implemented method of claim 1 , wherein the transaction is processed when the tracked movement indicates that the selected item is exiting the facility.
5. The computer-implemented method of claim 1 , further comprising:
associating the selected item with a user;
determining that the user has left the facility, wherein the tracked movement indicates that the selected item continues to remain within the facility; and
determining a deselection of the selected item based on the indication.
6. The computer-implemented method of claim 1 , wherein the machine-learning model is trained using sample images that depict items associated with the facility.
7. The computer-implemented method of claim 1 , wherein generating the three-dimensional representation includes calibrating the one or more images to create a consensus between images that represent the selected item.
8. The computer-implemented method of claim 1 , wherein the machine-learning model is trained based on a loss determined between the generated output and a label provided for the one or more images.
9. A system, comprising:
one or more processors; and
memory storing thereon instructions that, as a result of being executed by the one or more processors, cause the system to perform operations comprising:
receiving one or more images captured by cameras associated with a facility;
applying a machine-learning model to the one or more images to generate an output indicative of whether an item located within the facility has been selected;
detecting a selection of the item based on the generated output;
associating the selected item with a user;
generating a three-dimensional representation of the selected item, wherein the three-dimensional representation is identified from the one or more images;
tracking in real-time a movement of the selected item throughout the facility, wherein tracking includes determining a location of the three-dimensional representation relative to the facility;
determining that the user has left the facility based on the tracked movement of the selected item; and
processing a transaction for the selected item after determining that that the user has left the facility.
10. The system of claim 9 , wherein generating the three-dimensional representation includes modifying the three-dimensional representation into another representation associated with a different dimensional space.
11. The system of claim 9 , wherein the machine-learning model includes a convolutional neural network.
12. The system of claim 9 , wherein the transaction is processed when the tracked movement indicates that the selected item is exiting the facility.
13. The system of claim 9 , wherein the instructions further cause the system to perform operations comprising:
associating the selected item with a user;
determining that the user has left the facility, wherein the tracked movement indicates that the selected item continues to remain within the facility; and
determining a deselection of the selected item based on the indication.
14. The system of claim 9 , wherein the machine-learning model is trained using sample images that depict items associated with the facility.
15. The system of claim 9 , wherein generating the three-dimensional representation includes calibrating the one or more images to create a consensus between images that represent the selected item.
16. The system of claim 9 , wherein the machine-learning model is trained based on a loss determined between the generated output and a label provided for the one or more images.
17. A non-transitory, computer-readable storage medium storing thereon executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to perform operations comprising:
receiving one or more images captured by cameras associated with a facility;
applying a machine-learning model to the one or more images to generate an output indicative of whether an item located within the facility has been selected;
detecting a selection of the item based on the generated output;
associating the selected item with a user;
generating a three-dimensional representation of the selected item, wherein the three-dimensional representation is identified from the one or more images;
tracking in real-time a movement of the selected item throughout the facility, wherein tracking includes determining a location of the three-dimensional representation relative to the facility;
determining that the user has left the facility based on the tracked movement of the selected item; and
processing a transaction for the selected item after determining that that the user has left the facility.
18. The non-transitory, computer-readable storage medium of claim 17 , wherein generating the three-dimensional representation includes modifying the three-dimensional representation into another representation associated with a different dimensional space.
19. The non-transitory, computer-readable storage medium of claim 17 , wherein the machine-learning model includes a convolutional neural network.
20. The non-transitory, computer-readable storage medium of claim 17 , wherein the transaction is processed when the tracked movement indicates that the selected item is exiting the facility.
21. The non-transitory, computer-readable storage medium of claim 17 , wherein the instructions further cause the computer system to perform operations comprising:
associating the selected item with a user;
determining that the user has left the facility, wherein the tracked movement indicates that the selected item continues to remain within the facility; and
determining a deselection of the selected item based on the indication.
22. The non-transitory, computer-readable storage medium of claim 17 , wherein the machine-learning model is trained using sample images that depict items associated with the facility.
23. The non-transitory, computer-readable storage medium of claim 17 , wherein generating the three-dimensional representation includes calibrating the one or more images to create a consensus between images that represent the selected item.
24. The non-transitory, computer-readable storage medium of claim 17 , wherein the machine-learning model is trained based on a loss determined between the generated output and a label provided for the one or more images.