Automated detection of carton damage
Methods and systems for automated detection of carton defects are disclosed. One method includes capturing one or more images of a carton via a camera system at a routing location within a warehouse of a retail supply chain, and applying a machine learning model to determine a likelihood of damage of the carton. The method can include, based on the likelihood of damage being above a particular threshold, identifying the carton as damaged. A carton assessment record can be stored in a carton damage tracking database, including the one or more images of the carton alongside the likelihood of damage and the routing location.
1 . A method comprising:
capturing an image of the conveyor system via a camera system at a routing location of a warehouse;
applying a machine learning model to the image of the conveyor system;
identifying a carton boundary of a carton in the image of the conveyor system;
identifying a conveyor system segment boundary in the image of the conveyor system;
determining, using the carton boundary and the conveyor system segment boundary, that the conveyor system segment includes the carton;
identifying a second carton boundary of a second carton in the image of the conveyor system; and
determining, using the second carton boundary and the conveyor system segment boundary, that the conveyor system segment includes the second carton.
2 . The method of claim 1 , wherein the machine learning model is trained to recognize cartons and conveyor system segments using images of cartons and of conveyor system segments.
3 . The method of claim 1 , wherein the machine learning model is configured to receive a plurality of images captured within a common time period from a plurality of different angles, and wherein the machine learning model is applied to the plurality of images.
4 . The method of claim 1 , wherein identifying a carton boundary of a carton further comprises:
determining possible boundaries of objects in the image; and
determining a likelihood that the possible boundaries represent boundaries of the objects.
5 . The method of claim 1 , wherein identifying a conveyor system segment boundary further comprises:
determining possible boundaries of objects in the image; and
determining a likelihood that the possible boundaries represent boundaries of the objects.
6 . The method of claim 1 , further comprising, in response to determining that the conveyor system segment includes the first carton and the second carton, identifying a defect in handling the first carton and the second carton at the routing location.
7 . The method of claim 6 , further comprising, in response to identifying the defect, automatically triggering rerouting of at least one of the first carton or the second carton along the conveyor system.
8 . A method comprising:
capturing an image of a conveyor system via a camera system at a routing location of a warehouse;
applying a machine learning model to the image of the conveyor system to identify a defect, wherein applying the machine learning model to the image of the conveyor system to identify the defect comprises determining that a conveyor system segment of the conveyor system includes a plurality of cartons;
determining a likelihood that the conveyor system segment includes the plurality of cartons; and
determining that the likelihood that the conveyor system segment includes the plurality of cartons is greater than a threshold.
9 . The method of claim 8 , wherein the machine learning model is configured to receive a plurality of images.
10 . The method of claim 9 , wherein the machine learning model is applied to the plurality of images.
11 . The method of claim 8 , further comprising the machine learning model identifying a defect when the likelihood that the conveyor system segment includes the plurality of cartons is greater than the threshold.
12 . The method of claim 8 , further comprising identifying an empty conveyor system segment if the likelihood that the conveyor system segment includes zero cartons is greater than the threshold.
13 . The method of claim 8 , wherein the routing location is a location within the warehouse selected from among a plurality of routing locations at which the image of the conveyor system is captured.
14 . A carton defect detection system comprising:
an image capture system located at a routing location within a warehouse;
an image analysis server local to the image capture system, the image analysis server communicatively connected to the image capture system and configured to host a database, the image analysis server configured to:
receive an image of a conveyor system from the image capture system;
apply a machine learning model to the image of the conveyor system, wherein applying the machine learning model includes:
identifying a carton boundary of a carton in the image of the conveyor system;
identifying the conveyor system segment boundary in the image of the conveyor system;
determining, using the carton boundary and the conveyor system segment boundary that the conveyor system segment includes the carton;
identifying a second carton boundary of a second carton in the image of the conveyor system; and
determining, using the second carton boundary and the conveyor system segment boundary, that the conveyor system segment includes the second carton; and
in response to determining that the conveyor system segment includes the first carton and the second carton, identifying a defect in handling the first carton and the second carton at the routing location.
15 . The carton defect detection system of claim 14 , wherein the image analysis server is configured to store a carton assessment record in the database, the carton assessment record including the image of the conveyor system.
16 . The carton defect detection system of claim 14 , wherein the image analysis server is configured to receive user feedback regarding the carton defect, the user feedback being used to train the machine learning model.
17 . The carton defect detection system of claim 14 , further comprising a dashboard accessible via the image analysis server, the dashboard configured to present at least some contents of the carton assessment record.
18 . The carton defect detection system of claim 14 , wherein the conveyor system segment is a tray.
19 . The carton defect detection system of claim 14 , wherein the image analysis server is located at the warehouse.
20 . The carton defect detection system of claim 14 , wherein the image analysis server is further configured to:
capture images of the conveyor system segment via a plurality of camera systems positioned at different routing locations; and
based on outputs of the machine learning model in response to each of the images of the conveyor system, identify a routing location proximate to which the defect occurred.