Image-based barcode decoding
A barcode decoding system decodes item identifiers from images of barcodes. The barcode decoding system receives an image of a barcode and rotates the image to a pre-determined orientation. The barcode decoding system also may segment the barcode image to emphasize the portions of the image that correspond to the barcode. The barcode decoding system generates a binary sequence representation of the item identifier encoded in the barcode by applying a barcode classifier model to the barcode image, and decodes the item identifier from the barcode based on the binary sequence representation.
1. A cart system comprising:
a shopping cart;
a camera coupled to the shopping cart;
a processor; and
a non-transitory computer-readable medium storing instruction that, when executed by the processor, cause the processor to perform operations comprising:
receiving, from the camera coupled to the shopping cart, an image depicting a machine-readable label, wherein the machine-readable label represents an item identifier for an item comprising a sequence of values encoded in portions of the machine-readable label, wherein each portion corresponds to a encoding of a value of the sequence of values of the item identifier;
generating a rotated image of the machine-readable label by applying a rotation model to the image, wherein the rotation model comprises a machine-learning model stored on the computer-readable medium and trained to rotate images depicting machine-readable labels to a pre-determined orientation;
generating a segmented image of the machine-readable label by applying a segmentation model to the rotated image, wherein the segmentation model comprises a machine-learning model stored on the computer-readable medium and trained to remove backgrounds from rotated images of machine-readable labels;
generating a sequence representation of the item identifier for the item by applying a machine-readable label prediction model to the segmented image of the machine-readable label, wherein the sequence representation represents a prediction of the sequence of values, wherein the sequence representation represents one or more probabilistic predictions of the sequence of values encoded in the portions of the machine-readable label, and wherein the machine-readable label prediction model is a machine-learning model stored on the computer-readable medium and trained to generate sequence representations of item identifiers for machine-readable labels in segmented images; and
decoding the item identifier represented in the machine-readable label from the sequence representation of the item identifier generated by the machine-readable label prediction model.
2. The cart system of claim 1 , wherein the segmentation model comprises an attention mechanism.
3. The cart system of claim 1 , wherein the machine-readable label is a barcode.
4. The cart system of claim 1 , wherein the machine-readable label prediction model generates the prediction of the sequence of values by applying a connectionist temporal classification inference algorithm to the segmented image.
5. The cart system of claim 4 , wherein the machine-readable label prediction model is trained using a loss function comprising a connectionist temporal classification loss function.
6. The cart system of claim 1 , wherein the pre-determined orientation is horizontal.
7. The cart system of claim 1 , wherein the predicted sequence of values is an alphanumeric representation of the predicted item identifier.
8. The cart system of claim 1 , wherein the predicted sequence of values comprises a set of binary predictions that indicate probabilistic predictions of corresponding binary values of a binary encoding of the item identifier.
9. The cart system of claim 1 , further comprising:
adding the item represented by the item identifier to a shopping list associated with a user.
10. The cart system of claim 1 , further comprising:
causing information associated with the item to be displayed on a display device coupled to the shopping cart.
11. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving, from a camera coupled to a shopping cart, an image depicting a machine-readable label, wherein the machine-readable label represents an item identifier for an item comprising a sequence of values encoded in portions of the machine-readable label, wherein each portion corresponds to a encoding of a value of the sequence of values of the item identifier;
generating a rotated image of the machine-readable label by applying a rotation model to the image, wherein the rotation model comprises a machine-learning model stored on the computer-readable medium and trained to rotate images depicting machine-readable labels to a pre-determined orientation;
generating a segmented image of the machine-readable label by applying a segmentation model to the rotated image, wherein the segmentation model comprises a machine-learning model stored on the computer-readable medium and trained to remove backgrounds from rotated images of machine-readable labels;
generating a sequence representation of the item identifier for the item by applying a machine-readable label prediction model to the segmented image of the machine-readable label, wherein the sequence representation represents a prediction of the sequence of values, wherein the sequence representation represents one or more probabilistic predictions of the sequence of values encoded in the portions of the machine-readable label, and wherein the machine-readable label prediction model is a machine-learning model stored on the computer-readable medium and trained to generate sequence representations of item identifiers for machine-readable labels in segmented images; and
decoding the item identifier represented in the machine-readable label from the sequence representation of the item identifier generated by the machine-readable label prediction model.
12. The non-transitory computer-readable storage medium of claim 11 , wherein the segmentation model comprises an attention mechanism.
13. The non-transitory computer-readable storage medium of claim 11 , wherein the machine-readable label is a barcode.
14. The non-transitory computer-readable storage medium of claim 11 , wherein the machine-readable label prediction model generates the prediction of the sequence of values by applying a connectionist temporal classification inference algorithm to the segmented image.
15. The non-transitory computer-readable storage medium of claim 14 , wherein the machine-readable label prediction model is trained using a loss function comprising a connectionist temporal classification loss function.
16. The non-transitory computer-readable storage medium of claim 11 , wherein the pre-determined orientation is horizontal.
17. The non-transitory computer-readable storage medium of claim 11 , wherein the predicted sequence of values is an alphanumeric representation of the predicted item identifier.
18. The non-transitory computer-readable storage medium of claim 11 , wherein the predicted sequence of values comprises a set of binary predictions that indicate probabilistic predictions of corresponding binary values of a binary encoding of the item identifier.
19. The non-transitory computer-readable storage medium of claim 11 , further comprising:
adding the item represented by the item identifier to a shopping list associated with a user.
20. A method comprising:
receiving, from a camera coupled to a shopping cart, an image depicting a machine-readable label, wherein the machine-readable label represents an item identifier for an item comprising a sequence of values encoded in portions of the machine-readable label, wherein each portion corresponds to a encoding of a value of the sequence of values of the item identifier;
generating a rotated image of the machine-readable label by applying a rotation model to the image, wherein the rotation model comprises a machine-learning model stored on a computer-readable medium coupled to the shopping cart and trained to rotate images depicting machine-readable labels to a pre-determined orientation;
generating a segmented image of the machine-readable label by applying a segmentation model to the rotated image, wherein the segmentation model comprises a machine-learning model stored on the computer-readable medium and trained to remove backgrounds from rotated images of machine-readable labels;
generating a sequence representation of the item identifier for the item by applying a machine-readable label prediction model to the segmented image of the machine-readable label, wherein the sequence representation represents a prediction of the sequence of values, wherein the sequence representation represents one or more probabilistic predictions of the sequence of values encoded in the portions of the machine-readable label, and wherein the machine-readable label prediction model is a machine-learning model stored on the computer-readable medium and trained to generate sequence representations of item identifiers for machine-readable labels in segmented images; and
decoding the item identifier represented in the machine-readable label from the sequence representation of the item identifier generated by the machine-readable label prediction model.