Determining item quantity at checkout
A depth image of items on a scale are captured. Depth information for the depth image is processed to produce a three-dimensional (3D) map of the depth information. The items are identified as blobs within the 3D map. A plane is generated and projected into the 3D map and each separate blob that intersects the plane is counted to determine an item quantity for the items depicted in the original depth image. The item quantity is provided back to a terminal for item quantity verification of the items during a checkout transaction at the terminal. In an embodiment, the plane is projected onto the 3D map and animated to move from a top of the 3D map to a bottom of the 3D map while the separate blobs are counted for the item quantity determination as the plane moves.
1 . A method, comprising:
obtaining depth information for a three-dimensional (3D) image depicting items on a scale at a terminal during a transaction;
generating a 3D map from the depth information representing each item as a blob within the 3D map;
generating a plane projected onto the 3D map;
counting each intersection of the plane with each blob to obtain a total item count for the items; and
providing the total item count as an item quantity for the items back to the terminal to continue processing of the transaction.
2 . The method of claim 1 , wherein obtaining further includes identifying the depth information as infrared (IR) data captured in the 3D image by an IR sensor of the terminal.
3 . The method of claim 2 , wherein identifying further includes identifying from the IR data that the items are enclosed within a bag on the scale.
4 . The method of claim 1 , wherein obtaining further includes receiving the 3D image from the terminal when an operator of the terminal selects an item search from a transaction user interface.
5 . The method of claim 1 , wherein generating the 3D map further includes excluding certain blobs from the 3D map based on threshold sizes associated with noise present in the depth information.
6 . The method of claim 1 , wherein generating the plane further includes generating the plane as a horizontal line projected into the 3D map to intersect each blob.
7 . The method of claim 6 , wherein generating the plane further includes animating and dynamically moving the horizontal line from a top of the 3D map to a bottom of the 3D map.
8 . The method of claim 7 , wherein counting further includes counting each separate blob that intersects the horizontal line from left to right as the horizontal line moves downward from the top to the bottom of the 3D map.
9 . The method of claim 1 , wherein providing further includes instructing the terminal to verify that the item quantity matches an operator-entered item quantity before continuing with processing of the transaction.
10 . The method of claim 1 further comprising, processing the method as an item quantity determination service for the items when the items are non-barcoded items, and an operator of the terminal initiates an item search through a transaction interface of the terminal.
11 . The method of claim 10 , wherein processing further includes interacting with the terminal when the terminal is a self-service terminal, and the operator is a customer performing a self-checkout and when the terminal is a point-of-sale terminal, and the operator is a cashier performing a cashier-assisted checkout.
12 . A method, comprising:
detecting an item search initiated at a terminal through a transaction interface during a transaction;
activating an infrared (IR) camera to capture an IR image of items placed on a scale of the terminal;
creating a three-dimensional (3D) map from IR information associated with the IR image;
projecting a line into the 3D map;
counting each intersection of blobs, representing the items within the 3D map, with the line to obtain a total item count for the items on the scale; and
providing the total item count as an item quantity to the terminal for verification by the terminal during the transaction.
13 . The method of claim 12 , wherein creating further includes determining sizes for continuous sets of data represented in the IR information.
14 . The method of claim 13 , wherein determining further includes comparing the sizes against threshold sizes and retaining certain sets of data represented in the IR information as the blobs within the 3D map, each blob represents a unique one of the items.
15 . The method of claim 12 , wherein projecting further includes generating the line as a best-fitted line to a horizontal plane represented in the 3D map.
16 . The method of claim 12 , wherein counting further includes counting each intersection of a given blob with the line from left to right within the 3D map.
17 . The method of claim 16 , wherein counting further includes dynamically moving the line from a top of the 3D map to a bottom of the 3D map while performing the counting to identify certain stacked items represented within the 3D map.
18 . The method of claim 12 , further comprising, instructing the terminal to suspend the transaction for an audit when an operator of the terminal attempts to override the item quantity within an operator-provided item quantity for the items.
19 . A system, comprising:
at least one server comprising a processor and a non-transitory computer-readable storage medium;
the non-transitory computer-readable storage medium comprises executable instructions; and
the executable instructions when executed on the processor, cause the processor to perform operations comprising:
generating a three-dimensional (3D) map from depth information associated with a 3D image that depicts items on a scale of a terminal during a checkout at the terminal, wherein each separate item is identified as a blob within the 3D map;
projecting a horizontal plane onto the 3D map to intersect each of the blobs;
determining an item quantity for the items by counting the intersections of the horizontal plane with the blobs within the 3D map; and
providing the item quantity to the terminal during the checkout and instructing the terminal to suspend a transaction for an audit when an operator of the terminal attempts to override the item quantity with an operator-provided item quantity.
20 . The system of claim 19 , wherein the transaction is a self-service transaction, and the terminal is a self-service terminal or wherein the transaction is a cashier-assisted transaction, and the terminal is a point-of-sale terminal.