Object recognition for bottom of basket detection using neural network

1. A method of object recognition for a cart in a checkout lane at a retail store location, the method comprising:

collecting a stream of images from a checkout lane at a sampling rate, where each image includes color data and coordinate data;

applying each image from the stream of images to a convolutional neural network to determine a set of class scores for each image;

selecting a class score from the set of class scores for an image where the class score is the highest value in the set of class scores, meets a threshold, or is the highest value in the set of class scores and meets the threshold;

cross-referencing the selected class score to a BOB classification value from a table of BOB classification labels;

setting a BOB status for the checkout lane using the BOB classification value;

translating the BOB status to a BOB indicator; and

communicating the BOB indicator to a checkout lane device, wherein

communicating the BOB indicator to a checkout lane device comprises at least one of displaying the BOB indicator on a light device of the checkout lane device and sounding the BOB indicator on a speaker device of the checkout lane device.

2. The method of claim 1 , wherein the convolutional neural network comprises a convolutional layer, a pooling layer, and a fully-connected layer.

3. The method of claim 1 , wherein setting the BOB status using the BOB classification value comprises setting the BOB status to active, indicating a BOB item is located on a lower tray of the cart, when the BOB classification value represents a likelihood the cart is present in the checkout lane with at least one item on the lower tray of the cart.

4. The method of claim 1 , wherein setting the BOB status using the BOB classification value comprises setting the BOB status to inactive, indicating a BOB item is not located on a lower tray of the cart, when the BOB classification value represents a likelihood the cart is not present in the checkout lane.

5. The method of claim 1 , wherein setting the BOB status for the checkout lane using the BOB classification value further comprises:

setting the BOB status to inactive, indicating a BOB item is not located on a lower tray of the cart, when the BOB classification value for the image represents a likelihood the cart is not present in the checkout lane; and

a plurality of consecutive images collected subsequent to the image result in class scores that cross-reference to the same BOB classification value that represents a likelihood the cart is not present in the checkout lane.

6. The method of claim 1 , wherein setting the BOB status for the checkout lane using the BOB classification value further comprises:

setting the BOB status to active, indicating a BOB item is located on a lower tray of the cart, when the BOB classification value for the image represents a likelihood the cart is present in the checkout lane with at least one item on the lower tray of the cart; and

a plurality of consecutive images collected subsequent to the image result in class scores that cross-reference to the same BOB classification value that represents a likelihood the cart is present in the checkout lane with at least one item on the lower tray of the cart.

7. An object recognition system for a cart in a checkout lane at a retail store location, the system comprising:

at least one camera that collects a stream of images from a checkout lane, where each image includes color data and coordinate data;

a computing device that receives the stream of images from the camera according to a sampling rate, the computing device including

an application component that applies each image from the stream of images to a convolutional neural network to determine a set of class scores for each image,

an analyzing component that selects a class score from the set of class scores for an image where the class score is the highest value in the set of class scores, meets a threshold, or is the highest value in the set of class scores and meets the threshold; cross-references the selected class score to a BOB classification value from a table of BOB classification labels; and sets a BOB status for the checkout lane using the BOB classification value,

a translation component that translates the BOB status to a BOB indicator, and

a communication component that communicates the BOB indicator to a checkout lane device, wherein communication of the BOB indicator to a checkout lane device comprises production of the BOB indicator as at least one of light from a light device of the checkout lane device and sound from a speaker device of the checkout lane device.

8. The system of claim 7 , wherein the convolutional neural network comprises a convolutional layer, a pooling layer, and a fully-connected layer.

9. The system of claim 7 , wherein the analyzing component uses the BOB classification value to set the BOB status to active, indicating a BOB item is located on a lower tray of the cart, when the BOB classification value represents a likelihood the cart is present in the checkout lane with at least one item on the lower tray of the cart.

10. The system of claim 7 , wherein the analyzing component uses the BOB classification value to set the BOB status to inactive, indicating a BOB item is not located on a lower tray of the cart, when the BOB classification value represents a likelihood the cart is not present in the checkout lane.

11. The system of claim 7 , wherein the analyzing component sets the BOB status for the checkout lane using the BOB classification value to inactive, indicating a BOB item is not located on a lower tray of the cart, when:

the BOB classification value for the image represents a likelihood the cart is no present in the checkout lane; and

a plurality of consecutive images collected subsequent to the image result in class scores that cross-reference to the same BOB classification value that represents a likelihood the cart is not present in the checkout lane.

12. The system of claim 7 , wherein the analyzing component sets the BOB status for the checkout lane using the BOB classification value to active, indicating a BOB item is located on a lower tray of the cart, when:

the BOB classification value for the image represents a likelihood the cart is present in the checkout lane with at least one item on the lower tray of the cart; and

a plurality of consecutive images collected subsequent to the image result in class scores that cross-reference to the same BOB classification value that represents a likelihood the cart is present in the checkout lane with at least one item on the lower tray of the cart.

13. A computer program product for object recognition for a cart in a checkout lane at a retail store location, the computer program product stored on a non-transitory computer-readable medium and including instructions adapted to cause a computer to execute steps comprising:

collecting a stream of images from a checkout lane at a sampling rate, where each image includes color data and coordinate data;

applying each image from the stream of images to a convolutional neural network to determine a set of class scores for each image;

selecting a class score from the set of class scores for an image where the class score is the highest value in the set of class scores, meets a threshold, or is the highest value in the set of class scores and meets the threshold;

cross-referencing the selected class score to a BOB classification value from a table of BOB classification labels;

setting a BOB status for the checkout lane using the BOB classification value;

translating the BOB status to a BOB indicator; and

communicating the BOB indicator to a checkout lane device, wherein communicating the BOB indicator to a checkout lane device comprises at least one of displaying the BOB indicator on a light device of the checkout lane device and sounding the BOB indicator on a speaker device of the checkout lane device.

14. The computer program product of claim 13 , wherein the convolutional neural network comprises a convolutional layer, a pooling layer, and a fully-connected layer.

15. The computer program product of claim 13 , wherein setting the BOB status using the BOB classification value comprises setting the BOB status to active, indicating a BOB item is located on a lower tray of the cart, when the BOB classification value represents a likelihood the cart is present in the checkout lane with at least one item on the lower tray of the cart.

16. The computer program product of claim 13 , wherein setting the BOB status using the BOB classification value comprises setting the BOB status to inactive, indicating a BOB item is not located on a lower tray of the cart, when the BOB classification value represents a likelihood the cart is not present in the checkout lane.

17. The computer program product of claim 13 , wherein setting the BOB status for the checkout lane using the BOB classification value further comprises:

setting the BOB status to inactive, indicating a BOB item is not located on a lower tray of the cart, when the BOB classification value for the image represents a likelihood the cart is not present in the checkout lane; and

a plurality of consecutive images collected subsequent to the image result in class scores that cross-reference to the same BOB classification value that represents a likelihood the cart is not present in the checkout lane.

18. The computer program product of claim 13 , wherein setting the BOB status for the checkout lane using the BOB classification value further comprises

setting the BOB status to active, indicating a BOB item is located on a lower tray of the cart, when the BOB classification value for the image represents a likelihood the cart is present in the checkout lane with at least one item on the lower tray of the cart; and

a plurality of consecutive images collected subsequent to the image result in class scores that cross-reference to the same BOB classification value that represents a likelihood the cart is present in the checkout lane with at least one item on the lower tray of the cart.