Object tracking and cluster voting for tag detections and exceptions

1 . A system for pallet tag tracking and cluster voting, the system comprising:

a processor; and

a computer-readable medium storing instructions that are operative upon execution by the processor to:

obtain pallet and pallet tag detection results associated with a plurality of pallets within a plurality of images generated by an image capture device within a retail environment;

track a selected pallet appearing within a set of images within the plurality of images, the set of images comprising a sequence of images including a portion of the selected pallet;

analyze the set of images to determine whether a pallet tag associated with the selected pallet is present within any image in the set of images;

assign a confidence score indicating a degree of confidence the pallet tag associated with the selected pallet is absent, wherein a pallet tag missing exception is generated in response to the determination the pallet tag is absent, wherein handling the pallet tag missing exception is prioritized based on the confidence score; and

calculate a cluster voting score for each pallet tag text recognition result in a set of text recognition results associated with a set of pallet tag detections for the selected pallet in response to the determination the pallet tag is present within the set of images, wherein a pallet tag text recognition result having a highest score is used to identify an accurate tag identification (ID) number on the pallet tag.

2 . The system of claim 1 , wherein the instructions are further operative to:

responsive to identifying the pallet tag in an image in the set of images, generate the pallet tag text recognition result based on optical character recognition (OCR) on the image, wherein the pallet tag text recognition result comprises at least a portion of a pallet ID number.

3 . The system of claim 1 , wherein the instructions are further operative to:

responsive to identifying the pallet tag in a set of two images from the set of images that includes a portion of the pallet tag, generate a first pallet tag text recognition result based on a first image from the set of two images that includes the pallet tag and a second pallet tag text recognition result based on a second image from the set of two images;

generate a first confidence score associated with the first pallet tag text recognition result and a second confidence score associated with the second pallet tag text recognition result; and

identify a pallet ID of the selected pallet based on the first pallet tag text recognition result associated with a highest confidence score, wherein the second pallet tag text recognition result associated with a lowest confidence score is disregarded.

4 . The system of claim 1 , wherein the instructions are further operative to:

responsive to identifying the pallet tag in a cluster of images from the set of images, generate a cluster of pallet tag text recognition results corresponding to the cluster of images;

calculate a combined confidence score for each repeated instance of a pallet ID identified in the cluster of pallet tag text recognition results, the combined confidence score comprising an identical pallet ID count times a confidence score; and

select a pallet ID associated with a highest combined confidence score.

5 . The system of claim 1 , wherein the instructions are further operative to:

obtain a plurality of pallet tag missing exceptions associated with the plurality of pallets;

generate a plurality of confidence scores associated with the plurality of pallet tag missing exceptions; and

rank each pallet tag missing exception based on the plurality of confidence scores, wherein the plurality of pallet tag missing exceptions is resolved in accordance with the rank for each pallet tag missing exception.

6 . The system of claim 1 , wherein the instructions are further operative to:

detect the plurality of pallets and pallet tags by a trained object detection model based on the plurality of images;

enclose the selected pallet within a large bounding box in each image in the set of images by the trained object detection model; and

enclose each pallet tag associated with the selected pallet within a small bounding box in image data associated with the set of images, wherein the image data is used for tracking the selected pallet through the sequence of images.

7 . The system of claim 1 , wherein the instructions are further operative to:

calculate a confidence score for each pallet tag text recognition result in a plurality of pallet tag text recognition results for the pallet tag;

apply a threshold minimum confidence score to the plurality of pallet tag text recognition results; and

filter any pallet tag text recognition results having a score that is less than the threshold minimum confidence score.

8 . A method for pallet tracking and cluster voting, the method comprising:

obtaining pallet and pallet tag detection results associated with a plurality of pallets within image data associated with a plurality of images generated by an image capture device within a retail facility;

tracking a selected pallet appearing within a set of images within the plurality of images, the set of images comprising images including a portion of the selected pallet;

determining whether a pallet tag associated with the selected pallet is present within any image in the set of images using a set of coordinates associated with the pallet;

responsive to determining the pallet tag is absent from the set of images, generating a tag missing confidence score indicating a degree of confidence the pallet tag associated with the selected pallet is absent based on quality of image data associated with the set of images; and

triggering a pallet tag missing exception associated with the selected pallet, the pallet tag missing exception including the tag missing confidence score, wherein the pallet tag missing exception is prioritized based on the tag missing confidence score.

9 . The method of claim 8 , further comprising:

calculating a cluster voting score for each pallet tag text recognition result in a set of text recognition results associated with a set of pallet tag detections for the selected pallet in response to presence of the pallet tag within the set of images, wherein a pallet tag text recognition result having a highest score is used to identify an accurate tag identification (ID) number on the pallet tag.

10 . The method of claim 8 , further comprising:

responsive to identifying the pallet tag in a set of two images from the set of images that includes a portion of the pallet tag, generating a first pallet tag text recognition result based on a first image from the set of two images that includes the pallet tag and a second pallet tag text recognition result based on a second image from the set of two images;

generating a first confidence score associated with the first pallet tag text recognition result and a second confidence score associated with the second pallet tag text recognition result; and

identifying a pallet ID of the selected pallet based on the first pallet tag text recognition result associated with a highest confidence score, wherein the second pallet tag text recognition result associated with a lowest confidence score is disregarded.

11 . The method of claim 8 , further comprising:

assigning a first rank to a first pallet tag missing exception based on a first tag missing confidence score;

assigning a second rank to a second pallet tag missing exception based on a second tag missing, wherein the first rank is a higher priority rank than the second rank; and

assigning a user to resolve the first pallet tag missing exception prior to resolving the second pallet tag missing exception.

12 . The method of claim 8 , further comprising:

selecting a pallet tag ID from a predicted pallet tag IDs associated with the selected pallet based on a confidence score associated with each predicted pallet tag ID, wherein the confidence score indicates a quality of optical character recognition results associated with each image containing a portion of the pallet tag ID.

13 . The method of claim 8 , further comprising:

determining whether the set of coordinates associated with the selected pallet overlaps with a pallet tag in any image in the set of images; and

triggering the pallet tag missing exception responsive to determining bounding box coordinates fail to overlap with any portion of the pallet tag in any image in the set of images.

14 . The method of claim 8 , further comprising:

generating a plurality of confidence scores associated with a plurality of pallet tag missing exceptions; and

prioritizing handling of the plurality of pallet tag missing exceptions based on the plurality of confidence scores.

15 . One or more computer storage devices having computer-executable instructions stored thereon, which, upon execution by a computer, cause the computer to perform operations comprising:

obtain pallet and pallet tag detection results associated with a plurality of pallets within a plurality of images generated by an image capture device within a retail facility;

track a selected pallet appearing within a set of images within the plurality of images, the set of images comprising a sequence of images including a portion of the selected pallet;

analyze the set of images to determine whether a pallet tag associated with the selected pallet is present within any image in the set of images;

responsive to a determination a portion of the pallet tag is present in a sub-set of the images in the set of images, generate a cluster of images comprising the sub-set of images;

generate a set of pallet tag text recognition results based on text recognized in the portion of the pallet tag present in the cluster of images; and

calculate a cluster voting score for each pallet tag text recognition result in a set of text recognition results associated with a set of pallet tag detections for the selected pallet in response to the determination the pallet tag is present within the set of images, wherein a pallet tag text recognition result having a highest score is used to identify an accurate tag identification (ID) number on the pallet tag.

16 . The one or more computer storage devices of claim 15 , wherein the operations further comprise:

responsive to determining the pallet tag is absent from all images in the set of images, calculate a tag missing confidence score indicating a degree of confidence the pallet tag is missing;

generate a pallet tag missing exception; and

assign the missing tag confidence score to the pallet tag missing exception.

17 . The one or more computer storage devices of claim 15 , wherein the operations further comprise:

determine whether bounding box coordinates associated with the selected pallet overlaps with pallet tag coordinates in the set of images; and

responsive to failure of the bounding box coordinates to overlap with the pallet tag coordinates, identify the pallet tag as missing from the set of images.

18 . The one or more computer storage devices of claim 15 , wherein the operations further comprise:

responsive to identifying the pallet tag in a set of two images from the set of images that includes a portion of the pallet tag, generating a first pallet tag text recognition result based on a first image from the set of two images that includes the pallet tag and a second pallet tag text recognition result based on a second image from the set of two images;

generating a first confidence score associated with the first pallet tag text recognition result and a second confidence score associated with the second pallet tag text recognition result; and

identifying a pallet ID of the selected pallet based on the first pallet tag text recognition result associated with a highest confidence score, wherein the second pallet tag text recognition result associated with a lowest confidence score is disregarded.

19 . The one or more computer storage devices of claim 15 , wherein the operations further comprise:

calculate a combined confidence score for each repeated instance of a pallet ID identified in the cluster of pallet tag text recognition results, the combined confidence score comprising an identical pallet ID count times a confidence score; and

selecting a pallet ID associated with a highest combined confidence score.

20 . The one or more computer storage devices of claim 15 , wherein the operations further comprise:

apply a threshold minimum confidence score to a plurality of pallet tag text recognition results; and

filter any pallet tag text recognition results having a score that is less than the threshold minimum confidence score.