Granted Patent B2

US 12,646,285 · App. 18/476,555 · Granted Jun 2, 2026

Inventors: Sumedh Vilas Datar (Grapevine, TX); Sailesh Bharathwaaj Krishnamurthy (Irving, TX); Shashipal Reddy Masini (Austin, TX)

Assignee: 7-Eleven, Inc.

G06V10/25G06T5/77G06T7/20G06V20/44G06V20/70G06T2207/10024G06T2207/20132G06V2201/07

Abstract

Claims (100)

1 . An item tracking system, comprising:

a plurality of cameras, wherein each camera is configured to capture images of at least a portion of a platform; and

one or more processors configured to:

detect a triggering event at the platform, wherein the triggering event corresponds to a placement of a first item on the platform;

in response to detecting the triggering event, capture a plurality of images of the first item on the platform using two or more cameras of the plurality of cameras;

for each image of the first item:

generate a cropped image for the image by editing the image to isolate at least a portion of the first item, wherein generating the cropped image comprises:

establishing a bounding box around the first item in the image, wherein the bounding box comprises an enclosed shape around the first item, wherein the first item occupies a portion of a total area contained within the bounding box;

calculate a ratio between the portion of the total area within the bounding box occupied by the first item and the total area within the bounding box;

determine whether the ratio equals or exceeds a minimum threshold area occupied by the first item in the bounding box;

if the ratio equals or exceeds the minimum threshold area, identify an item identifier for the first item based on the cropped image, wherein the ratio equaling or exceeding the minimum threshold area indicates that the cropped image comprises sufficient image information relating to the first item to identify the first item; and

if the ratio is below the minimum threshold area, discard the cropped image, wherein the ratio being below the minimum threshold area indicates that the cropped image comprises insufficient image information relating to the first item to identify the first item;

select a particular item identifier identified for a particular cropped image from a set of cropped images that were not discarded;

associate the particular item identifier with the first item; and

display an indicator of the particular item identifier on a user interface device.

2 . The item tracking system of claim 1 , wherein:

the total area contained within the bounding box associated with the cropped image comprises a plurality of pixels from the image captured by one of the two or more cameras;

the portion of the total area within the bounding box occupied by the first item comprises a first number of pixels from the plurality of pixels representing the first item;

the minimum threshold area comprises a minimum ratio between the first number of pixels and the plurality of pixels; and

the one or more processors are configured to calculate the ratio between the portion of the total area within the bounding box occupied by the first item and the total area within the bounding box by dividing the first number of pixels representing the item within the bounding box by the total number of pixels in the plurality of pixels contained in the bounding box.

3 . The item tracking system of claim 2 , wherein the one or more processors are further configured to:

fill a remaining portion of the bounding box not occupied by the first item with a second number of pixels of a particular pre-selected color that is different from one or more colors included in the image of the first item, wherein each pixel of the second number of pixels of the particular pre-selected color is associated with a unique combination of numerical values assigned to component colors red, green and blue (RGB) of the pixel; and

identify the first number of pixels representing the first item from the plurality of pixels in the bounding box by identifying each pixel within the bounding box that is not associated with the unique combination of numerical values assigned to component colors (RGB) of the pixel.

4 . The item tracking system of claim 3 , wherein the particular pre-selected color of each of the second number of pixels comprises a white color with the unique combination of numerical values including (255, 255, 255) assigned to the components (RGB) of the pixel.

5 . The item tracking system of claim 1 , further comprising a memory configured to store an encoded vector library, wherein the encoded vector library comprises a plurality of encoded vectors, wherein each encoded vector describes one or more attributes of a particular item and is associated with an item identifier for the particular item; and

wherein the one or more processors are further configured to identify an item identifier based on the cropped image by:

generating a first encoded vector for the cropped image, wherein the first encoded vector describes one or more attributes of the first item based on the cropped image;

comparing the first encoded vector to the encoded vectors in the encoded vector library;

selecting a second encoded vector from the encoded vector library that most closely matches with the first encoded vector, wherein a numerical similarity value indicates a degree of similarity between the first encoded vector and the selected second encoded vector; and

identifying the item identifier in the encoded vector library that is associated with the second encoded vector.

6 . The item tracking system of claim 5 , wherein the one or more processors are further configured to:

input the cropped image to a machine learning model, wherein the machine learning model is configured to output whether the cropped image is a back image of an item or a front image of an item;

obtain the output from the machine learning model indicating whether the cropped image is a back image of an item or a front image of an item; and

tag the cropped image as a back image or a front image based on the output from the machine learning model.

7 . The item tracking system of claim 6 , wherein the one or more processors are further configured to select the particular item identifier by:

determine whether any of the cropped images generated for the plurality of images are tagged as front images; and

in response to determining that one or more cropped images are tagged as front images, select the particular item identifier identified for the particular cropped image from the one or more cropped images tagged as front images.

8 . The item tracking system of claim 7 , wherein the one or more processors are further configured to select the particular item identifier by:

determining that only a single cropped image is tagged as a front image; and

selecting a first item identifier identified for the single cropped image as the particular item identifier to associate with the first item.

9 . The item tracking system of claim 6 , wherein the one or more processors are further configured to:

determine that none of the cropped images are tagged as front images; and

in response to determining that none of the cropped images are tagged as front images:

display, on a user interface device, an instruction to rotate or flip the first item on the platform.

10 . An item tracking system, comprising:

a camera, wherein the camera is configured to capture images of at least a portion of a platform; and

one or more processors configured to:

detect a triggering event at the platform, wherein the triggering event corresponds to a placement of a first item on the platform;

in response to detecting the triggering event, capture an image of the first item on the platform using the camera;

generate a cropped image for the image by editing the image to isolate at least a portion of the first item, wherein generating the cropped image comprises:

establishing a bounding box around the first item in the image, wherein the bounding box comprises an enclosed shape around the first item, wherein the first item occupies a portion of a total area contained within the bounding box;

calculate a ratio between the portion of the total area within the bounding box occupied by the first item and the total area within the bounding box;

determine whether the ratio equals or exceeds a minimum threshold area occupied by the first item in the bounding box;

if the ratio equals or exceeds the minimum threshold area, identify an item identifier for the first item based on the cropped image, wherein the ratio equaling or exceeding the minimum threshold area indicates that the cropped image comprises sufficient image information relating to the first item to identify the first item; and

if the ratio is below the minimum threshold area, discard the cropped image, wherein the ratio being below the minimum threshold area indicates that the cropped image comprises insufficient image information relating to the first item to identify the first item.

11 . The item tracking system of claim 10 , wherein:

the total area contained within the bounding box associated with the cropped image comprises a plurality of pixels from the image captured by the camera;

the portion of the total area within the bounding box occupied by the first item comprises a first number of pixels from the plurality of pixels representing the first item;

the minimum threshold area comprises a minimum ratio between the first number of pixels and the plurality of pixels; and

the one or more processors are further configured to calculate the ratio between the portion of the total area within the bounding box occupied by the first item and the total area within the bounding box by dividing the first number of pixels representing the item within the bounding box by the total number of pixels in the plurality of pixels contained in the bounding box.

12 . The item tracking system of claim 11 , wherein the one or more processors are further configured to:

fill a remaining portion of the bounding box not occupied by the first item with a second number of pixels of a particular pre-selected color that is different from one or more colors included in the image of the first item, wherein each pixel of the second number of pixels of the particular pre-selected color is associated with a unique combination of numerical values assigned to component colors red, green and blue (RGB) of the pixel; and

identify the first number of pixels representing the first item from the plurality of pixels in the bounding box by identifying each pixel within the bounding box that is not associated with the unique combination of numerical values assigned to component colors (RGB) of the pixel.

13 . The item tracking system of claim 12 , wherein the particular pre-selected color of each of the second number of pixels comprises a white color with the unique combination of numerical values including (255, 255, 255) assigned to the components (RGB) of the pixel.

14 . A method for identifying an item, comprising:

detecting a triggering event at a platform, wherein the triggering event corresponds to a placement of a first item on the platform;

in response to detecting the triggering event, capturing a plurality of images of the first item on the platform using two or more cameras of a plurality of cameras, wherein each camera is configured to capture images of at least a portion of a platform;

for each image of the first item:

generating a cropped image for the image by editing the image to isolate at least a portion of the first item, wherein generating the cropped image comprises:

establishing a bounding box around the first item in the image, wherein the bounding box comprises an enclosed shape around the first item, wherein the first item occupies a portion of a total area contained within the bounding box;

calculating a ratio between the portion of the total area within the bounding box occupied by the first item and the total area within the bounding box;

determining whether the ratio equals or exceeds a minimum threshold area occupied by the first item in the bounding box;

if the ratio equals or exceeds the minimum threshold area, identifying an item identifier for the first item based on the cropped image, wherein the ratio equaling or exceeding the minimum threshold area indicates that the cropped image comprises sufficient image information relating to the first item to identify the first item; and

if the ratio is below the minimum threshold area, discarding the cropped image, wherein the ratio being below the minimum threshold area indicates that the cropped image comprises insufficient image information relating to the first item to identify the first item;

selecting a particular item identifier identified for a particular cropped image from a set of cropped images that were not discarded;

associating the particular item identifier with the first item; and

displaying an indicator of the particular item identifier on a user interface device.

15 . The method of claim 14 , wherein:

the total area contained within the bounding box associated with the cropped image comprises a plurality of pixels from the image captured by one of the two or more cameras;

the portion of the total area within the bounding box occupied by the first item comprises a first number of pixels from the plurality of pixels representing the first item;

the minimum threshold area comprises a minimum ratio between the first number of pixels and the plurality of pixels; and

calculating the ratio between the portion of the total area within the bounding box occupied by the first item and the total area within the bounding box comprises dividing the first number of pixels representing the item within the bounding box by the total number of pixels in the plurality of pixels contained in the bounding box.

16 . The method of claim 15 , further comprising:

filling a remaining portion of the bounding box not occupied by the first item with a second number of pixels of a particular pre-selected color that is different from one or more colors included in the image of the first item, wherein each pixel of the second number of pixels of the particular pre-selected color is associated with a unique combination of numerical values assigned to component colors red, green and blue (RGB) of the pixel; and

identifying the first number of pixels representing the first item from the plurality of pixels in the bounding box by identifying each pixel within the bounding box that is not associated with the unique combination of numerical values assigned to component colors (RGB) of the pixel.

17 . The method of claim 16 , wherein the particular pre-selected color of each of the second number of pixels comprises a white color with the unique combination of numerical values including (255, 255, 255) assigned to the components (RGB) of the pixel.

18 . The method of claim 14 , further comprising:

storing in a memory an encoded vector library that comprises a plurality of encoded vectors, wherein each encoded vector describes one or more attributes of a particular item and is associated with an item identifier for the particular item; and

wherein identifying an item identifier based on the cropped image comprises:

generating a first encoded vector for the cropped image, wherein the first encoded vector describes one or more attributes of the first item based on the cropped image;

comparing the first encoded vector to the encoded vectors in the encoded vector library;

selecting a second encoded vector from the encoded vector library that most closely matches with the first encoded vector, wherein a numerical similarity value indicates a degree of similarity between the first encoded vector and the selected second encoded vector; and

identifying the item identifier in the encoded vector library that is associated with the second encoded vector.

19 . The method of claim 18 , further comprising:

inputting the cropped image to a machine learning model, wherein the machine learning model is configured to output whether the cropped image is a back image of an item or a front image of an item;

obtaining the output from the machine learning model indicating whether the cropped image is a back image of an item or a front image of an item; and

tagging the cropped image as a back image or a front image based on the output from the machine learning model.

20 . The method of claim 19 , wherein selecting the particular item identifier comprises:

determining whether any of the cropped images generated for the plurality of images are tagged as front images; and

in response to determining that one or more cropped images are tagged as front images, selecting the particular item identifier identified for the particular cropped image from the one or more cropped images tagged as front images.

Assignments (1)

Continuity (4)

References Cited (56)