IP Library › Granted Patent US 12,536,683
Granted Patent B2
US 12,536,683 · App. 18/476,432 · Granted Jan 27, 2026

System and method for confirming the identity of an item based on item height

Inventors: Sumedh Vilas Datar (Grapevine, TX); Crystal Maung (Dallas, TX); Sailesh Bharathwaaj Krishnamurthy (Irving, TX); Nithya Thyagarajan (Flower Mound, TX)
Assignee: 7-Eleven, Inc.
G06T7/55G06V10/40G06V10/751G06V10/764G06Q30/0633
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,536,683
App. No.
18/476,432
Granted
Jan 27, 2026
Kind
B2
Abstract

A device captures an image of a first item and generates a first encoded vector for the image. The device identifies a set of items that have at least one attribute in common with the first item. The device determines the identity of the first item based at least on attributes of the first item. The device determines that a confidence score associated with the identity of the first item is less than a threshold percentage. In response, the device determines a height of the first item. The device identifies item(s) with average heights within a threshold range from the height of the first item. The device compares the first encoded vector with a second encoded vector associated with a second item from the identified item(s). If the first encoded vector corresponds to the second encoded vector, the device determines that the first item corresponds to the second item.

Claims (113)

1 . A system, comprising:

a plurality of cameras, wherein each camera is configured to capture images of at least a portion of a platform;

a memory configured to store an encoded vector library comprising a plurality of encoded vectors, wherein:

each encoded vector describes one or more attributes of a respective item; and

each encoded vector is associated with a respective average height and a standard deviation from the respective average height associated with the respective item;

the standard deviation is a statistical measurement that quantifies an amount of dispersion or variation within a set of height values whose average is the average height; and

one or more processors communicatively coupled to the memory, and configured to:

detect a triggering event at the platform, wherein the triggering event corresponds to a placement of a first item on the platform;

in response to detecting the triggering event, capture an image of the first item using a camera from among the plurality of cameras;

generate a first encoded vector for the image, wherein the first encoded vector describes a plurality of attributes of the first item;

identify a set of items in the encoded vector library that have at least one attribute in common with the first item;

determine an identity of the first item based at least in part upon the plurality of attributes of the first item and the at least one attribute;

determine a confidence score associated with the identity of the first item, wherein the confidence score indicates an accuracy of the identity of the first item;

determine that the confidence score is less than a threshold percentage; and

in response to determining that the confidence score is less than the threshold percentage:

determine, from the image, a height of the first item;

identify one or more items from among the set of items that are associated with average heights within a threshold range from the determined height of the first item;

compare the first encoded vector with a second encoded vector associated with a second item from among the one or more items;

determine that the first encoded vector corresponds to the second encoded vector; and

in response to determining that the first encoded vector corresponds to the second encoded vector, determine that the first item corresponds to the second item.

2 . The system of claim 1 , wherein the one or more processors are further configured to add the first item to a virtual shopping cart associated with a user.

3 . The system of claim 1 , wherein to determine the height associated with the first item, the one or more processors are further configured to:

determine a first distance between the camera and a top surface of the first item;

determine a second distance between the camera and the platform; and

determine a difference between the first distance and the second distance, wherein the height associated with the first item corresponds to the difference between the first distance and the second distance.

4 . The system of claim 1 , wherein for the respective item, the respective average height is determined by:

capturing a plurality of images of the respective item placed on different parts of the platform, wherein each of the plurality of images shows the respective item placed on a different part of the platform;

determining a plurality of heights of the respective item, wherein each of the plurality of heights is determined based at least in part upon a respective distance between the camera and a top surface of the respective item; and

determining an average of the plurality of heights, wherein the average of the plurality of heights corresponds to the respective average height of the respective item.

5 . The system of claim 1 , wherein to determine that the first encoded vector corresponds to the second encoded vector, the one or more processors are further configured to:

identify a first set of attributes associated with the first item, wherein the first set of attributes is indicated in the first encoded vector;

identify a second set of attributes associated with the second item, wherein the second set of attributes is indicated in the second encoded vector;

compare each attribute from among the first set of attributes with a counterpart attribute from among the second set of attributes; and

determine that more than a threshold percentage of the first set of attributes correspond to counterpart attributes from among the second set of attributes.

6 . The system of claim 1 , wherein the standard deviation corresponds to the threshold range from the determined height of the first item.

7 . The system of claim 1 , wherein to determine the confidence score is less than the threshold percentage, the one or more processors are further configured to:

identify a first set of attributes associated with the first item, wherein the first set of attributes is indicated in the first encoded vector; and

for each item from among the set of items:

identify a second set of attributes associated with the item, wherein the second set of attributes is indicated in a respective encoded vector associated with the item;

compare each attribute from among the first set of attributes with a counterpart attribute from among the second set of attributes; and

determine that less than the first set of attributes corresponds to counterpart attributes from among the second set of attributes.

8 . A method comprising:

detecting a triggering event at a platform, wherein the triggering event corresponds to a placement of a first item on the platform;

in response to detecting the triggering event, capturing an image of the first item using a camera from among a plurality of cameras wherein each camera is configured to capture images of at least a portion of a platform;

generating a first encoded vector for the image, wherein the first encoded vector describes a plurality of attributes of the first item;

identifying a set of items in an encoded vector library that have at least one attribute in common with the first item, wherein:

the encoded vector library comprising a plurality of encoded vectors;

each encoded vector describes one or more attributes of a respective item;

each encoded vector is associated with a respective average height and a standard deviation from the respective average height associated with the respective item; and

the standard deviation is a statistical measurement that quantifies an amount of dispersion or variation within a set of height values whose average is the average height;

determining an identity of the first item based at least in part upon the plurality of attributes of the first item and the at least one attribute;

determining a confidence score associated with the identity of the first item, wherein the confidence score indicates an accuracy of the identity of the first item;

determining that the confidence score is less than a threshold percentage; and

in response to determining that the confidence score is less than the threshold percentage:

determining, from the image, a height of the first item;

identifying one or more items from among the set of items that are associated with average heights within a threshold range from the determined height of the first item;

comparing the first encoded vector with a second encoded vector associated with a second item from among the one or more items;

determining that the first encoded vector corresponds to the second encoded vector; and

in response to determining that the first encoded vector corresponds to the second encoded vector, determining that the first item corresponds to the second item.

9 . The method of claim 8 , further comprising adding the first item to a virtual shopping cart associated with a user.

10 . The method of claim 8 , wherein determining the height associated with the first item is in response to:

determining a first distance between the camera and a top surface of the first item;

determining a second distance between the camera and the platform; and

determining a difference between the first distance and the second distance, wherein the height associated with the first item corresponds to the difference between the first distance and the second distance.

11 . The method of claim 8 , wherein for the respective item, the respective average height is determined by:

capturing a plurality of images of the respective item placed on different parts of the platform, wherein each of the plurality of images shows the respective item placed on a different part of the platform;

determining a plurality of heights of the respective item, wherein each of the plurality of heights is determined based at least in part upon a respective distance between the camera and a top surface of the respective item; and

determining an average of the plurality of heights, wherein the average of the plurality of heights corresponds to the respective average height of the respective item.

12 . The method of claim 8 , wherein determining that the first encoded vector corresponds to the second encoded vector is in response to:

identifying a first set of attributes associated with the first item, wherein the first set of attributes is indicated in the first encoded vector;

identifying a second set of attributes associated with the second item, wherein the second set of attributes is indicated in the second encoded vector;

comparing each attribute from among the first set of attributes with a counterpart attribute from among the second set of attributes; and

determining that more than a threshold percentage of the first set of attributes correspond to counterpart attributes from among the second set of attributes.

13 . The method of claim 8 , wherein the standard deviation corresponds to the threshold range from the determined height of the first item.

14 . The method of claim 8 , wherein determining the confidence score is less than the threshold percentage is in response to:

identifying a first set of attributes associated with the first item, wherein the first set of attributes is indicated in the first encoded vector; and

for each item from among the set of items:

identifying a second set of attributes associated with the item, wherein the second set of attributes is indicated in a respective encoded vector associated with the item;

comparing each attribute from among the first set of attributes with a counterpart attribute from among the second set of attributes; and

determining that less than the first set of attributes corresponds to counterpart attributes from among the second set of attributes.

15 . A non-transitory computer-readable medium storing instructions that when executed by one or more processors, cause the one or more processors to:

detect a triggering event at a platform, wherein the triggering event corresponds to a placement of a first item on the platform;

in response to detecting the triggering event, capture an image of the first item using a camera from among a plurality of cameras wherein each camera is configured to capture images of at least a portion of a platform;

generate a first encoded vector for the image, wherein the first encoded vector describes a plurality of attributes of the first item;

identify a set of items in an encoded vector library that have at least one attribute in common with the first item, wherein:

the encoded vector library comprising a plurality of encoded vectors;

each encoded vector describes one or more attributes of a respective item;

each encoded vector is associated with a respective average height and a standard deviation from the respective average height associated with the respective item; and

the standard deviation is a statistical measurement that quantifies an amount of dispersion or variation within a set of height values whose average is the average height;

determine an identity of the first item based at least in part upon the plurality of attributes of the first item and the at least one attribute;

determine a confidence score associated with the identity of the first item, wherein the confidence score indicates an accuracy of the identity of the first item;

determine that the confidence score is less than a threshold percentage; and

in response to determining that the confidence score is less than the threshold percentage:

determine, from the image, a height of the first item;

identify one or more items from among the set of items that are associated with average heights within a threshold range from the determined height of the first item;

compare the first encoded vector with a second encoded vector associated with a second item from among the one or more items;

determine that the first encoded vector corresponds to the second encoded vector; and

in response to determining that the first encoded vector corresponds to the second encoded vector, determine that the first item corresponds to the second item.

16 . The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the one or more processors to add the first item to a virtual shopping cart associated with a user.

17 . The non-transitory computer-readable medium of claim 15 , wherein to determine the height associated with the first item, the instructions further cause the one or more processors to:

determine a first distance between the camera and a top surface of the first item;

determine a second distance between the camera and the platform; and

determine a difference between the first distance and the second distance, wherein the height associated with the first item corresponds to the difference between the first distance and the second distance.

18 . The non-transitory computer-readable medium of claim 15 , wherein for the respective item, the respective average height is determined by:

capturing a plurality of images of the respective item placed on different parts of the platform, wherein each of the plurality of images shows the respective item placed on a different part of the platform;

determining a plurality of heights of the respective item, wherein each of the plurality of heights is determined based at least in part upon a respective distance between the camera and a top surface of the respective item; and

determining an average of the plurality of heights, wherein the average of the plurality of heights corresponds to the respective average height of the respective item.

19 . The non-transitory computer-readable medium of claim 15 , wherein to determine that the first encoded vector corresponds to the second encoded vector, the instructions further cause the one or more processors to:

identify a first set of attributes associated with the first item, wherein the first set of attributes is indicated in the first encoded vector;

identify a second set of attributes associated with the second item, wherein the second set of attributes is indicated in the second encoded vector;

compare each attribute from among the first set of attributes with a counterpart attribute from among the second set of attributes; and

determine that more than a threshold percentage of the first set of attributes correspond to counterpart attributes from among the second set of attributes.

20 . The non-transitory computer-readable medium of claim 15 , wherein a plurality of attributes associated with the first item comprises a brand identifier, a flavor variation identifier, and a container size associated with the first item.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2023
From: DATAR, SUMEDH VILAS; MAUNG, CRYSTAL; KRISHNAMURTHY, SAILESH BHARATHWAAJ; THYAGARAJAN, NITHYA
To: 7-ELEVEN, INC.
Reel/Frame 065058/0068 →
Continuity (4)
Continuation In Part 18366155 · Aug 7, 2023
Continuation In Part 17455903 · Nov 19, 2021
Continuation In Part 17362261 · Jun 29, 2021
Related Publication 20240029284A1 · Jan 25, 2024
References Cited (29)
US 11335092B2 · Zou · 2022 [cited by examiner]
US 11640701B2 · Singh · 2023 [cited by examiner]
US 20130223673A1 · Davis et al. · 2013 [cited by applicant]
US 20130248593A1 · Bonner et al. · 2013 [cited by applicant]
US 20160065856A1 · Sohn et al. · 2016 [cited by applicant]
US 20190108396A1 · Dal Mutto et al. · 2019 [cited by applicant]
US 20240029274A1 · Wang et al. · 2024 [cited by applicant]
CA 3077830A1 · 2018 [cited by examiner]
EP 4358025A1 · 2024 [cited by examiner]
KR 101350387B1 · 2014 [cited by applicant]
KR 20180074976A · 2018 [cited by applicant]
Xinan Wang, System and method for detecting a trigger event for identification of an item, U.S. Appl. No. 18/476,425, filed Sep. 28, 2023. [cited by applicant]
Sailesh B. Krishnamurthy, System and method for selecting an item from a plurality of identified items based on a similarity value, U.S. Appl. No. 18/476,429, filed Sep. 28, 2023. [cited by applicant]
Sailesh B. Krishnamurthy, System and method for camera re-calibration based on an updated homography, U.S. Appl. No. 18/476,468, filed Sep. 28, 2023. [cited by applicant]
Sailesh B. Rishnamurthy, System and method for identifying unmoved items on a platform during item identification, U.S. Appl. No. 18/476,436, filed Sep. 28, 2023. [cited by applicant]
Xinan Wang, System and method for identifying moved items on a platform during item identification, U.S. Appl. No. 18/476,445, filed Sep. 28, 2023. [cited by applicant]
Sailesh Bharathwaaj Krishnamurthy, System and method for item identification using container-based classification, U.S. Appl. No. 18/476,497, filed Sep. 28, 2023. [cited by applicant]
Sumedh Vilas Datar, System and method for selecting an item from a plurality of identified items by filtering out back images of the items, U.S. Appl. No. 18/476,479, filed Sep. 28, 2023. [cited by applicant]
Sailesh B. Krishnamurthy, System and method for identifying an item based on interaction history of a user, U.S. Appl. No. 18/476,512, filed Sep. 28, 2023. [cited by applicant]
Crystal Maung, System and method for space search reduction in identifying items from images via item height, U.S. Appl. No. 18/476,541, filed Sep. 28, 2023. [cited by applicant]
Sumedh V. Datar, System and method for search space reduction for identifying an item, U.S. Appl. No. 18/476,511, filed Sep. 28, 2023. [cited by applicant]
Sumedh V. Datar, System and method for identifying an item based on an amount of image information in an image of the item, U.S. Appl. No. 18/476,555, filed Sep. 28, 2023. [cited by applicant]
Krishnamurthy, S. B. et al., “Hand Detection Trigger for Item Identification,” U.S. Appl. No. 17/455,902, filed Nov. 19, 2021, 108 pages. [cited by applicant]
Datar, S. V. et al., “Item Identification Using Multiple Cameras,” U.S. Appl. No. 17/455,905, filed Nov. 19, 2021, 107 pages. [cited by applicant]
Krishnamurthy, S. B. et al., “Reducing a Search Space for Item Identification Using Machine Learning,”U.S. Appl. No. 17/455,906, filed Nov. 19, 2021, 106 pages. [cited by applicant]
Krishnamurthy, S. B et al., “Image Cropping Using Depth Information,” U.S. Appl. No. 17/455,907, filed Nov. 19, 2021, 108 pages. [cited by applicant]
Maung, C. et al., “Determining Dimensions of an Item Using Point Cloud Information,” U.S. Appl. No. 17/455,908, filed Nov. 19, 2021, 107 pages. [cited by applicant]
Partial Search Report for PCT Patent Application No. PCT/US2022/072832, mailed Oct. 6, 2022; 12 pages. [cited by applicant]
PCT International Search Report and Written Opinion of the Intl. Searching Authority, or the Declaration for International Application No. PCT/US2024/048094, date of mailing Dec. 24, 2024 (Dec. 24, 2024), 11 pages. [cited by applicant]