IP Library Granted Patent US 10,762,330
Granted Patent B2
US 10,762,330 · App. 16/354,717 · Granted Sep 1, 2020

Selective usage of product models

Inventors: Yair Adato (Kfar Shmuel, IL); Aviv Eisenschtat (Ramat Hasharon, IL)
Assignee: TRAX TECHNOLOGY SOLUTIONS PTE LTD.
G06K9/00201G06F16/23G06F16/235G06F16/288G06F16/55G06F16/583G06F16/5846G06F16/90335G06F17/18G06K9/00362G06K9/00624G06K9/00671G06K9/00771G06K9/325G06K9/6231G06Q10/06311G06Q10/06316G06Q10/063112G06Q10/087G06Q10/0875G06Q20/203G06Q30/0629G06Q30/0639G06Q30/0643G06Q50/28G06T7/0002G06T7/13G06T7/20G06T7/521G06T7/55G06T7/70G06T7/75G08B21/18G08B21/182H04N5/2252H04N5/2253H04N5/23203H04N5/23219H04N5/23229H04N5/247G06K2209/01G06K2209/17G06T2207/30196G06T2207/30232G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,762,330
App. No.
16/354,717
Granted
Sep 1, 2020
Kind
B2
Abstract

A system for identifying products in a retail store is provided. The system may include a processor configured to: receive an image depicting a store shelf and a product displayed thereon; select a product model subset based on a characteristic of the store shelf determined based on analysis of the image; determine whether the product model subset is applicable to the product; and when the product model subset is applicable to the product, analyze a representation of the product depicted in the image using the product model subset, and identify the product based on the analysis of the representation of the product.

Claims (67)

1. A system for identifying products in a retail store based on analysis of captured images, the system comprising:

at least one processor configured to:

access a database storing a set of product models relating to a plurality of products;

receive at least one image depicting at least one store shelf and at least one product displayed thereon;

select a product model subset from among the set of product models based on at least one characteristic of the at least one store shelf determined based on analysis of the received at least one image, wherein a number of product models included in the product model subset is less than a number of product models included in the set of product models;

determine whether the selected product model subset is applicable to the at least one product; wherein, when the at least one processor determines that the selected product model subset is applicable to the at least one product:

analyze a representation of the at least one product depicted in the at least one image using the product model subset; and

identify the at least one product based on the analysis of the representation of the at least one product depicted in the at least one image using the product model subset;

when the at least one processor determines that the selected product model subset is not applicable to the at least one product:

update the selected product model subset to include at least one additional product model from the stored set of product models not previously included in the selected product model subset to provide an updated product model subset;

analyze the representation of the at least one product depicted in the at least one image in comparison to the updated product model subset; and

identify the at least one product based on the analysis of the representation of the at least one product depicted in the at least one image in comparison to the updated product model subset; and

initiate an action that updates the product model subset associated with the at least one store shelf upon determining that the product model subset is obsolete, wherein updating the product model subset comprises modifying at least one existing product model included in the product model subset based on a detected change of an appearance of the at least one product depicted in the at least one image over the existing product model.

2. The system of claim 1 , wherein when the selected product model subset is not applicable to the at least one product, the at least one processor is further configured to:

provide a visual representation of the at least one product to a user;

receive input from the user regarding the at least one product; and

determine that the at least one product is associated with the at least one additional product model from the stored set of product models not previously included in the selected product model subset based on the received input.

3. The system of claim 1 , wherein when the selected product model subset is not applicable to the at least one product, the at least one processor is further configured to:

obtain information related to recognized types of products adjacent to the at least one product; and

determine that the at least one product is associated with the at least one additional product model from the stored set of product models not previously included in the selected product model subset based on the information related to recognized types of products adjacent to the at least one product.

4. The system of claim 3 , wherein the information related to recognized types of products adjacent to the at least one product used to determine that the at least one product is associated with the at least one additional product model is obtained from analyzing the plurality of images.

5. The system of claim 4 , wherein the information related to recognized types of products adjacent to the at least one product used to determine that the at least one product is associated with the at least one additional product model is obtained from analyzing portions of the plurality of images not depicting the at least one product.

6. The system of claim 1 , wherein the updated product model subset is updated by adding a product model associated with a product brand of the at least one product.

7. The system of claim 1 , wherein the updated product model subset is updated by adding a product model associated with a same logo as the at least one product.

8. The system of claim 1 , wherein the updated product model subset is updated by adding a product model associated with a category of the at least one product.

9. The system of claim 1 , wherein the at least one processor is further configured to select the product model subset based on a characteristic of the at least one store shelf, wherein the characteristic is a location of the at least one store shelf in the store.

10. The system of claim 1 , wherein the at least one processor is further configured to select the product model subset based on a characteristic of the at least one store shelf, wherein the characteristic is at least one of an expected product type and a previously identified product type associated with the at least one store shelf.

11. The system of claim 1 , wherein the at least one processor is further configured to select the product model subset based on a characteristic of the at least one store shelf, wherein the characteristic is a recognized product type of an additional product on the at least one store shelf adjacent the at least one product.

12. The system of claim 1 , wherein determining that the product model subset is obsolete comprises:

determining an elapsed time since a last identification of a product on the at least one store shelf;

comparing the elapsed time with a threshold; and

determining that the product model subset is obsolete based on a result of the comparison.

13. The system of claim 1 , wherein determining that the product model subset is obsolete comprises:

determining a number of product detections since a last identification of a product on the at least one store shelf;

comparing the determined number of product detections with a threshold; and

determining that the product model subset is obsolete based on a result of the comparison.

14. The system of claim 1 , wherein the action that updates the product model subset includes deactivating an existing product model from the product model subset.

15. A The system of claim 1 , wherein the modification to the at least one existing product model is based on a detected change in an appearance of the at least one existing product.

16. The system of claim 1 , wherein the action that updates the product model subset includes replacing an existing product model from the product model subset with a new product model.

17. The system of claim 1 , wherein the at least one processor is further configured to identify a type of the at least one product in a confidence level above a predetermined threshold using the updated product model subset.

18. A method for identifying products in a retail store based on image analysis of captured images, the method comprising:

accessing a database storing a set of product models relating to a plurality of products;

receiving at least one image depicting at least one store shelf and at least one product displayed thereon;

selecting a product model subset from among the set of product models based on at least one characteristic of the at least one store shelf and based on analysis of the at least one image, wherein a number of product models included in the product model subset is less than a number of product models included in the set of product models;

determining whether the selected product model subset is applicable to the at least one product;

when the selected product model subset is determined to be applicable to the at least one product:

analyzing a representation of the at least one product depicted in the at least one image using the product model subset; and

identifying the at least one product based on the analysis of the representation of the at least one product depicted in the at least one image using the product model subset;

when the selected product model subset is determined to be not applicable to the at least one product:

updating the selected product model subset to include at least one additional product model from the stored set of product models not previously included in the selected product model subset to provide an updated product model subset;

analyzing the representation of the at least one product depicted in the at least one image in comparison to the updated product model subset; and

identifying the at least one product based on the analysis of the representation of the at least one product depicted in the at least one image in comparison to the updated product model subset; and

initiating an action that updates the product model subset associated with the at least one store shelf upon determining that the product model subset is obsolete, wherein updating the product model subset comprises modifying at least one existing product model included in the product model subset based on a detected change of an appearance of the at least one product depicted in the at least one image compared to the existing product model.

19. The method of claim 18 , wherein the modification to the at least one existing product model is based on a detected change in an appearance of the at least one existing product.

20. A computer program product for identifying products in a retail store based on image analysis of captured images embodied in a non-transitory computer-readable medium and executable by at least one processor, the computer program product including instructions for causing the at least one processor to execute a method comprising:

accessing a database storing a set of product models relating to a plurality of products;

receiving at least one image depicting at least one store shelf and at least one product displayed thereon;

selecting a product model subset from among the set of product models based on at least one characteristic of the store shelf determined based on analysis of the at least one image, wherein a number of product models included in the product model subset is less than a number of product models included in the set of product models;

determining whether the selected product model subset is applicable to the at least one product;

when the selected product model subset is determined to be applicable to the at least one product:

analyzing a representation of the at least one product depicted in the received at least one image using the product model subset; and

identifying the at least one product based on the analysis of the representation of the at least one product depicted in the received image using the product model subset;

when the selected product model subset is determined to be not applicable to the at least one product:

updating the selected product model subset to include at least one additional product model from the stored set of product models not previously included in the selected product model subset to provide an updated product model subset;

analyzing the representation of the at least one product depicted in the received at least one image in comparison to the updated product model subset; and

identifying the at least one product based on the analysis of the representation of the at least one product depicted in the received at least one image in comparison to the updated product model subset; and

initiating an action that updates the product model subset associated with the at least one store shelf upon determining that the product model subset is obsolete, wherein updating the product model subset comprises modifying at least one existing product model included in the product model subset based on a detected change of an appearance of the at least one product depicted in the at least one image compared to the existing product model.

Assignments (7)
RELEASE OF SECURITY INTEREST Recorded Feb 10, 2026
From: COMPUTERSHARE TRUST COMPANY, N.A., AS ADMINISTRATIVE AGENT
To: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK, INC.
Reel/Frame 074717/0014 →
RELEASE OF SECURITY INTEREST Recorded Sep 26, 2023
From: ALTER DOMUS (US) LLC
To: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK INC.
Reel/Frame 065028/0871 →
SECURITY INTEREST Recorded Sep 22, 2023
From: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK, INC.
To: COMPUTERSHARE TRUST COMPANY, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 065016/0744 →
RELEASE OF SECURITY INTEREST AT REEL/FRAME NO. 054048/0646 Recorded Oct 27, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: TRAX TECHNOLOGY SOLUTIONS PTE. LTD; SHOPKICK, INC.; CVDM SOLUTIONS SAS; TRAX RETAIL, INC.
Reel/Frame 057944/0338 →
SECURITY INTEREST Recorded Oct 15, 2021
From: TRAX TECHNOLOGY PTE. LTD.; SHOPKICK, INC.
To: ALTER DOMUS (US) LLC
Reel/Frame 058184/0438 →
SECURITY INTEREST Recorded Oct 14, 2020
From: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK, INC.; CVDM SOLUTIONS SAS; TRAX RETAIL, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054048/0646 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 15, 2019
From: ADATO, YAIR; EISENSCHTAT, AVIV
To: TRAX TECHNOLOGY SOLUTIONS PTE LTD.
Reel/Frame 048610/0751 →
Continuity (5)
Continuation PCTUS2019013054 · Jan 10, 2019
Provisional Application 62615512 · Jan 10, 2018
Provisional Application 62681718 · Jun 7, 2018
Provisional Application 62695469 · Jul 9, 2018
Related Publication 20190213441A1 · Jul 11, 2019
Cited By (1)
US 12,646,347