IP Library Granted Patent US 10,521,645
Granted Patent B2
US 10,521,645 · App. 16/354,175 · Granted Dec 31, 2019

Monitoring a display and shelf of the same product type

Inventors: Yair Adato (Kfar Shmuel, IL); David M. Gottlieb (Denver, CO)
Assignee: TRAX TECHNOLOLGY SOLUTIONS PTE LTD.
G06K9/00201G06F16/23G06F16/235G06F16/288G06F16/55G06F16/583G06F16/5846G06F16/90335G06F17/18G06K9/00362G06K9/00624G06K9/00671G06K9/00771G06K9/325G06K9/6231G06Q10/06311G06Q10/06316G06Q10/063112G06Q10/087G06Q10/0875G06Q20/203G06Q30/0629G06Q30/0639G06Q30/0643G06Q50/28G06T7/0002G06T7/13G06T7/20G06T7/521G06T7/55G06T7/70G06T7/75G08B21/18G08B21/182H04N5/2252H04N5/2253H04N5/23203H04N5/23219H04N5/23229H04N5/247G06K2209/01G06K2209/17G06T2207/30196G06T2207/30232G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,521,645
App. No.
16/354,175
Granted
Dec 31, 2019
Kind
B2
Abstract

The present disclosure relates to systems and methods for identifying products in retail stores based on analysis of image data and for automatically generating performance indicators relating to the identified products. In one implementation, the system may include a processor configured to receive a first set of images depicting a first plurality of products associated with single product type displayed on a first shelving unit in a retail store; analyze the first set of images to determine first product turnover data associated with the first shelving unit; receive a second set of images depicting a second plurality of products also associated with the single product type displayed on a second shelving unit in the retail store nonadjacent to the first shelving unit; and analyze the second set of images to determine second product turnover data associated with the second shelving unit.

Claims (58)

1. A system for identifying products in retail stores based on analysis of image data and for automatically generating performance indicators relating to the identified products, the system comprising:

at least one processor configured to:

receive a first set of images depicting a first plurality of products associated with single product type displayed on a first shelving unit in a retail store;

analyze the first set of images to determine first product turnover data associated with the first shelving unit;

receive a second set of images depicting a second plurality of products also associated with the single product type displayed on a second shelving unit in the retail store nonadjacent to the first shelving unit;

analyze the second set of images to determine second product turnover data associated with the second shelving unit;

automatically generate a performance indicator associated with the single product type using the first product turnover data and the second product turnover data, wherein the performance indicator is determined based on a number of return events associated with the first shelving unit and a number of return events associated with the second shelving unit; and

use image data to identify a return event where a customer placed a product on at least one of the first shelving unit and the second shelving unit.

2. The system of claim 1 , wherein the performance indicator accounts for a contribution of each of the first shelving unit and the second shelving unit to overall sales of the single product type by the retail store.

3. The system of claim 2 , wherein the at least one processor is further configured to:

store records of the performance indicator associated with a period of time; and

generate a report of the contribution of each of the first shelving unit and the second shelving unit to the overall sales of the single product type during the period of time.

4. The system of claim 2 , wherein the at least one processor is further configured to:

determine, based on image analysis of the first and second set of images, demographics of customers that picked up the single product type; and

generate a report showing the demographics of a first group of customers that picked up the single product type from the first shelving unit and the demographics of a second group of customers that picked up the single product type from the second shelving unit.

5. The system of claim 4 , wherein the at least one processor is further configured to:

determine other types of products that customers buy in addition to the single product type; and

generate a report listing types of products that the first group of customers picked up and types of products that the second group of customers picked up.

6. The system of claim 2 , wherein the at least one processor is further configured to:

receive performance indicators from multiple retail stores; and

generate a report showing an average contribution of each of the first shelving unit and the second shelving unit to the overall sales of the single product type.

7. The system of claim 2 , wherein the at least one processor is further configured to:

identify existence of promotions for the single product type in proximity to each of the first shelving unit and the second shelving unit, based on image analysis of the first and second set of images; and

generate a report showing promotional effectiveness on the overall sales of the single product type at each of the first shelving unit and the second shelving unit.

8. The system of claim 2 , wherein the at least one processor is further configured to provide a recommendation to increase the overall sales of the single product type by changing a display parameter associated with at least one of the first shelving unit and the second shelving unit.

9. The system of claim 8 , wherein the display parameter includes at least one of a display location, a display size, and adjacent product types.

10. The system of claim 1 , wherein the first shelving unit is located in a different area of the retail store than the second shelving unit.

11. The system of claim 1 , wherein the first shelving unit has a different display size than the second shelving unit and wherein the first shelving unit is located adjacent to at least one product type that is different from product types adjacent to the second shelving unit.

12. The system of claim 1 , wherein the at least one processor is further configured to:

analyze the first set of images to recognize a first price listed on a label associated with the first shelving unit;

analyze the second set of images to recognize a second price listed on a label associated with the second shelving unit; and

inform a user when there is a price mismatch between the first price and the second price.

13. The system of claim 1 , wherein the at least one processor is further configured to use image data to identify a plurality of sell events where customers picked up products associated with the single product type, and wherein the performance indicator is determined based on a number of sell events associated with the first shelving unit and a number of sell events associated with the second shelving unit.

14. The system of claim 1 , wherein the at least one processor is further configured to use image data to identify a plurality of near-sell events where customers showed interest in the single product type but did not pick up a product, and wherein the performance indicator is determined based on a number of identified near-sell events associated with the first shelving unit and a number of identified near-sell events associated with the second shelving unit.

15. The system of claim 1 , wherein the product is associated with the single product type.

16. The system of claim 1 , wherein the product is associated with a product type other than the single product type.

17. A method identifying products in a retail store based on image analysis of captured images and for automatically generating performance indicators relating to the identified products, the method comprising:

receiving a first set of images depicting a first plurality of products associated with single product type displayed on a first shelving unit in a retail store;

analyzing the first set of images to determine first product turnover data associated with the first shelving unit;

receiving a second set of images depicting a second plurality of products also associated with the single product type displayed on a second shelving unit in the retail store nonadjacent to the first shelving unit;

analyzing the second set of images to determine second product turnover data associated with the second shelving unit;

automatically generating a performance indicator associated with the single product type using the first product turnover data and the second product turnover data, wherein the performance indicator is determined based on a number of return events associated with the first shelving unit and a number of return events associated with the second shelving unit; and

using image data to identify a return event where a customer placed a product on at least one of the first shelving unit and the second shelving unit.

18. The method of claim 17 , wherein the performance indicator accounts for a contribution of each of the first shelving unit and the second shelving unit to overall sales of the single product type by the retail store.

19. A computer program product for identifying products in a retail store based on image analysis of captured images embodied in a non-transitory computer-readable medium and executable by at least one processor, the computer program product including instructions for causing the at least one processor to execute a method comprising:

receiving a first set of images depicting a first plurality of products associated with single product type displayed on a first shelving unit in a retail store;

analyzing the first set of images to determine first product turnover data associated with the first shelving unit;

receiving a second set of images depicting a second plurality of products also associated with the single product type displayed on a second shelving unit in the retail store nonadjacent to the first shelving unit;

analyzing the second set of images to determine second product turnover data associated with the second shelving unit;

automatically generating a performance indicator associated with the single product type using the first product turnover data and the second product turnover data, wherein the performance indicator is determined based on a number of return events associated with the first shelving unit and a number of return events associated with the second shelving unit; and

using image data to identify a return event where a customer placed a product on at least one of the first shelving unit and the second shelving unit.

20. A computer program product for identifying products in a retail store based on image analysis of captured images embodied in a non-transitory computer-readable medium and executable by at least one processor, the computer program product including instructions for causing the at least one processor to execute a method comprising:

receiving a first set of images depicting a first plurality of products associated with single product type displayed on a first shelving unit in a retail store;

analyzing the first set of images to determine first product turnover data associated with the first shelving unit;

receiving a second set of images depicting a second plurality of products also associated with the single product type displayed on a second shelving unit in the retail store nonadjacent to the first shelving unit;

analyzing the second set of images to determine second product turnover data associated with the second shelving unit;

automatically generating a performance indicator associated with the single product type using the first product turnover data and the second product turnover data, wherein the performance indicator is determined based on a number of identified near-sell events associated with the first shelving unit and a number of identified near-sell events associated with the second shelving unit; and

using image data to identify a plurality of near-sell events where customers showed interest in the single product type but did not pick up a product.

Assignments (7)
RELEASE OF SECURITY INTEREST Recorded Feb 10, 2026
From: COMPUTERSHARE TRUST COMPANY, N.A., AS ADMINISTRATIVE AGENT
To: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK, INC.
Reel/Frame 074717/0014 →
RELEASE OF SECURITY INTEREST Recorded Sep 26, 2023
From: ALTER DOMUS (US) LLC
To: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK INC.
Reel/Frame 065028/0871 →
SECURITY INTEREST Recorded Sep 22, 2023
From: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK, INC.
To: COMPUTERSHARE TRUST COMPANY, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 065016/0744 →
RELEASE OF SECURITY INTEREST AT REEL/FRAME NO. 054048/0646 Recorded Oct 27, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: TRAX TECHNOLOGY SOLUTIONS PTE. LTD; SHOPKICK, INC.; CVDM SOLUTIONS SAS; TRAX RETAIL, INC.
Reel/Frame 057944/0338 →
SECURITY INTEREST Recorded Oct 15, 2021
From: TRAX TECHNOLOGY PTE. LTD.; SHOPKICK, INC.
To: ALTER DOMUS (US) LLC
Reel/Frame 058184/0438 →
SECURITY INTEREST Recorded Oct 14, 2020
From: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK, INC.; CVDM SOLUTIONS SAS; TRAX RETAIL, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054048/0646 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 15, 2019
From: ADATO, YAIR; GOTTLIEB, DAVID
To: TRAX TECHNOLOGY SOLUTIONS PTE LTD.
Reel/Frame 048604/0886 →
Continuity (5)
Continuation PCTUS2019013054 · Jan 10, 2019
Provisional Application 62615512 · Jan 10, 2018
Provisional Application 62681718 · Jun 7, 2018
Provisional Application 62695469 · Jul 9, 2018
Related Publication 20190213390A1 · Jul 11, 2019
Cited By (6)
US 12,189,915 US 12,211,161 US 12,430,856 US 12,511,712 US 12,646,347 US 12,688,510