IP Library Granted Patent US 11,475,404
Granted Patent B2
US 11,475,404 · App. 16/569,645 · Granted Oct 18, 2022

Aggregating product shortage information

Inventors: Yair Adato (Kfar Shmuel, IL); Ran Taig (Beer Sheva, IL); Alon Grubshtein (Lehavim, IL); Mark Cook (Tel Aviv, IL)
Assignee: Trax Technology Solutions Pte Ltd.
G06Q10/0875G06F16/5866G06K9/6267G06Q10/06315G06Q10/087G06V20/00G06V20/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,475,404
App. No.
16/569,645
Granted
Oct 18, 2022
Kind
B2
Abstract

A system for reducing product shortage durations in retail stores based on analysis of image data is provided. The system may comprise: a communication interface configured to receive image data from retail stores indicative of a product shortage of a product type relative to information describing a placement of products of a product type on a store shelf; and at least one processor configured to: analyze the image data to detect occurrences of product shortages of the product type in the retail stores and determine durations associated with the occurrences; identify a common factor contributing to the duration of part of the occurrences of the product shortages; determine an action, associated with the at least one common factor, for potentially reducing product shortage durations of future shortages of the product type in the retail stores; and provide information associated with the identified action to an entity.

Claims (72)

1. A system for reducing product shortage durations in retail stores based on analysis of image data, the system comprising:

a communication interface; and

at least one processor configured to:

receive digital image data from a plurality of retail stores, wherein the digital image data is (i) indicative of a product shortage of at least one product type relative to information describing a desired placement of products of the at least one product type on at least one store shelf and (ii) based on images captured by at least two image capturing devices in each of the retail stores;

transform the digital image data to generate transformation digital image data by using a transformation function, the transformation function comprising (i) at least one of a convolution, a visual filter, or a nonlinear function and (ii) a depth calculation based on image parallax information associated with at least one product positioned on an opposing retail shelving unit;

analyze the transformation digital image data to detect a plurality of occurrences of product shortages of the at least one product type in the plurality of retail stores and determine product shortage durations associated with the plurality of occurrences by:

identifying the at least one product by applying pixel-based detection to the transformation image data;

comparing the transformation digital image data to a product quantity profile pattern, the product quantity profile pattern having been generated by a first machine learning model, the first machine learning model having been trained using combinations of images and corresponding product quantities; and

determining that values of the transformation digital image data match values of the product quantity profile pattern within a value threshold;

receive employment data from the plurality of retail stores, wherein the employment data includes details about store employees that worked during shifts in which a product shortage of the at least one product type occurred;

identify, by applying a second machine learning model to the analyzed transformation digital image data and the employment data, at least one common factor contributing to product shortage durations of at least part of the plurality of occurrences of the product shortages of the at least one product type in the plurality of retail stores, wherein the second machine learning model is configured to learn relationships between product shortages and contributing factors and predict product shortage data based on potential contributing factors;

determine an action, associated with the at least one common factor, for potentially reducing product shortage durations of future shortages of the at least one product type in the plurality of retail stores; and

provide information associated with the identified action to an entity.

2. The system of claim 1 , wherein the communication interface is configured to provide the information associated with the identified action to a communication device associated with a managing entity of the plurality of retail stores, wherein the information includes a likelihood that the determined action will reduce product shortage durations of future shortages.

3. The system of claim 1 , wherein the communication interface is configured to provide the information associated with the identified action to a communication device associated with a marketing entity, wherein the information includes a prediction of how a change in a shelf size allocated to the at least one product type will reduce product shortage durations of future shortages.

4. The system of claim 1 , wherein the communication interface is configured to provide the information associated with the identified action to a communication device associated with a retail store, wherein the information includes a prediction that a product shortage of a certain product type is about to occur.

5. The system of claim 1 , wherein the communication interface is configured to provide the information associated with the identified action to a communication device associated with a supplier of the at least one product type, wherein the information includes an indication that the product shortage is in noncompliance with contractual agreements of a retail store.

6. The system of claim 1 , wherein the at least one processor is further configured to:

receive inventory data from the plurality of retail stores, wherein the inventory data includes details about a chain-of-supply of products of the at least one product type associated with times in which product shortages of the at least one product type occur; and

identify the at least one common factor based on analysis of the transformation digital image data and the inventory data.

7. The system of claim 6 , wherein the action for potentially reducing product shortage durations of future shortages of the at least one product type includes changing an element in the chain-of-supply of products of the at least one product type.

8. The system of claim 1 , wherein the common factor is based on a number of employees during a shift.

9. The system of claim 1 , wherein the action for potentially reducing product shortage durations of future shortages of the at least one product type includes changing employment dynamics in future shifts associated with shifts in which a product shortage of the at least one product type has occurred.

10. The system of claim 1 , wherein the at least one processor is further configured to:

receive restocking data from the plurality of retail stores, wherein the restocking data includes details about restocking practices of retail stores in which a product shortage of the at least one product type has occurred; and

identify the at least one common factor based on analysis of the transformation digital image data and the restocking data.

11. The system of claim 10 , wherein the action for potentially reducing product shortage durations of future shortages of the at least one product type includes changing restocking practices of retail stores in which a product shortage of the at least one product type has occurred.

12. The system of claim 1 , wherein the at least one processor is further configured to:

receive geographic data from the plurality of retail stores, wherein the geographic data includes locations of retail stores in which a product shortage of the at least one product type has occurred; and

identify the at least one common factor based on analysis of the transformation digital image data and the geographic data.

13. The system of claim 1 , wherein the at least one processor is further configured to:

access historical data associated with the plurality of retail stores, wherein the historical data is indicative of time periods during which a product shortage of the at least one product type has occurred; and

identify the at least one common factor based on analysis of the transformation digital image data and the historical data.

14. The system of claim 1 , wherein the at least one processor is further configured to:

predict a level of effectiveness for the determined action in reducing product shortage durations of future shortages of the at least one product type, and wherein the provided information includes an identification of the determined action and its level of effectiveness.

15. The system of claim 1 , wherein the at least one processor is further configured to:

analyze the transformation digital image data to detect a plurality of occurrences of product shortages of multiple product types in at least one retail store;

identify a pattern associated with the product shortages of the multiple product types; and

provide information associated with the identified pattern to the entity.

16. The system of claim 1 , wherein the at least one processor is further configured to analyze the transformation digital image data by distinguishing a first product represented in the transformation digital image data from a second product using a classification algorithm generated by a third model learning model.

17. The system of claim 1 , wherein comparing the transformation digital image data to the product quantity profile pattern is performed as part of a sequence of fuzzy matching between the transformation digital image data and a plurality of product quantity profile patterns stored in a database.

18. The system of claim 1 , wherein the at least one common factor contributing to the product shortage durations comprises a common location in the plurality of retail stores.

19. The system of claim 1 , wherein identifying the at least one common factor contributing to product shortage using the second machine learning model is based on a quantity or quality threshold dependent on at least one of: the at least one product type being a perishable product type, a current date, a current time, a product location within a retail store, or a type of retail store.

20. The system of claim 1 , wherein the first and second machine learning models are each at least one of: a deep neural network or a convolutional neural network.

21. The system of claim 1 , wherein the digital image data is received as a plurality of digital video data streams from the plurality of retail stores.

22. The system of claim 1 , wherein transforming the digital image data comprises constructing a 3D image.

23. The system of claim 1 , wherein the depth calculation is further based on LIDAR digital image data.

24. A method for reducing product shortage durations in retail stores based on analysis of image data, the method comprising:

receiving digital image data from a plurality of retail stores, wherein the digital image data is (i) indicative of a product shortage of at least one product type relative to information describing a desired placement of products of the at least one product type on at least one store shelf and (ii) based on images captured by at least two image capturing devices in each of the retail stores;

transforming the digital image data to generate transformation digital image data by using a transformation function, the transformation function comprising (i) at least one of a convolution, a visual filter, or a nonlinear function and (ii) a depth calculation based on image parallax information associated with at least one product positioned on an opposing retail shelving unit;

analyzing the transformation digital image data to detect a plurality of occurrences of product shortages of the at least one product type in the plurality of retail stores and determine product shortage durations for the plurality of occurrences by:

identifying the at least one product by applying pixel-based detection to the transformation image data;

comparing the transformation digital image data to a product quantity profile pattern, the product quantity profile pattern having been generated by a first machine learning model, the first machine learning model having been trained using combinations of images and corresponding product quantities; and

determining that values of the transformation digital image data match values of the product quantity profile pattern within a value threshold;

receiving employment data from the plurality of retail stores, wherein the employment data includes details about store employees that worked during shifts in which a product shortage of the at least one product type occurred;

identifying, by applying a second machine learning model to the analyzed transformation digital image data and the employment data, at least one common factor contributing to product shortage durations of at least part of the plurality of occurrences of the product shortages of the at least one product type in the plurality of retail stores, wherein the second machine learning model is configured to learn relationships between product shortages and contributing factors and predict product shortage data based on potential contributing factors;

determining an action, associated with the at least one common factor, for potentially reducing product shortage durations of future shortages of the at least one product type in the plurality of retail stores; and

providing information associated with the identified action to an entity.

25. The method of claim 24 , wherein the at least one product type is considered to be in a state of shortage when a number of products of the at least one product type on the at least one store shelf is below a predefined time duration threshold.

26. The method of claim 25 , wherein the predefined time duration threshold is determined based on the at least one product type.

27. The method of claim 25 , wherein the predefined time duration threshold is determined per retail store.

28. A computer program product for reducing product shortage durations in retail stores based on analysis of image data embodied in a non-transitory computer-readable medium and executable by at least one processor, the computer program product including instructions for causing the at least one processor to execute a method comprising:

receiving digital image data from a plurality of retail stores, wherein the digital image data is (i) indicative of a product shortage of at least one product type relative to information describing a desired placement of products of the at least one product type on at least one store shelf and (ii) based on images captured by at least two image capturing devices in each of the retail stores;

transforming the digital image data to generate transformation digital image data by using a transformation function, the transformation function comprising (i) at least one of a convolution, a visual filter, or a nonlinear function and (ii) a depth calculation based on image parallax information associated with at least one product positioned on an opposing retail shelving unit;

analyzing the transformation digital image data to detect a plurality of occurrences of product shortages of the at least one product type in the plurality of retail stores and determine product shortage durations for the plurality of occurrences by:

identifying the at least one product by applying pixel-based detection to the transformation image data;

comparing the transformation digital image data to a product quantity profile pattern, the product quantity profile pattern having been generated by a first machine learning model, the first machine learning model having been trained using combinations of images and corresponding product quantities; and

determining that values of the transformation digital image data match values of the product quantity profile pattern within a value threshold;

receiving employment data from the plurality of retail stores, wherein the employment data includes details about store employees that worked during shifts in which a product shortage of the at least one product type occurred;

identifying, by applying a second machine learning model to the analyzed transformation digital image data and the employment data, at least one common factor contributing to product shortage durations of at least part of the plurality of occurrences of the product shortages of the at least one product type in the plurality of retail stores, wherein the second machine learning model is configured to learn relationships between product shortages and contributing factors and predict product shortage data based on potential contributing factors;

determining an action, associated with the at least one common factor, for potentially reducing product shortage durations of future shortages of the at least one product type in the plurality of retail stores; and

providing information associated with the identified action to an entity.

Assignments (7)
RELEASE OF SECURITY INTEREST Recorded Feb 10, 2026
From: COMPUTERSHARE TRUST COMPANY, N.A., AS ADMINISTRATIVE AGENT
To: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK, INC.
Reel/Frame 074717/0014 →
RELEASE OF SECURITY INTEREST Recorded Sep 26, 2023
From: ALTER DOMUS (US) LLC
To: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK INC.
Reel/Frame 065028/0871 →
SECURITY INTEREST Recorded Sep 22, 2023
From: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK, INC.
To: COMPUTERSHARE TRUST COMPANY, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 065016/0744 →
RELEASE OF SECURITY INTEREST AT REEL/FRAME NO. 054048/0646 Recorded Oct 27, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: TRAX TECHNOLOGY SOLUTIONS PTE. LTD; SHOPKICK, INC.; CVDM SOLUTIONS SAS; TRAX RETAIL, INC.
Reel/Frame 057944/0338 →
SECURITY INTEREST Recorded Oct 15, 2021
From: TRAX TECHNOLOGY PTE. LTD.; SHOPKICK, INC.
To: ALTER DOMUS (US) LLC
Reel/Frame 058184/0438 →
SECURITY INTEREST Recorded Oct 14, 2020
From: TRAX TECHNOLOGY SOLUTIONS PTE. LTD.; SHOPKICK, INC.; CVDM SOLUTIONS SAS; TRAX RETAIL, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054048/0646 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2020
From: ADATO, YAIR; TAIG, RAN; GRUBSHTEIN, ALON; COOK, MARK
To: TRAX TECHNOLOGY SOLUTIONS PTE LTD.
Reel/Frame 051909/0270 →
Continuity (4)
Continuation PCTUS2019049528 · Sep 4, 2019
Provisional Application 62773427 · Nov 30, 2018
Provisional Application 62727301 · Sep 5, 2018
Related Publication 20200074391A1 · Mar 5, 2020
Cited By (2)
US 12,228,528 US 12,367,593