IP Library Granted Patent US 12,223,458
Granted Patent B2
US 12,223,458 · App. 18/316,606 · Granted Feb 11, 2025

Analyzing images to assess task performance

Inventors: Youval Bronicki (Los Altos, CA); Sheila Meftah (San Francisco, CA)
Assignee: Trax Technology Solutions Pte Ltd.
G06Q10/06395G01C21/206G01G19/52G05D1/0219G05D1/0246G06F18/2431G06F18/253G06F18/285G06N5/04G06N20/00G06Q10/06311G06Q10/06315G06Q10/06375G06Q10/0639G06Q30/018G06Q30/0201G06Q30/0208G06Q30/0281G06Q30/0601G06Q30/0609G06Q30/0623G06Q30/0639G06Q30/0643G06T7/0002G06T7/50G06T7/70G06T7/73G06T7/80G06T7/97G06V10/82G06V20/10G06V20/20G06V20/36G06V20/52H04L67/303H04W4/024H04W4/029A47L11/4011A47L2201/04G01G19/14G01G23/18G01J1/4204G01L1/00G06Q10/087G06Q30/0633G06T2207/30242G06V2201/09H04N7/188
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,223,458
App. No.
18/316,606
Granted
Feb 11, 2025
Kind
B2
Abstract

A non-transitory computer-readable medium includes instructions that when executed by a processor cause the processor to perform a method for assessing quality of tasks performed by persons in retail stores. The method may include receiving an indication that a person completed a task corresponding to at least one shelf in a retail store. The method may also include receiving at least one image of the at least one shelf, the at least one image being captured using an image sensor after the completion of the task. The method may further include analyzing the at least one image to determine at least one property associated with performing the task. The method may further also using the at least one property to determine a reward for performing the task.

Claims (53)

1. A non-transitory computer-readable medium including instructions that when executed by a processor cause the processor to perform a method for assessing quality of tasks performed by persons in retail stores, the method comprising:

receiving an indication that a person completed a task corresponding to at least one shelf in a retail store;

receiving at least one image of the at least one shelf, the at least one image being captured using an image sensor after the completion of the task;

analyzing the at least one image to determine at least one property associated with performing the task, wherein analyzing the at least one image to determine the at least one property comprises:

identifying at least one visual indicator in the at least one image, the visual indicator comprising at least one of: a color, a brightness, a character, or a shape;

comparing the at least one visual indicator to a reference visual indicator; and

determining a degree of similarity between the identified at least one visual indicator and the reference visual indicator; and

using the at least one property to determine a reward for performing the task.

2. The non-transitory computer-readable medium of claim 1 , wherein the indication is based on an input from the person.

3. The non-transitory computer-readable medium of claim 1 , wherein the indication is based on an analysis of the at least one image.

4. The non-transitory computer-readable medium of claim 1 , wherein the at least one property includes a quality indication of at least one aspect of performing the task, and the determination of the reward is based on the quality indication.

5. The non-transitory computer-readable medium of claim 1 , wherein the at least one image is at least one image captured by at least one of the person or an image sensor mounted to a shelf.

6. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises:

analyzing the at least one image to determine a property of the person; and basing the determination of the reward on the property of the person.

7. The non-transitory computer-readable medium of claim 1 , wherein the task includes at least one of a restocking of a product associated with the at least one shelf, a correction of product facings at the at least one shelf, removing a product from the at least one shelf, changing a price of at least one product, or changing an orientation of at least one product on the at least one shelf.

8. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises:

analyzing the at least one image to determine at least one additional available task corresponding to the at least one shelf in the retail store; and

providing an indication of the additional available task to the person.

9. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises:

analyzing the at least one image to determine that the person performed a positive action corresponding to the at least one shelf in the retail store, the positive action not being included in the task; and

basing the determination of the reward on the positive action.

10. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises:

determining an impact of the performed task; and basing the determination of the reward on the impact.

11. The non-transitory computer-readable medium of claim 1 , wherein the task includes at least one of positioning or removing a promotional sign at the at least one shelf.

12. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises:

analyzing the at least one image to determine at least one aspect lacking in the performance of the task;

providing an indication of the at least one aspect to the person; receiving at least one additional image of the at least one shelf, the at least one image being captured using the image sensor after the indication of the at least one aspect is provided; and

analyzing the at least one additional image to determine the reward for performing the task.

13. The non-transitory computer-readable medium of claim 1 , wherein the reward includes a monetary reward, and the determination of the reward includes a determination of an amount associated with the monetary reward.

14. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises:

receiving input from at least one pressure sensor positioned on the at least one shelf; and basing the determination of the at least one property on an analysis of the received input.

15. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises:

receiving input from at least one touch sensor positioned on the at least one shelf; and basing the determination of the at least one property on an analysis of the received input.

16. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises:

receiving input from at least one weight sensor positioned on the at least one shelf; and basing the determination of the at least one property on an analysis of the received input.

17. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises:

receiving input from at least one light sensor positioned on the at least one shelf; and basing the determination of the at least one property on an analysis the received input.

18. A system for assessing quality of tasks performed by persons in retail stores, the system comprising:

at least one processor; and

a non-transitory computer-readable medium containing instructions that, when executed by the at least one processor, cause the at least one processor to:

receive an indication that a person completed a task corresponding to at least one shelf in a retail store;

receive at least one image of the at least one shelf captured using an image sensor after the completion of the task;

analyze the at least one image to determine at least one property associated with performing the task, wherein analyzing the at least one image to determine the at least one property comprises:

identifying at least one visual indicator in the at least one image, the visual indicator comprising at least one of: a color, a brightness, a character, or a shape;

comparing the at least one visual indicator to a reference visual indicator; and determining a degree of similarity between the identified at least one visual indicator and the reference visual indicator; and

use the at least one property to determine a reward for performing the task.

19. A method for assessing quality of tasks performed by persons in retail stores, comprising:

receiving an indication that a person completed a task corresponding to at least one shelf in a retail store;

receiving at least one image of the at least one shelf captured using an image sensor after the completion of the task;

analyzing the at least one image to determine at least one property associated with performing the task, wherein analyzing the at least one image to determine the at least one property comprises:

identifying at least one visual indicator in the at least one image, the visual indicator comprising at least one of: a color, a brightness, a character, or a shape;

comparing the at least one visual indicator to a reference visual indicator; and determining a degree of similarity between the identified at least one visual indicator and the reference visual indicator; and

using the at least one property to determine a reward for performing the task.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 12, 2023
From: BRONICKI, YOUVAL; MEFTAH, SHEILA
To: TRAX TECHNOLOGY SOLUTIONS PTE LTD.
Reel/Frame 063626/0874 →
Continuity (8)
Continuation 17661929 · May 4, 2022
Continuation 17357554 · Jun 24, 2021
Continuation PCTUS2021034885 · May 28, 2021
Provisional Application 63113490 · Nov 13, 2020
Provisional Application 63091009 · Oct 13, 2020
Provisional Application 63084086 · Sep 28, 2020
Provisional Application 63033055 · Jun 1, 2020
Related Publication 20230281549A1 · Sep 7, 2023
References Cited (41)
US 8064896B2 · Bell et al. · 2011 [cited by applicant]
US 8577136B1 · Ascher et al. · 2013 [cited by applicant]
US 9720707B1 · Shook · 2017 [cited by examiner]
US 10755296B2 · Teevan et al. · 2020 [cited by applicant]
US 20040034684A1 · Payne · 2004 [cited by applicant]
US 20090216629A1 · James · 2009 [cited by examiner]
US 20120265574A1 · Olding et al. · 2012 [cited by applicant]
US 20130018701A1 · Dusig et al. · 2013 [cited by applicant]
US 20130132987A1 · Narasimhan et al. · 2013 [cited by applicant]
US 20140257951A1 · Celis et al. · 2014 [cited by applicant]
US 20140278850A1 · Boccuzzi, Jr. et al. · 2014 [cited by applicant]
US 20150092061A1 · Chao et al. · 2015 [cited by applicant]
US 20150121387A1 · Chang et al. · 2015 [cited by applicant]
US 20150199641A1 · Napoli · 2015 [cited by applicant]
US 20160307143A1 · Mongeon · 2016 [cited by examiner]
US 20160350697A1 · Gruen · 2016 [cited by applicant]
US 20170177969A1 · Zaremski et al. · 2017 [cited by applicant]
US 20170221156A1 · Mingarelli et al. · 2017 [cited by applicant]
US 20180005176A1 · Williams et al. · 2018 [cited by applicant]
US 20180005500A1 · Robertson · 2018 [cited by examiner]
US 20180018687A1 · Ouimet · 2018 [cited by applicant]
US 20180288722A1 · Blaster et al. · 2018 [cited by applicant]
US 20180336513A1 · Smith et al. · 2018 [cited by applicant]
US 20190057486A1 · Kumar et al. · 2019 [cited by applicant]
US 20190076731A1 · Hickman et al. · 2019 [cited by applicant]
US 20190149725A1 · Adato · 2019 [cited by examiner]
US 20190384989A1 · Yamazaki et al. · 2019 [cited by applicant]
US 20210097478A1 · Yang et al. · 2021 [cited by applicant]
WO WO2018002709A2 · 2018 [cited by applicant]
WO WO2018002709A3 · 2018 [cited by applicant]
WO WO2019048924A1 · 2019 [cited by applicant]
WO WO2019140091A1 · 2019 [cited by applicant]
Search Report on PCT Application No. PCT/US2021/034885 dated Nov. 12, 2021 (17 pages). [cited by applicant]
Office Action issued on Oct. 26, 2021 for U.S. Appl. No. 17/364,998. [cited by applicant]
“Crowd Delegate: An MCS-Based Approach for Improving Retail Labor Cost-Efficiency” SB Azmy, N Zorba . . . —2019 IEEE Global, 2019—ieeexplore.ieee.org (Year: 2019). [cited by applicant]
“Indirect Object-sensing Technology to Prevent Out-of-stock at Retail-level” C Metzger—Proceedings of the Workshop on Smart Object at the, 2005—Citeseer (Year: 2005). [cited by applicant]
“Robust shelf monitoring using supervised learning for improving on-shelf availability in retail stores” K Higa, K Iwamoto—Sensors, 2019—mdpi.com (Year: 2019). [cited by applicant]
Context-aware hybrid classification system for fine-grained retail product recognition I Baz, E Yoruk, M Cetin—2016 IEEE 12th Image, Video, and , 2016—ieeexplore.ieee.org (Year: 2016). [cited by applicant]
An Automated Vision Based Change Detection Method for Planogram Compliance in Retail Stores M Muthugnanambika, T Bagyammal—Vision and Bio Inspired, 2018—Springer (Year: 2018). [cited by applicant]
Non-Final Office Action issued on Sep. 13, 2021 for U.S. Appl. No. 17/363,220. [cited by applicant]
Final Office Action issued on Mar. 17, 2022 for U.S. Appl. No. 17/363,220. [cited by applicant]
Cited By (1)
US 12,518,109