IP Library › Granted Patent US 12,254,750
Granted Patent B1
US 12,254,750 · App. 18/544,904 · Granted Mar 18, 2025

Image analysis in a mechanized store for monitoring items on a display

Inventor: Bernd Schoner (New York, NY)
Assignee: Standard Cognition Corp.
G07G1/0045G06Q10/087
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,254,750
App. No.
18/544,904
Granted
Mar 18, 2025
Kind
B1
Abstract

Image analysis from one or more cameras are used for monitoring items on a display. A count of items on the display is received. Images from the one or more cameras are used to count a number of items removed from the display. A report is generated containing a number of items on the display, after counting the number of items removed from the display using the images from the one or more cameras.

Claims (39)

1. A system for a mechanized store comprising:

a display is in a store operated by a retailer, wherein a space for the display is leased from the retailer by a vendor;

one or more cameras; and

one or more memory devices comprising instructions that, when executed, causes the one or more processors to perform the following steps:

receiving a count of items placed on the display;

counting a number of items removed from the display, using images from the one or more cameras;

generating a report containing a number of items on the display, after counting the number of items removed from the display using the images from the one or more cameras; and

transmitting the report to the vendor.

2. The system of claim 1 , wherein counting the number of items removed from the display comprises ascertaining that a hand is holding an object and moving away from the display.

3. The system of claim 1 , wherein the display is in a refrigerated cooler.

4. The system of claim 1 , wherein the display is in a vehicle.

5. The system of claim 1 , wherein the report is an out-of-stock report.

6. The system of claim 1 , wherein the images from the one or more cameras are relayed on a dedicated cellular uplink.

7. A method for a mechanized store comprising:

receiving a count of items placed on a display;

counting a number of items removed from the display, using images from one or more cameras, wherein counting the number of items removed from the display comprises ascertaining that a hand is holding an object and moving away from the display; and

generating a report containing a number of items on the display, after counting the number of items removed from the display using the images from the one or more cameras.

8. The method of claim 7 , further comprising counting a number of items put on the display, using the images from the one or more cameras, wherein the number of items on the display in the report is based on the count of items placed on the display, the number of items removed from the display, and the number of items put on the display.

9. The method of claim 7 , wherein the report is an out-of-stock report.

10. The method of claim 7 , wherein the report is a quasi-real-time inventory.

11. The method of claim 7 , wherein:

the display is in a store operated by a retailer;

a space for the display is leased from the retailer by a vendor; and

the method further comprises transmitting the report to the vendor.

12. The method of claim 7 , wherein the report is one of many reports transmitted continuously over a period of time.

13. The method of claim 7 , wherein:

the display is in a store; and

the one or more cameras are part of an information technology (IT) system of the store.

14. The method of claim 7 , wherein the images from the one or more cameras are relayed on a dedicated cellular uplink.

15. The method of claim 14 , wherein the dedicated cellular uplink is operated at a relatively low duty cycle and powered by a battery.

16. The method of claim 7 , further comprising reading an electronic key on a mobile device of a user before the user removes an item from the display.

17. One or more memory devices comprising instructions that, when executed, cause one or more processors to perform the following steps:

receiving a count of items placed on a display, wherein the display is in a store operated by a retailer, and a space for the display is leased from the retailer by a vendor;

counting a number of items removed from the display, using images from one or more cameras;

generating a report containing a number of items on the display, after counting the number of items removed from the display using the images from the one or more cameras; and

transmitting the report to the vendor.

18. The one or more memory devices of claim 17 , wherein counting the number of items removed from the display comprises ascertaining that a hand is holding an object and moving away from the display.

19. The one or more memory devices of claim 17 , wherein the report is an out-of-stock report.

20. The one or more memory devices of claim 17 , wherein the images from the one or more cameras are relayed on a dedicated cellular uplink.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 19, 2023
From: SCHONER, BERND
To: DEEPMAGIC, INC.
Reel/Frame 065909/0386 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 19, 2023
From: DEEPMAGIC, INC.
To: STANDARD COGNITION CORP.
Reel/Frame 066074/0990 →
Continuity (13)
Continuation 18046279 · Oct 13, 2022
Continuation 17393095 · Aug 3, 2021
Continuation 17208588 · Mar 22, 2021
Continuation 16726714 · Dec 24, 2019
Continuation 16217442 · Dec 12, 2018
Continuation In Part 15853252 · Dec 22, 2017
Continuation 15495555 · Apr 24, 2017
Provisional Application 62776038 · Dec 6, 2018
Provisional Application 62696990 · Jul 12, 2018
Provisional Application 62640018 · Mar 8, 2018
Provisional Application 62612317 · Dec 29, 2017
Provisional Application 62486665 · Apr 18, 2017
Provisional Application 62326931 · Apr 25, 2016
References Cited (97)
US 1037779A · Kusebauch et al. · 1912 [cited by applicant]
US 4746830A · Holland · 1988 [cited by applicant]
US 5734722A · Halpern · 1998 [cited by applicant]
US 5745036A · Clare · 1998 [cited by applicant]
US 6584375B2 · Bancroft et al. · 2003 [cited by applicant]
US 7240027B2 · Mcconnell et al. · 2007 [cited by applicant]
US 7693757B2 · Zimmerman et al. · 2010 [cited by applicant]
US 7742623B1 · Moon et al. · 2010 [cited by applicant]
US 8452868B2 · Shafer et al. · 2013 [cited by applicant]
US 8577705B1 · Baboo et al. · 2013 [cited by applicant]
US 8751316B1 · Fletchall et al. · 2014 [cited by applicant]
US 9033229B2 · Matsuhisa et al. · 2015 [cited by applicant]
US 9147174B2 · Glickman et al. · 2015 [cited by applicant]
US 9886827B2 · Schoner · 2018 [cited by applicant]
US 9911290B1 · Zalewski et al. · 2018 [cited by applicant]
US 9996818B1 · Ren et al. · 2018 [cited by applicant]
US 10165194B1 · Baldwin · 2018 [cited by applicant]
US 10169660B1 · Ren · 2019 [cited by examiner]
US 10192408B2 · Schoner · 2019 [cited by applicant]
US 10268983B2 · Kumar et al. · 2019 [cited by applicant]
US 10504230B1 · Stahl et al. · 2019 [cited by applicant]
US 10699536B1 · Schoner · 2020 [cited by applicant]
US 10699537B1 · Schoner · 2020 [cited by applicant]
US 10846996B2 · Schoner · 2020 [cited by applicant]
US 10950099B1 · Schoner · 2021 [cited by applicant]
US 10950100B1 · Schoner · 2021 [cited by applicant]
US 10957168B1 · Schoner · 2021 [cited by applicant]
US 11488455B1 · Schoner · 2022 [cited by applicant]
US 11881091B1 · Schoner · 2024 [cited by applicant]
US 20060163349A1 · Neugebauer · 2006 [cited by applicant]
US 20060289637A1 · Brice et al. · 2006 [cited by applicant]
US 20080159634A1 · Sharma et al. · 2008 [cited by applicant]
US 20100138281A1 · Zhang et al. · 2010 [cited by applicant]
US 20130060708A1 · Oskolkov et al. · 2013 [cited by applicant]
US 20130223673A1 · Davis et al. · 2013 [cited by applicant]
US 20130235206A1 · Smith et al. · 2013 [cited by applicant]
US 20130282522A1 · Hassan · 2013 [cited by applicant]
US 20130332271A1 · Hay · 2013 [cited by applicant]
US 20140316916A1 · Hay · 2014 [cited by applicant]
US 20150012396A1 · Puerini et al. · 2015 [cited by applicant]
US 20150019391A1 · Kumar et al. · 2015 [cited by applicant]
US 20150029339A1 · Kobres et al. · 2015 [cited by applicant]
US 20150039458A1 · Reid · 2015 [cited by applicant]
US 20150120364A1 · Deshpande · 2015 [cited by examiner]
US 20150222861A1 · Fujii et al. · 2015 [cited by applicant]
US 20150254603A1 · Bashkin · 2015 [cited by applicant]
US 20150262116A1 · Katircioglu et al. · 2015 [cited by applicant]
US 20150379366A1 · Nomura · 2015 [cited by examiner]
US 20160042602A1 · Phan et al. · 2016 [cited by applicant]
US 20160110679A1 · Putnam et al. · 2016 [cited by applicant]
US 20160162715A1 · Luk et al. · 2016 [cited by applicant]
US 20160358145A1 · Montgomery · 2016 [cited by applicant]
US 20170309136A1 · Schoner · 2017 [cited by applicant]
US 20170323376A1 · Glaser et al. · 2017 [cited by applicant]
US 20180070056A1 · DeAngelis et al. · 2018 [cited by applicant]
US 20180122194A1 · Schoner · 2018 [cited by applicant]
US 20180150788A1 · Vepakomma et al. · 2018 [cited by applicant]
US 20190147709A1 · Schoner · 2019 [cited by applicant]
US 20190236635A1 · Taylor et al. · 2019 [cited by applicant]
US 20190392318A1 · Ghafoor et al. · 2019 [cited by applicant]
US 20200074165A1 · Ghafoor et al. · 2020 [cited by applicant]
CA 2497795A1 · 2006 [cited by applicant]
GB 2566762A · 2019 [cited by applicant]
WO 2016044814A1 · 2016 [cited by applicant]
WO 2020225562A1 · 2020 [cited by applicant]
U.S. Appl. No. 15/495,555, First Action Interview Pilot Program Pre-Interview Communication, Mailed On Aug. 10, 2017, 7 pages. [cited by applicant]
U.S. Appl. No. 15/495,555, Notice of Allowance, Mailed On Sep. 28, 2017, 9 pages. [cited by applicant]
U.S. Appl. No. 15/853,252, Non-Final Office Action, Mailed On Feb. 23, 2018, 7 pages. [cited by applicant]
U.S. Appl. No. 15/853,252, Notice of Allowance, Mailed On Sep. 6, 2018, 7 pages. [cited by applicant]
U.S. Appl. No. 16/217,442, Non- Final Office Action, Mailed On Nov. 27, 2019, 5 pages. [cited by applicant]
U.S. Appl. No. 16/217,442, Notice of Allowance, Mailed On Jul. 23, 2020, 7 pages. [cited by applicant]
U.S. Appl. No. 16/691,395, First Action Interview Pilot Program Pre-Interview Communication, Mailed On Jan. 23, 2020, 8 pages. [cited by applicant]
U.S. Appl. No. 16/691,395, Notice of Allowance, Mailed On May 1, 2020, 8 pages. [cited by applicant]
U.S. Appl. No. 16/708,149, First Action Interview Pilot Program Pre-Interview Communication, Mailed On Jan. 23, 2020, 8 pages. [cited by applicant]
U.S. Appl. No. 16/708,149, Notice of Allowance, Mailed On May 1, 2020, 8 pages. [cited by applicant]
U.S. Appl. No. 16/713,576, First Action Interview Pilot Program Pre-Interview Communication, Mailed On Sep. 3, 2020, 7 pages. [cited by applicant]
U.S. Appl. No. 16/713,576, Notice of Allowance, Mailed On Nov. 12, 2020, 9 pages. [cited by applicant]
U.S. Appl. No. 16/726,615, First Action Interview Pilot Program Pre-Interview Communication, Mailed On Sep. 3, 2020, 7 pages. [cited by applicant]
U.S. Appl. No. 16/726,615, Notice of Allowance, Mailed On Nov. 12, 2020, 9 pages. [cited by applicant]
U.S. Appl. No. 16/726,714, First Action Interview Pilot Program Pre-Interview Communication, Mailed On Sep. 3, 2020, 7 pages. [cited by applicant]
U.S. Appl. No. 16/726,714, Notice of Allowance, Mailed On Nov. 12, 2020, 9 pages. [cited by applicant]
U.S. Appl. No. 17/208,588, Advisory Action, Mailed On Nov. 28, 2022, 7 pages. [cited by applicant]
U.S. Appl. No. 17/208,588, Final Office Action, Mailed On Sep. 1, 2022, 16 pages. [cited by applicant]
U.S. Appl. No. 17/208,588, Non-Final Office Action, Mailed On Mar. 3, 2022, 14 pages. [cited by applicant]
U.S. Appl. No. 17/208,588, Non-Final Office Action, Mailed On Mar. 2, 2023, 18 pages. [cited by applicant]
U.S. Appl. No. 17/393,095, Final Office Action, Mailed On Mar. 21, 2022, 10 pages. [cited by applicant]
U.S. Appl. No. 17/393,095, Non-Final Office Action, Mailed On Nov. 12, 2021, 9 pages. [cited by applicant]
U.S. Appl. No. 17/393,095, Notice of Allowance, Mailed On Jun. 27, 2022, 9 pages. [cited by applicant]
U.S. Appl. No. 18/046,279, Non-Final Office Action, Mailed On Mar. 30, 2023, 10 pages. [cited by applicant]
U.S. Appl. No. 18/046,279, Notice of Allowance, Mailed On Sep. 15, 2023, 9 pages. [cited by applicant]
Unknown Author, “Amazon Go Frequently Asked Questions”, Retrieved from: https://www.amazon.com/b?node=16008589011, May 5, 2017. [cited by applicant]
Cao et al., “Leveraging Convolutional Pose Machines for Fast and Accurate Head Pose Estimation”, 2018 IEEE/RSJ International Conference On Intelligent Robots and Systems (IROS), IEEE, Oct. 1, 2018, 6 pages. [cited by applicant]
Unknown Author, Oakland, CA, https://edyn.com/, May 5, 2017. [cited by applicant]
Keulian , “Naraffar, The First Staffless Convenience Store”, Your fix of retail insights Blog. Retrieved from: http://stephanekeulian.com/en/naraffar-first-staffeesconvenience-store/, May 5, 2017. [cited by applicant]
Prindle , “Digital Trends”, This Automated Store in Sweden Doesn't have Any Human Employees—Only a Smartphone App Retrieved from: https://www.digitaltrends.com/cooltech/sweden-app-enabled-automated-store/, Apr. 27, 2017. [cited by applicant]
Salvagnini et al., “Person Re-Identification With A PTZ Camera: An Introductory Study”, Istituto Italiano di Tecnologia Pattern Analysis & Computer Vision Via Morego 30, 2013, 5 pages. [cited by applicant]
Wei et al., “Convolutional Pose Machines”, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Jun. 27, 2016, pp. 4724-4732. [cited by applicant]