IP Library › Granted Patent US 12,231,818
Granted Patent B2
US 12,231,818 · App. 18/379,629 · Granted Feb 18, 2025

Managing constraints for automated design of camera placement and cameras arrangements for autonomous checkout

Inventors: Nagasrikanth Kallakuri (San Francisco, CA); Akshay Malhotra (Fremont, CA); Luis Yoichi Morales Saiki (San Francisco, CA); Tushar Dadlani (Dublin, CA); Dhananjay Singh (San Francisco, CA)
Assignee: STANDARD COGNITION, CORP.
H04N7/181A47F10/02G06N20/00G06Q20/208G06T7/70A47F2010/025G06T2207/20081G06T2207/30232
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,231,818
App. No.
18/379,629
Granted
Feb 18, 2025
Kind
B2
Abstract

Techniques for managing coverage constraints are provided for determining an improved camera coverage plan including a number, a placement, and a pose of cameras that are arranged to track puts and takes of items by subjects in a three-dimensional real space. The method includes receiving an initial camera coverage plan including a three-dimensional map of a real space, an initial number and initial pose of a plurality of cameras and a camera model including characteristics of the cameras. The method can iteratively apply a machine learning process to an objective function of number and poses of cameras, and subject to a set of constraints, obtain an improved camera coverage plan. The improved camera coverage plan is provided to an installer to arrange cameras to track puts and takes of items by subjects in the three-dimensional real space.

Claims (45)

1. A computer-implemented method for determining an improved camera coverage plan including a number, a placement, and a pose of cameras that are arranged to track puts and takes of items by subjects in a three-dimensional real space, the computer-implemented method including:

beginning with an initial number and initial pose of a plurality of cameras and a camera model including characteristics of the cameras, iteratively applying a machine learning process to an objective function of number and poses of cameras, and subject to a set of constraints, to obtain, an improved camera coverage plan using one or more of: (i) a changed number of cameras, and (ii) a changed number of camera poses, and having an improved coverage score and concurrently using a same or reduced number of cameras, wherein the set of constraints includes physical constraints and coverage constraints; and wherein coverage constraints include a number of cameras having a voxel in a structure or display holding inventory within view, a number of cameras having a voxel in a tracking zone of volume in which subjects are tracked within view, a difference in angles of incidence between cameras having a voxel within view, and an overall coverage of the three-dimensional real space; and

providing the improved camera coverage plan to an installer to arrange cameras to track puts and takes of items by subjects in the three-dimensional real space.

2. The computer-implemented method of claim 1 , wherein applying a machine learning process to an objective function of number and poses of cameras, and subject to a set of constraints to obtain the improved camera coverage plan includes:

determining a set of camera coverage maps per camera including one of a first set of occupied voxels representing positions on a shelf in field of view; and

aggregating camera coverage maps to obtain a first aggregate coverage map for the shelf based upon the first set of occupied voxels.

3. The computer-implemented method of claim 2 , wherein applying a machine learning process to an objective function of number and poses of cameras, and subject to a set of constraints to obtain the improved camera coverage plan further includes:

applying a first coverage threshold to the first aggregate coverage map covering a number of cameras having voxels representing positions on a shelf in field of view.

4. The computer-implemented method of claim 3 , wherein applying a first coverage threshold to coverage map covering a number of cameras having voxels representing positions on a shelf in field of view further includes:

applying to coverage maps, a first coverage threshold comprising at least 3 cameras having voxels representing positions on a shelf in field of view.

5. The computer-implemented method of claim 2 , wherein applying a machine learning process to an objective function of number and poses of cameras, and subject to a set of constraints to obtain the improved camera coverage plan includes:

determining a set of camera coverage maps per camera including one of a second set of occupied voxels representing positions of simulated subjects on a plane at some height above a floor of the three-dimensional real space through which simulated subjects would move through; and

aggregating camera coverage maps to obtain a second aggregate coverage map based upon the second set of occupied voxels.

6. The computer-implemented method of claim 5 , wherein applying a machine learning process to an objective function of number and poses of cameras, and subject to a set of constraints to obtain the improved camera coverage plan further includes:

applying a second coverage threshold to the second aggregate coverage map covering a select portions of a plane at some height above a floor of the three-dimensional real space through which simulated subjects would move.

7. The computer-implemented method of claim 6 , wherein applying a second coverage threshold to second aggregate coverage map covering a select portions of a plane at some height above a floor of the three-dimensional real space through which simulated subjects would move further includes:

applying to the second aggregate coverage map, a second coverage threshold comprising a range of 80% or greater of a plane at some height above a floor of the three-dimensional real space through which simulated subjects would move through.

8. The computer-implemented method of claim 5 , wherein applying a machine learning process to an objective function of number and poses of cameras, and subject to a set of constraints to obtain the improved camera coverage plan further includes:

applying to the second aggregate coverage map, a third coverage threshold comprising at least 2 cameras with at least 60 degrees angle of incidence covering select portions of a plane at some height above a floor of the three-dimensional real space through which simulated subjects would move through.

9. The computer-implemented method of claim 1 , wherein the initial number and initial pose of a plurality of cameras and a camera model are selected from (i) a random initialized coverage plan comprising an initial number of cameras randomly distributed in the three-dimensional real space and (ii) a proto-coverage plan comprising a received input of an initial number of cameras approximately positioned in the three-dimensional real space.

10. The computer-implemented method of claim 1 , wherein a camera model for a camera includes a camera intrinsic matrix and distortion values of a lens used on the camera.

11. The computer-implemented method of claim 1 , wherein physical constraints are detected from images of the three-dimensional real space by a machine learning algorithm.

12. The computer-implemented method of claim 11 , wherein physical constraints include a presence of a fixture, a presence of a speaker, a presence of a heating, ventilation, air conditioning (HVAC) vent.

13. The computer-implemented method of claim 1 , wherein the machine learning process includes a mixed integer programming algorithm.

14. The computer-implemented method of claim 1 , wherein the machine learning process includes a gradient descent algorithm.

15. The computer-implemented method of claim 1 , further including providing a plurality of improved coverage plans meeting or exceeding constraints and receiving a selection from the plurality as provided to use in tracking puts and takes.

16. A tool in an automated design environment for configuring an autonomous store, the tool comprising one or more processors coupled to a storage medium storing instructions for determining an improved camera coverage plan including a number, a placement, and a pose of cameras that are arranged to track puts and takes of items by subjects in a three-dimensional real space of the autonomous store, which when executed by the one or more processors implement a method of claim 1 .

17. A computer-implemented method of claim 1 used in a physical store having a volume of real space encompassed by a building, the volume of real space including cameras placed using one or more processors implement the method of claim 1 .

18. A computer-implemented method of claim 1 , wherein the improved coverage score is an improvement over a previous coverage score computed for a previous application of the machine learning process to the objective function.

19. An automated check-out system, comprising:

a classification engine including one or more processors executing stored instructions that when executed by the one or more processors implement tracking of puts and takes by subjects; and

a plurality of cameras sending sequences of images to the classification engine, placed using one or more processors that implement a method including:

beginning with an initial number and initial pose of a plurality of cameras and a camera model including characteristics of the cameras, iteratively applying a machine learning process to an objective function of number and poses of cameras, and subject to a set of constraints, to obtain an improved camera coverage plan using one or more of: (i) a changed number of cameras, and (ii) a changed number of camera poses, and having an improved coverage score and concurrently using a same or reduced number of cameras, wherein the set of constraints includes physical constraints and coverage constraints; and wherein coverage constraints include a number of cameras having a voxel in a structure or display holding inventory within view, a number of cameras having a voxel in a tracking zone of volume in which subjects are tracked within view, a difference in angles of incidence between cameras having a voxel within view, and an overall coverage of a three-dimensional real space; and

providing the improved camera coverage plan to an installer to arrange cameras to track puts and takes of items by subjects in a three-dimensional real space.

20. A non-transitory computer readable storage medium storing instructions for determining an improved camera coverage plan including a number, a placement, and a pose of cameras that are arranged to track puts and takes of items by subjects in a three-dimensional real space, which instructions when executed by one or more processors, implement a method including:

beginning with an initial number and initial pose of a plurality of cameras and a camera model including characteristics of the cameras, iteratively applying a machine learning process to an objective function of number and poses of cameras, and subject to a set of constraints, to obtain an improved camera coverage plan using one or more of: (i) a changed number of cameras, and (ii) a changed number of camera poses, and having an improved coverage score and concurrently using a same or reduced number of cameras, wherein the set of constraints includes physical constraints and coverage constraints; and wherein coverage constraints include a number of cameras having a voxel in a structure or display holding inventory within view, a number of cameras having a voxel in a tracking zone of volume in which subjects are tracked within view, a difference in angles of incidence between cameras having a voxel within view, and an overall coverage of the three-dimensional real space; and

providing the improved camera coverage plan to an installer to arrange cameras to track puts and takes of items by subjects in the three-dimensional real space.

21. The non-transitory computer readable storage medium of claim 20 , further including instructions that, when executed by one or more processors, implement:

determining a set of camera coverage maps per camera including one of a first set of occupied voxels representing positions on a shelf in field of view;

aggregating camera coverage maps to obtain a first aggregate coverage map for the shelf based upon the first set of occupied voxels;

applying a first coverage threshold to the first aggregate coverage map covering a number of cameras having voxels representing positions on a shelf in field of view;

determining a set of camera coverage maps per camera including one of a second set of occupied voxels representing positions of simulated subjects on a plane at some height above a floor of the three-dimensional real space through which simulated subjects would move through;

aggregating camera coverage maps to obtain a second aggregate coverage map based upon the second set of occupied voxels;

applying a second coverage threshold comprising a coverage metric to the second aggregate coverage map covering a select portions of a plane at some height above a floor of the three-dimensional real space through which simulated subjects would move; and

applying a third coverage threshold comprising at least one of number and angle of incidence of cameras to the second aggregate coverage map covering a select portions of a plane at some height above a floor of the three-dimensional real space through which simulated subjects would move.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2023
From: KALLAKURI, NAGASRIKANTH; MALHOTRA, AKSHAY; MORALES SAIKI, LUIS YOICHI; DADLANI, TUSHAR; SINGH, DHANANJAY
To: STANDARD COGNITION, CORP.
Reel/Frame 065357/0386 →
Continuity (4)
Continuation 17681505 · Feb 25, 2022
Continuation 17358864 · Jun 25, 2021
Provisional Application 63044971 · Jun 26, 2020
Related Publication 20240048669A1 · Feb 8, 2024
References Cited (400)
US 6154559A · Beardsley · 2000 [cited by applicant]
US 6561417B1 · Gadd · 2003 [cited by applicant]
US 6768102B1 · Skoll · 2004 [cited by applicant]
US 7050624B2 · Dialameh et al. · 2006 [cited by applicant]
US 7050652B2 · Stanek · 2006 [cited by applicant]
US 7742623B1 · Moon et al. · 2010 [cited by applicant]
US 8009863B1 · Sharma et al. · 2011 [cited by applicant]
US 8219438B1 · Moon et al. · 2012 [cited by applicant]
US 8261256B1 · Adler et al. · 2012 [cited by applicant]
US 8279325B2 · Pitts et al. · 2012 [cited by applicant]
US 8577705B1 · Baboo et al. · 2013 [cited by applicant]
US 8624725B1 · MacGregor · 2014 [cited by applicant]
US 8749630B2 · Alahi et al. · 2014 [cited by applicant]
US 9036028B2 · Buehler · 2015 [cited by applicant]
US 9058523B2 · Merkel et al. · 2015 [cited by applicant]
US 9262681B1 · Mishra · 2016 [cited by applicant]
US 9269012B2 · Fotland · 2016 [cited by applicant]
US 9269093B2 · Lee et al. · 2016 [cited by applicant]
US 9294873B1 · MacGregor · 2016 [cited by applicant]
US 9449233B2 · Taylor · 2016 [cited by applicant]
US 9460524B1 · Curlander et al. · 2016 [cited by applicant]
US 9483704B2 · Schwartz · 2016 [cited by applicant]
US 9489623B1 · Sinyavskiy et al. · 2016 [cited by applicant]
US 9494532B2 · Xie et al. · 2016 [cited by applicant]
US 9536167B2 · Schwartz · 2017 [cited by applicant]
US 9536177B2 · Chalasani et al. · 2017 [cited by applicant]
US 9582891B2 · Geiger et al. · 2017 [cited by applicant]
US 9595127B2 · Champion et al. · 2017 [cited by applicant]
US 9652751B2 · Aaron et al. · 2017 [cited by applicant]
US 9811754B2 · Schwartz · 2017 [cited by applicant]
US 9846810B2 · Partis · 2017 [cited by applicant]
US 9881221B2 · Bala et al. · 2018 [cited by applicant]
US 9886827B2 · Schoner · 2018 [cited by applicant]
US 9911290B1 · Zalewski et al. · 2018 [cited by applicant]
US 10055853B1 · Fisher et al. · 2018 [cited by applicant]
US 10083453B2 · Campbell · 2018 [cited by applicant]
US 10127438B1 · Fisher et al. · 2018 [cited by applicant]
US 10133933B1 · Fisher et al. · 2018 [cited by applicant]
US 10165194B1 · Baldwin · 2018 [cited by applicant]
US 10169677B1 · Ren et al. · 2019 [cited by applicant]
US 10175340B1 · Abari et al. · 2019 [cited by applicant]
US 10176452B2 · Rizzolo et al. · 2019 [cited by applicant]
US 10192408B2 · Schoner · 2019 [cited by applicant]
US 10202135B2 · Mian et al. · 2019 [cited by applicant]
US 10210603B2 · Venable et al. · 2019 [cited by applicant]
US 10210737B2 · Zhao · 2019 [cited by applicant]
US 10217120B1 · Shin et al. · 2019 [cited by applicant]
US 10242393B1 · Kumar et al. · 2019 [cited by applicant]
US 10262331B1 · Sharma et al. · 2019 [cited by applicant]
US 10282720B1 · Buibas et al. · 2019 [cited by applicant]
US 10282852B1 · Buibas et al. · 2019 [cited by applicant]
US 10332089B1 · Asmi et al. · 2019 [cited by applicant]
US 10354262B1 · Hershey et al. · 2019 [cited by applicant]
US 10373322B1 · Buibas et al. · 2019 [cited by applicant]
US 10387896B1 · Hershey et al. · 2019 [cited by applicant]
US 10438277B1 · Jiang et al. · 2019 [cited by applicant]
US 10445694B2 · Fisher et al. · 2019 [cited by applicant]
US 10474877B2 · Huang et al. · 2019 [cited by applicant]
US 10474988B2 · Fisher et al. · 2019 [cited by applicant]
US 10474991B2 · Fisher et al. · 2019 [cited by applicant]
US 10474992B2 · Fisher et al. · 2019 [cited by applicant]
US 10474993B2 · Fisher et al. · 2019 [cited by applicant]
US 10529137B1 · Black et al. · 2020 [cited by applicant]
US 10535146B1 · Buibas et al. · 2020 [cited by applicant]
US 10650545B2 · Fisher et al. · 2020 [cited by applicant]
US 10776926B2 · Shrivastava · 2020 [cited by applicant]
US 10805556B1 · Sorgi et al. · 2020 [cited by applicant]
US 10810539B1 · Mohanty et al. · 2020 [cited by applicant]
US 10853942B1 · Boyd et al. · 2020 [cited by applicant]
US 10853965B2 · Fisher et al. · 2020 [cited by applicant]
US 11132810B2 · Kume et al. · 2021 [cited by applicant]
US 11303853B2 · Kallakuri et al. · 2022 [cited by applicant]
US 11361468B2 · Kallakuri et al. · 2022 [cited by applicant]
US 20030078849A1 · Snyder · 2003 [cited by applicant]
US 20030107649A1 · Flickner et al. · 2003 [cited by applicant]
US 20030154141A1 · Capazario et al. · 2003 [cited by applicant]
US 20040099736A1 · Neumark · 2004 [cited by applicant]
US 20040131254A1 · Liang et al. · 2004 [cited by applicant]
US 20050177446A1 · Hoblit · 2005 [cited by applicant]
US 20050201612A1 · Park et al. · 2005 [cited by applicant]
US 20060132491A1 · Riach et al. · 2006 [cited by applicant]
US 20060279630A1 · Aggarwal et al. · 2006 [cited by applicant]
US 20070021863A1 · Mountz et al. · 2007 [cited by applicant]
US 20070021864A1 · Mountz et al. · 2007 [cited by applicant]
US 20070182718A1 · Schoener et al. · 2007 [cited by applicant]
US 20070282665A1 · Buehler et al. · 2007 [cited by applicant]
US 20080001918A1 · Hsu et al. · 2008 [cited by applicant]
US 20080159634A1 · Sharma et al. · 2008 [cited by applicant]
US 20080170776A1 · Albertson et al. · 2008 [cited by applicant]
US 20080181507A1 · Gope et al. · 2008 [cited by applicant]
US 20080211915A1 · McCubbrey · 2008 [cited by applicant]
US 20080243614A1 · Tu et al. · 2008 [cited by applicant]
US 20090041297A1 · Zhang et al. · 2009 [cited by applicant]
US 20090057068A1 · Lin et al. · 2009 [cited by applicant]
US 20090083815A1 · McMaster et al. · 2009 [cited by applicant]
US 20090217315A1 · Malik et al. · 2009 [cited by applicant]
US 20090222313A1 · Kannan et al. · 2009 [cited by applicant]
US 20090307226A1 · Koster et al. · 2009 [cited by applicant]
US 20100021009A1 · Yao · 2010 [cited by applicant]
US 20100103104A1 · Son et al. · 2010 [cited by applicant]
US 20100208941A1 · Broaddus et al. · 2010 [cited by applicant]
US 20100283860A1 · Nader · 2010 [cited by applicant]
US 20110141011A1 · Lashina et al. · 2011 [cited by applicant]
US 20110209042A1 · Porter · 2011 [cited by applicant]
US 20110228976A1 · Fitzgibbon et al. · 2011 [cited by applicant]
US 20110317012A1 · Hammadou · 2011 [cited by applicant]
US 20110317016A1 · Saeki et al. · 2011 [cited by applicant]
US 20110320322A1 · Roslak et al. · 2011 [cited by applicant]
US 20120119879A1 · Estes et al. · 2012 [cited by applicant]
US 20120154604A1 · Chen et al. · 2012 [cited by applicant]
US 20120159290A1 · Pulsipher et al. · 2012 [cited by applicant]
US 20120209749A1 · Hammad et al. · 2012 [cited by applicant]
US 20120245974A1 · Bonner et al. · 2012 [cited by applicant]
US 20120271712A1 · Katzin et al. · 2012 [cited by applicant]
US 20120275686A1 · Wilson et al. · 2012 [cited by applicant]
US 20120290401A1 · Neven · 2012 [cited by applicant]
US 20130011007A1 · Muriello et al. · 2013 [cited by applicant]
US 20130011049A1 · Kimura · 2013 [cited by applicant]
US 20130076898A1 · Philippe et al. · 2013 [cited by applicant]
US 20130103608A1 · Scipioni et al. · 2013 [cited by applicant]
US 20130156260A1 · Craig · 2013 [cited by applicant]
US 20130169822A1 · Zhu et al. · 2013 [cited by applicant]
US 20130182114A1 · Zhang et al. · 2013 [cited by applicant]
US 20130201339A1 · Venkatesh · 2013 [cited by applicant]
US 20130325673A1 · Abdic et al. · 2013 [cited by applicant]
US 20140168477A1 · David · 2014 [cited by applicant]
US 20140172476A1 · Goulart · 2014 [cited by applicant]
US 20140188648A1 · Argue et al. · 2014 [cited by applicant]
US 20140207615A1 · Li et al. · 2014 [cited by applicant]
US 20140222501A1 · Hirakawa et al. · 2014 [cited by applicant]
US 20140282162A1 · Fein et al. · 2014 [cited by applicant]
US 20140285660A1 · Jamtgaard et al. · 2014 [cited by applicant]
US 20140289009A1 · Campbell · 2014 [cited by applicant]
US 20140300736A1 · Reitinger et al. · 2014 [cited by applicant]
US 20140304123A1 · Schwartz · 2014 [cited by applicant]
US 20150002675A1 · Kundu et al. · 2015 [cited by applicant]
US 20150009323A1 · Lei · 2015 [cited by applicant]
US 20150012396A1 · Puerini et al. · 2015 [cited by applicant]
US 20150019391A1 · Kumar et al. · 2015 [cited by applicant]
US 20150026010A1 · Ellison · 2015 [cited by applicant]
US 20150026646A1 · Ahn et al. · 2015 [cited by applicant]
US 20150039458A1 · Reid · 2015 [cited by applicant]
US 20150049914A1 · Alves · 2015 [cited by applicant]
US 20150124107A1 · Muriello et al. · 2015 [cited by applicant]
US 20150193761A1 · Svetal · 2015 [cited by applicant]
US 20150206188A1 · Tanigawa et al. · 2015 [cited by applicant]
US 20150208043A1 · Lee et al. · 2015 [cited by applicant]
US 20150213391A1 · Hasan · 2015 [cited by applicant]
US 20150221094A1 · Marcheselli et al. · 2015 [cited by applicant]
US 20150262116A1 · Katircioglu et al. · 2015 [cited by applicant]
US 20150269740A1 · Mazurenko et al. · 2015 [cited by applicant]
US 20150294397A1 · Croy et al. · 2015 [cited by applicant]
US 20150302593A1 · Mazurenko et al. · 2015 [cited by applicant]
US 20150310459A1 · Bernal et al. · 2015 [cited by applicant]
US 20150327794A1 · Rahman et al. · 2015 [cited by applicant]
US 20150332312A1 · Cosman · 2015 [cited by applicant]
US 20150341599A1 · Carey · 2015 [cited by applicant]
US 20150356610A1 · Ponoth · 2015 [cited by applicant]
US 20150363868A1 · Kleinhandler et al. · 2015 [cited by applicant]
US 20150379366A1 · Nomura et al. · 2015 [cited by applicant]
US 20160095511A1 · Taguchi et al. · 2016 [cited by applicant]
US 20160110760A1 · Herring et al. · 2016 [cited by applicant]
US 20160125245A1 · Saitwal et al. · 2016 [cited by applicant]
US 20160155011A1 · Sulc et al. · 2016 [cited by applicant]
US 20160171707A1 · Schwartz · 2016 [cited by applicant]
US 20160188962A1 · Taguchi · 2016 [cited by applicant]
US 20160189286A1 · Zohar et al. · 2016 [cited by applicant]
US 20160203525A1 · Hara et al. · 2016 [cited by applicant]
US 20160217157A1 · Shih et al. · 2016 [cited by applicant]
US 20160217417A1 · Ma et al. · 2016 [cited by applicant]
US 20160259994A1 · Ravindran et al. · 2016 [cited by applicant]
US 20160358145A1 · Montgomery · 2016 [cited by applicant]
US 20160371726A1 · Yamaji et al. · 2016 [cited by applicant]
US 20160381328A1 · Zhao · 2016 [cited by applicant]
US 20170011260A1 · Mihara et al. · 2017 [cited by applicant]
US 20170024806A1 · High et al. · 2017 [cited by applicant]
US 20170032193A1 · Yang · 2017 [cited by applicant]
US 20170068861A1 · Miller et al. · 2017 [cited by applicant]
US 20170104979A1 · Shaw et al. · 2017 [cited by applicant]
US 20170116473A1 · Sashida et al. · 2017 [cited by applicant]
US 20170124096A1 · Hsi et al. · 2017 [cited by applicant]
US 20170148005A1 · Murn · 2017 [cited by applicant]
US 20170154212A1 · Feris et al. · 2017 [cited by applicant]
US 20170161555A1 · Kumar et al. · 2017 [cited by applicant]
US 20170168586A1 · Sinha et al. · 2017 [cited by applicant]
US 20170178226A1 · Graham et al. · 2017 [cited by applicant]
US 20170193434A1 · Shah et al. · 2017 [cited by applicant]
US 20170205337A1 · Akhtman et al. · 2017 [cited by applicant]
US 20170206664A1 · Shen · 2017 [cited by applicant]
US 20170206669A1 · Saleemi et al. · 2017 [cited by applicant]
US 20170249339A1 · Lester · 2017 [cited by applicant]
US 20170255990A1 · Ramamurthy et al. · 2017 [cited by applicant]
US 20170278255A1 · Shingu et al. · 2017 [cited by applicant]
US 20170308911A1 · Barham et al. · 2017 [cited by applicant]
US 20170309136A1 · Schoner · 2017 [cited by applicant]
US 20170323376A1 · Glaser et al. · 2017 [cited by applicant]
US 20180003315A1 · Reed · 2018 [cited by applicant]
US 20180012072A1 · Glaser et al. · 2018 [cited by applicant]
US 20180012080A1 · Glaser et al. · 2018 [cited by applicant]
US 20180014382A1 · Glaser et al. · 2018 [cited by applicant]
US 20180025175A1 · Kato · 2018 [cited by applicant]
US 20180032799A1 · Marcheselli et al. · 2018 [cited by applicant]
US 20180033015A1 · Opalka et al. · 2018 [cited by applicant]
US 20180033151A1 · Matsumoto et al. · 2018 [cited by applicant]
US 20180068431A1 · Takeda et al. · 2018 [cited by applicant]
US 20180070056A1 · DeAngelis et al. · 2018 [cited by applicant]
US 20180088900A1 · Glaser et al. · 2018 [cited by applicant]
US 20180108195A1 · Bashkin · 2018 [cited by applicant]
US 20180150788A1 · Vepakomma et al. · 2018 [cited by applicant]
US 20180165728A1 · McDonald et al. · 2018 [cited by applicant]
US 20180181995A1 · Burry et al. · 2018 [cited by applicant]
US 20180189600A1 · Astrom et al. · 2018 [cited by applicant]
US 20180217223A1 · Kumar et al. · 2018 [cited by applicant]
US 20180225625A1 · DiFatta et al. · 2018 [cited by applicant]
US 20180232796A1 · Glaser et al. · 2018 [cited by applicant]
US 20180240180A1 · Glaser et al. · 2018 [cited by applicant]
US 20180295424A1 · Taylor et al. · 2018 [cited by applicant]
US 20180322616A1 · Guigues · 2018 [cited by applicant]
US 20180329762A1 · Li et al. · 2018 [cited by applicant]
US 20180332235A1 · Glaser · 2018 [cited by applicant]
US 20180332236A1 · Glaser et al. · 2018 [cited by applicant]
US 20180343417A1 · Davey · 2018 [cited by applicant]
US 20180365481A1 · Tolbert et al. · 2018 [cited by applicant]
US 20180365755A1 · Bekbolatov et al. · 2018 [cited by applicant]
US 20180373928A1 · Glaser et al. · 2018 [cited by applicant]
US 20190005479A1 · Glaser et al. · 2019 [cited by applicant]
US 20190019309A1 · Herrli et al. · 2019 [cited by applicant]
US 20190034735A1 · Cuban et al. · 2019 [cited by applicant]
US 20190043003A1 · Fisher et al. · 2019 [cited by applicant]
US 20190057435A1 · Chomley et al. · 2019 [cited by applicant]
US 20190147709A1 · Schoner · 2019 [cited by applicant]
US 20190156273A1 · Fisher et al. · 2019 [cited by applicant]
US 20190156274A1 · Fisher et al. · 2019 [cited by applicant]
US 20190156275A1 · Fisher et al. · 2019 [cited by applicant]
US 20190156276A1 · Fisher et al. · 2019 [cited by applicant]
US 20190156277A1 · Fisher et al. · 2019 [cited by applicant]
US 20190156506A1 · Fisher et al. · 2019 [cited by applicant]
US 20190158813A1 · Rowell et al. · 2019 [cited by applicant]
US 20190188876A1 · Song et al. · 2019 [cited by applicant]
US 20190244386A1 · Fisher et al. · 2019 [cited by applicant]
US 20190244500A1 · Fisher et al. · 2019 [cited by applicant]
US 20190251340A1 · Brown et al. · 2019 [cited by applicant]
US 20190311625A1 · Anvari · 2019 [cited by applicant]
US 20190347611A1 · Fisher et al. · 2019 [cited by applicant]
US 20190377957A1 · Johnston et al. · 2019 [cited by applicant]
US 20190378205A1 · Glaser et al. · 2019 [cited by applicant]
US 20190392318A1 · Ghafoor et al. · 2019 [cited by applicant]
US 20200019921A1 · Buibas et al. · 2020 [cited by applicant]
US 20200020113A1 · Buibas et al. · 2020 [cited by applicant]
US 20200025931A1 · Liang et al. · 2020 [cited by applicant]
US 20200034988A1 · Zhou · 2020 [cited by applicant]
US 20200074165A1 · Ghafoor et al. · 2020 [cited by applicant]
US 20200074393A1 · Fisher et al. · 2020 [cited by applicant]
US 20200074394A1 · Fisher et al. · 2020 [cited by applicant]
US 20200074432A1 · Valdman et al. · 2020 [cited by applicant]
US 20200118400A1 · Zalewski et al. · 2020 [cited by applicant]
US 20200125824A1 · Mabyalaht et al. · 2020 [cited by applicant]
US 20200134588A1 · Nelms et al. · 2020 [cited by applicant]
US 20200151692A1 · Gao et al. · 2020 [cited by applicant]
US 20200184230A1 · Liu et al. · 2020 [cited by applicant]
US 20200184442A1 · Gu et al. · 2020 [cited by applicant]
US 20200184444A1 · Gu et al. · 2020 [cited by applicant]
US 20200184447A1 · Gu et al. · 2020 [cited by applicant]
US 20200193507A1 · Glaser et al. · 2020 [cited by applicant]
US 20200202177A1 · Buibas et al. · 2020 [cited by applicant]
US 20200234463A1 · Fisher et al. · 2020 [cited by applicant]
US 20200258241A1 · Liu et al. · 2020 [cited by applicant]
US 20200293992A1 · Bogolea et al. · 2020 [cited by applicant]
US 20200334834A1 · Fisher · 2020 [cited by applicant]
US 20200334835A1 · Buibas et al. · 2020 [cited by applicant]
US 20200380701A1 · Buibas et al. · 2020 [cited by applicant]
US 20200410713A1 · Auer et al. · 2020 [cited by applicant]
US 20210049772A1 · Buibas et al. · 2021 [cited by applicant]
US 20210067744A1 · Buibas et al. · 2021 [cited by applicant]
US 20210125355A1 · Mirza et al. · 2021 [cited by applicant]
US 20210158430A1 · Buibas et al. · 2021 [cited by applicant]
US 20210182922A1 · Zheng et al. · 2021 [cited by applicant]
US 20210201253A1 · Fisher et al. · 2021 [cited by applicant]
US 20210272086A1 · Buibas et al. · 2021 [cited by applicant]
US 20210279475A1 · Tusch et al. · 2021 [cited by applicant]
US 20210295081A1 · Berry et al. · 2021 [cited by applicant]
US 20210407131A1 · Kallakuri et al. · 2021 [cited by applicant]
US 20220262069A1 · Singh et al. · 2022 [cited by applicant]
US 20220270295A1 · Kallakuri et al. · 2022 [cited by applicant]
US 20230162398A1 · Varekamp et al. · 2023 [cited by applicant]
CN 104850846A · 2015 [cited by applicant]
CN 105069413A · 2015 [cited by applicant]
CN 105701519A · 2016 [cited by applicant]
CN 104778690B · 2017 [cited by applicant]
EP 1574986B1 · 2008 [cited by applicant]
EP 2555162A1 · 2013 [cited by applicant]
EP 3002710A1 · 2016 [cited by applicant]
GB 2560387A · 2018 [cited by applicant]
GB 2566762A · 2019 [cited by applicant]
JP 2011253344A · 2011 [cited by applicant]
JP 2013196199A · 2013 [cited by applicant]
JP 2014089626A · 2014 [cited by applicant]
JP 2016206782A · 2016 [cited by applicant]
JP 2017157216A · 2017 [cited by applicant]
JP 2018099317A · 2018 [cited by applicant]
KR 1020180032400A · 2018 [cited by applicant]
KR 1020190093733A · 2019 [cited by applicant]
KR 102223570B1 · 2021 [cited by applicant]
TW 201228380A · 2012 [cited by applicant]
TW 201911119A · 2019 [cited by applicant]
WO 0021021A1 · 2000 [cited by applicant]
WO 0243352A2 · 2002 [cited by applicant]
WO 02059836A3 · 2003 [cited by applicant]
WO 2008029159A1 · 2008 [cited by applicant]
WO 2013041444A1 · 2013 [cited by applicant]
WO 2013043590A1 · 2013 [cited by applicant]
WO 2013103912A1 · 2013 [cited by applicant]
WO 2014133779A1 · 2014 [cited by applicant]
WO 2015133699A1 · 2015 [cited by applicant]
WO 2016136144A1 · 2016 [cited by applicant]
WO 2016166508A1 · 2016 [cited by applicant]
WO 2017015390A1 · 2017 [cited by applicant]
WO 2017151241A2 · 2017 [cited by applicant]
WO 2017196822A1 · 2017 [cited by applicant]
WO 2018013438A1 · 2018 [cited by applicant]
WO 2018013439A1 · 2018 [cited by applicant]
WO 2018148613A1 · 2018 [cited by applicant]
WO 2018162929A1 · 2018 [cited by applicant]
WO 2018209156A1 · 2018 [cited by applicant]
WO 2018237210A1 · 2018 [cited by applicant]
WO 2019032304A1 · 2019 [cited by applicant]
WO 2019032305A2 · 2019 [cited by applicant]
WO 2019032306A1 · 2019 [cited by applicant]
WO 2019032307A1 · 2019 [cited by applicant]
WO 2020023795 · 2020 [cited by applicant]
WO 2020023796 · 2020 [cited by applicant]
WO 2020023798 · 2020 [cited by applicant]
WO 2020023799 · 2020 [cited by applicant]
WO 2020023801 · 2020 [cited by applicant]
WO 2020023926 · 2020 [cited by applicant]
WO 2020023930 · 2020 [cited by applicant]
WO 2020047555A1 · 2020 [cited by applicant]
WO 2020214775A1 · 2020 [cited by applicant]
WO 2020225562A1 · 2020 [cited by applicant]
Baek, “Aging and Soiling Simulators for Improved Banknote Classification and Validation”, Graduate School, Yonsei University Department of Electrical and Electronic Engineering, pp. 1-9 Aug. 4, 2018 [retrieved on Sep. 6… [cited by applicant]
Camplani et al., “Background foreground segmentation with RGB-D Kinect data: An efficient combination of classifiers”, Journal of Visual Communication and Image Representation, Academic Press, Inc., US, vol. 25, No. 1, … [cited by applicant]
Ceballos, Scikit-Learn Decision Trees Explained, https://towardsdatascience.com/scikit-learn-decision-trees-explained-803f3812290d, Feb. 22, 2019, 13 pages. [cited by applicant]
Feng et al., “2D3D-MatchNet: Learning to Match Keypoints Across 2D Image and 3D Point Cloud,” 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, May 20, 2019, pp. 4790-4796. [cited by applicant]
Grinciunaite et al. “Human Pose Estimation in Space and Time Using 3D CNN,” ECCV Workshop on Brave new ideas for motion representations in videos, Oct. 2016, 7 pages. [cited by applicant]
Harville, “Stereo person tracking with adaptive plan-view templates of height and occupancy statistics,” Image and Vision Computing, vol. 22, Issue 2, Feb. 1, 2004, pp. 127-142. [cited by applicant]
He et al. “Identity mappings in deep residual networks” (published at https://arxiv.org/pdf/1603.05027.pdf), Jul. 25, 2016, 15 pages. [cited by applicant]
Huang, et al. “Driver's view and vehicle surround estimation using omnidirectional video stream,” IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No. 03TH8683), Jun. 9-11, 2003, pp. 444-449. [cited by applicant]
Jayabalan, et al., “Dynamic Action Recognition: A convolutional neural network model for temporally organized joint location data,” Cornell University, Computer Science, Dec. 20, 2016, 11 pages. [cited by applicant]
Longuet-Higgens, “A computer algorithm for reconstructing a scene from two projections,” Nature 293, Sep. 10, 1981, pp. 133-135. [cited by applicant]
PCT/US2021/039225—International Search Report and Written Opinion dated Oct. 19, 2021, 9 pages. [cited by applicant]
Redmon et al., “You Only Look Once: Unified, Real-Time Object Detection,” University of Washington, Allen Institute for Aly, Facebook AI Research, May 9, 2016, 10 pages. [cited by applicant]
Redmon et al., YOL09000: Better, Faster, Stronger, (available at https://arxiv.org/pdf/1612.08242.pdf), Dec. 25, 2016, 9 pages. [cited by applicant]
Rossi et al., “Tracking and Counting Moving People,” IEEE Int'l Conf. on Image Processing, ICIP-94, Nov. 13-16, 1994 5 pages. [cited by applicant]
Toshev et al. “DeepPose: Human Pose Estimation via Deep Neural Networks,” IEEE Conf. on Computer Vision and Pattern Recognition, Aug. 2014, 8 pages. [cited by applicant]
Vincze, “Robust tracking of ellipses at frame rate,” Pattern Recognition, vol. 34, Issue 2, Feb. 2001, pp. 487-498. [cited by applicant]
Yuanzhouhan, Cao et al, “Leveraging Convolutional Pose Machines for Fast and Accurate Head Pose Estimation”, 2018 IEEE/RSJ International Conference On Intelligent Robots and Systems (IROS), IEEE, Oct. 1, 2018 (Oct. 1, 2… [cited by applicant]
Zhang “A Flexible New Technique for Camera Calibration,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, No. 11, Nov. 2000, 22pages. [cited by applicant]
PCT/US2021/039225—International Preliminary Report on Patentability dated Jan. 5, 2023, 7 pages. [cited by applicant]
Yusoff et al. “Optimal Camera Placement for 3D Environment,” ICSECS 2011: Software Engineering and Computer Systems, Jun. 27-29, 2011, 448-459. [cited by applicant]
Erdem et al. “Automated camera layout to satisfy task-specific and floor plan-specific coverage requirements,” Computer Vision and Image Undertanding 103, Aug. 1, 2006, 156-169. [cited by applicant]
PCT/US2018/043933—International Search Report and Written Opinion dated Nov. 7, 2018, 12 pages. [cited by applicant]
PCT/US2018/043937—International Search Report and Written Opinion dated Oct. 31, 2018, 17 pages. [cited by applicant]
PCT/US2018/043937—International Preliminary Report on Patentability dated Feb. 20, 2020, 14 pages. [cited by applicant]
PCT/US2018/043939—International Search Report and Written Opinion dated Jan. 2, 2019, 17 pages. [cited by applicant]
EP 18843163.9—Extended European Search Report dated Nov. 27, 2020, 8 pages. [cited by applicant]
EP 18843486.4—Extended European Search Report dated Dec. 8, 2020, 8 pages. [cited by applicant]
EP 18844509.2—Extended European Search Report dated Dec. 8, 2020, 8 pages. [cited by applicant]
EP-18844384.0—European Extended Search Report dated Dec. 2, 2020, 7 pages. [cited by applicant]
PCT/US2018/043939—International Preliminary Report on Patentability dated Feb. 20, 2020, 14 pages. [cited by applicant]
PCT/US2019-043520—International Search Report and Written Opinion dated May 8, 2020, 10 pages. [cited by applicant]
PCT/US2019/043519—International Preliminary Report on Patentability dated Feb. 4, 2021, 7 pages. [cited by applicant]
PCT/US2019/043519—International Search Report and Written Opinion dated Oct. 31, 2019, 10 pages. [cited by applicant]
PCT/US2019/043522—International Search Report and Written Opinion dated Nov. 15, 2019, 11 pages. [cited by applicant]
PCT/US2019/043523—International Preliminary Report on Patentability dated Feb. 4, 2021, 15 pages. [cited by applicant]
PCT/US2019/043523—International Search Report and Written Opinion dated Nov. 20, 2019, 18 pages. [cited by applicant]
PCT/US2019/043526—International Search Report and Written Opinion dated Nov. 18, 2019, 9 pages. [cited by applicant]
PCT/US2019/043770—International Search Report and Written Opinion dated Nov. 13, 2019, 15 pages. [cited by applicant]
PCT/US2019/043775—International Search Report and Written Opinion dated Nov. 13, 2019, 10 pages. [cited by applicant]
PCT/US2019/049388—International Search Report and Wirtten Opinion dated Dec. 20, 2019, 11 pages. [cited by applicant]
PCT/US2020/028454—International Search Report and Written Opinion dated Jul. 27, 2020, 12 pages. [cited by applicant]
PCT/US2018/043934—International Search Report and Written Opinion dated Feb. 1, 2019, 13 pages. [cited by applicant]
PCT/US2019/049388—International Preliminary Report on Patentability dated Mar. 11, 2021, 7 pages. [cited by applicant]
PCT/US2019-043520—International Preliminary Report and Written Opinion dated Feb. 4, 2021, 7 pages. [cited by applicant]
PCT/US2019/043526—International Preliminary Report on Patentability dated Feb. 4, 2021, 6 pages. [cited by applicant]
PCT/US2018/043934—International Preliminary Report on Patentability dated Feb. 20, 2020, 12 pages. [cited by applicant]
PCT/US2019/043770—International Search Report and Written Opinion dated Nov. 13, 2019, 13 pages. [cited by applicant]
PCT/US2019/043770—International Preliminary Report on Patentability dated Feb. 4, 2021, 9 pages. [cited by applicant]
Gkioxari et al., “R-CNN's for Pose Estimation and Action Detection”, <https://arxiv.org/pdf/1406.5212.pdf>, Jun. 19, 2014. [cited by applicant]
GB 1703914.0—Office Action dated Jul. 14, 2021, 5 pages. [cited by applicant]
GB 1703914.0—Search Report dated Jan. 18, 2018, 1 pages. [cited by applicant]
GB 1703914.0—Intention to Grant dated Nov. 29, 2021, 2 pages. [cited by applicant]
PCT/US2019/043775—International Preliminary Report on Patentability dated Feb. 4, 2021, 7 pages. [cited by applicant]
PCT/US2021/031173—International Search Report and Written Opinion dated Aug. 20, 2021, 9 pages. [cited by applicant]
TW 107126341—Office Action dated Dec. 14, 2021, 2 pages. [cited by applicant]
TW 107126341—Search Report dated Dec. 14, 2021, 1 pages. [cited by applicant]
TW 107126341—Partial English Translation of Office Action and Search Report dated Dec. 14, 2021, 5 pages. [cited by applicant]
Cao et al, “Leveraging Convolutional Pose Machines for Fast and Accurate Head Pose Estimation”, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 1, 2018. pp. 1089-1094. [cited by applicant]
Wei et al., “Convolutional Pose Machine”, Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Jun. 27, 2016, pp. 4724-4732. [cited by applicant]
EP 18709743.1—Office Action dated Mar. 11, 2021, 5 pages. [cited by applicant]
PCT/GB2020/051120—International Search Report and Written Opinion dated Jul. 27, 2020, 12 pages. [cited by applicant]