IP Library Granted Patent US 12,330,311
Granted Patent B2
US 12,330,311 · App. 17/885,617 · Granted Jun 17, 2025

Intelligent monitoring of entry points in multi-cell workspaces

Inventors: Clara Vu (Cambridge, MA); Scott Denenberg (Newton, MA); Marek Wartenberg (Chelsea, MA); Ilya Kriveshko (Boxborough, MA); Antonio Fiol-Mahon (Cambridge, MA); Alberto Moel (Cambridge, MA)
Assignee: Symbotic LLC
B25J9/1666B25J9/1676B25J9/1697G01S7/4808G01S17/04G01S17/87G01S17/89G06T17/10G01V8/20G05B2219/40202G06T17/05Y10S901/47Y10S901/49
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,330,311
App. No.
17/885,617
Granted
Jun 17, 2025
Kind
B2
Abstract

One or more computational strategies is used to reduce the complexity of handling detected point clusters appearing at a portal of a monitored workcell. If the other side of the portal is a second, adjacent workcell, the monitoring system for a first workcell predicts when a human in that workcell may pass through the portal to a second workcell and alerts the monitoring system for the second workcell. The prediction may be based on absolute proximity of a human to the portal or movement toward it. Other strategies are employed if the workcells overlap, or if the portal leads to an unmonitored space.

Claims (38)

1. A method of monitoring a portal between adjacent first and second three-dimensional workcells each of which includes controlled machinery, the portal being traversable by humans from either of the workcells into the other workcell, at least the second workcell including a monitoring system comprising a plurality of cameras distributed thereabout, the method comprising the steps of:

computationally predicting that a human will cross the portal from the first workcell into the second workcell; and

electronically signaling the monitoring system of the second workcell of expected entry of the identified first human into the second workcell,

wherein the monitoring system of the second workcell does not analyze detected features at the portal to determine whether they are associated with a human except following receipt of the signal indicating expected entry of a human into the second workcell through the portal.

2. The method of claim 1 , further comprising the steps of:

computationally identifying, by the monitoring system of the second workcell, a second human in the second workcell based on images of the second workcell recorded by at least one of the cameras therein;

computationally predicting, based at least in part on the recorded images, that the identified second human will cross the portal into the first workcell; and

electronically signaling, by the monitoring system of the second workcell, a monitoring system of the first workcell of expected entry of the identified second human into the first workcell,

wherein neither of the monitoring systems analyzes detected features at the portal to determine whether they are associated with a human except following receipt of a signal from the other monitoring system indicating expected entry of a human through the portal.

3. The method of claim 1 , wherein predicting that an identified human will cross the portal into an adjacent workcell is based on proximity of the identified human to the portal.

4. The method of claim 1 , wherein predicting that an identified human will cross the portal into an adjacent workcell is based on computationally projected motion of the identified human.

5. The method of claim 1 , wherein the first and second workcells do not overlap.

6. The method of claim 1 , wherein the second three-dimensional workcell is a monitored workcell and the first three-dimensional workcell is an unmonitored workcell, the method further comprising the steps of:

computationally identifying, by the monitoring system of the monitored workcell, a human in the monitored workcell based on images recorded by at least one of the cameras therein;

computationally predicting, based at least in part on the recorded images, that the identified human will cross the portal into the unmonitored workcell; and

based on the computational prediction, electronically signaling a control system of the machinery to put the machinery into a safe state.

7. The method of claim 6 , wherein putting the machinery into a safe state comprises shutting down the machinery.

8. The method of claim 6 , wherein predicting that the identified human will cross the portal is based on proximity of the identified human to the portal.

9. The method of claim 6 , wherein predicting that an identified human will cross the portal is based on computationally projected motion of the identified human.

10. A method of monitoring adjacent first and second three-dimensional workcells each of which includes controlled machinery, the first and second workcells overlapping along a shared region, at least the second workcell including a monitoring system comprising plurality of cameras distributed thereabout, the method comprising the steps of:

computationally predicting that a human will cross a first portal in the shared region into the second workcell; and

electronically signaling the monitoring system of the second workcell of expected crossing of the first portal by the identified first human,

wherein the second workcell does not analyze detected features in the shared region to determine whether they are associated with a human except following receipt of the signal indicating expected entry of a human into the second workcell through the portal.

11. The method of claim 10 , further comprising the steps of:

computationally identifying, by the monitoring system of the second workcell, a second human in the second workcell based on images of the second workcell recorded by at least one of the cameras therein;

computationally predicting, based at least in part on the recorded images, that the identified second human will cross a second portal in the shared region; and

electronically signaling, by the monitoring system of the second workcell, a monitoring system of the first workcell of expected crossing of the second portal by the identified second human,

wherein neither of the monitoring systems analyzes detected features in the shared region to determine whether they are associated with a human except following receipt of a signal from the other monitoring system indicating expected entry of a human through a portal.

12. The method of claim 11 , wherein the first and second portals are the same portal located within the shared region.

13. The method of claim 11 , wherein the first and second portals are the same portal located midway within the shared region.

14. The method of claim 11 , wherein the first portal is a boundary of the second workcell and the second portal is a boundary of the first workcell.

15. The method of claim 11 , wherein predicting that an identified human will cross a portal is based on proximity of the identified human to the portal.

16. The method of claim 11 , wherein predicting that an identified human will cross a portal is based on computationally projected motion of the identified human.

17. The method of claim 11 , further comprising the step of preventing crosstalk between cameras of the first and second workcells by causing otherwise interfering cameras to operate simultaneously in accordance with a noninterference scheme.

18. The method of claim 17 , wherein the noninterference scheme comprises time-division multiplexing at least some interfering light sources.

19. The method of claim 17 , wherein the noninterference scheme comprises wavelength-division multiplexing at least some interfering light sources.

20. The method of claim 17 , wherein the cameras have light sources that emit radiation having a modulation frequency and the noninterference scheme comprises multiplexing the modulation frequencies of at least some interfering camera light sources.

21. The method of claim 17 , wherein the noninterference scheme comprises a background interference map and the step of causing the cameras of the first and second workcells to operate simultaneously in accordance with the noninterference scheme comprises subtracting background illumination specified in the map.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 4, 2024
From: VEO ROBOTICS, INC.
To: SYMBOTIC LLC
Reel/Frame 068839/0710 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 29, 2022
From: VU, CLARA; DENENBERG, SCOTT; WARTENBERG, MAREK; KRIVESHKO, ILYA A; FIOL-MAHON, ANTONIO
To: VEO ROBOTICS, INC.
Reel/Frame 060928/0797 →
Continuity (3)
Continuation In Part 17375447 · Jul 14, 2021
Continuation 16800427 · Feb 25, 2020
Related Publication 20220379474A1 · Dec 1, 2022
References Cited (30)
US 5574637A · Obata et al. · 1996 [cited by applicant]
US 9452531B2 · Kikkeri et al. · 2016 [cited by applicant]
US 10099372B2 · Vu et al. · 2018 [cited by applicant]
US 10445944B2 · Galera et al. · 2019 [cited by applicant]
US 10812778B1 · Wang et al. · 2020 [cited by applicant]
US 10899007B2 · Vu et al. · 2021 [cited by applicant]
US 11097422B2 · Denenberg et al. · 2021 [cited by applicant]
US 11613017B2 · Denenberg et al. · 2023 [cited by applicant]
US 20050224479A1 · Watanabe · 2005 [cited by applicant]
US 20160354927A1 · Kikkeri et al. · 2016 [cited by applicant]
US 20170302905A1 · Shteinfeld et al. · 2017 [cited by applicant]
US 20180151012A1 · Kuraoka · 2018 [cited by applicant]
US 20180222052A1 · Vu et al. · 2018 [cited by applicant]
US 20180324393A1 · Ryan · 2018 [cited by examiner]
US 20200206928A1 · Denenberg et al. · 2020 [cited by applicant]
US 20210252706A1 · Zanchettin · 2021 [cited by examiner]
US 20220019949A1 · Shobu · 2022 [cited by examiner]
US 20220379474A1 · Vu · 2022 [cited by examiner]
US 20240314480A1 · Lee · 2024 [cited by examiner]
JP 03266101A · 1991 [cited by applicant]
JP 06309011A · 1994 [cited by applicant]
JP 11110041A · 1999 [cited by applicant]
JP H11110041A · 1999 [cited by applicant]
JP 2003029832A · 2003 [cited by applicant]
JP 2006501487A · 2006 [cited by applicant]
JP 2018185727A · 2018 [cited by applicant]
JP 2018207349A · 2018 [cited by applicant]
International Search Report and the Written Opinion of the International Searching Authority, for International Application No. PCT/US2020/019658, dated May 20, 2020, 12 pages. [cited by applicant]
Japanese Office Action, Japanese Application No. 2021-549787, dated Oct. 4, 2022, 5 pages. [cited by applicant]
Notice of Reasons for Rejection issued in corresponding Japanese Application No. 2021-549787, 5 pages, with English translation. [cited by applicant]