Granted Patent B2

US 12,643,237 · App. 18/740,522 · Granted Jun 2, 2026

Inventors: Clara Vu (Cambridge, MA); Scott Denenberg (Newton, MA); Marek Wartenberg (Westbrook, ME); Paul Jakob Schroeder (Chapel Hill, NC); Ilya A. Kriveshko (Boxborough, MA); Alberto Moel (Cambridge, MA)

Assignee: Symbotic LLC

B25J9/1694B25J9/1666B25J9/1676B25J9/1697G01S7/4808G01S17/04G01S17/87G01S17/89G06T17/10G01V8/20G05B2219/40202G06T17/05Y10S901/47Y10S901/49

Abstract

Claims (70)

1 . A system for monitoring a workspace, the system comprising:

sensors configured to capture images of a workspace;

memory configured to store (i) captured images from the sensors and (ii) a model of machinery located within the workspace, wherein the model comprises movement capabilities of the machinery; and

a processor configured to:

generate a spatial representation of the workspace based on the captured images;

identify first and second non-machinery entities within the workspace;

generate a first object that specifies attributes of the first non-machinery entity comprising (i) a spatial extent of the first non-machinery entity and (ii) a protective separation distance for the first non-machinery entity;

generate a second object that specifies attributes of the second non-machinery entity comprising (i) a spatial extent of the second non-machinery entity and (ii) a protective separation distance for the second non-machinery entity;

recognize an interaction between the first and second non-machinery entities in the workspace;

responsive to recognizing the interaction, merge the first and second objects into a merged object that specifies a protective separation distance for the merged object; and

control operation of the machinery in accordance with (i) a safety protocol and (ii) the protective separation distance for the merged object.

2 . The system of claim 1 , wherein the processor is further configured to:

recognize dissociation between the first and second non-machinery entities; and

generate and store separate objects corresponding to the first and second non-machinery entities and specifying at least some different attributes thereof.

3 . The system of claim 1 , wherein:

the first object further specifies a first associated computer-executable method; and

the second object further specifies a second associated computer-executable method.

4 . The system of claim 3 , wherein:

the first associated computer-executable method comprises a first method for generating a hierarchically superior object that comprises the first object; or

the second associated computer-executable method comprises a second method for generating a hierarchically superior object that comprises the second object.

5 . The system of claim 1 , wherein at least one of the first and second non- machinery entities is a non-human entity.

6 . The system of claim 1 , wherein:

the protective separation distance for the first non-machinery entity is different from the protective separation distance for the second non-machinery entity; and

the protective separation distance for the merged object is greater than each of (i) the protective separation distance for the first non-machinery entity and (ii) the protective separation distance for the second non-machinery entity.

7 . The system of claim 1 , wherein the interaction between the first and second non-machinery entities corresponds to contact between the entities.

8 . The system of claim 1 , wherein the interaction between the first and second non-machinery entities corresponds to proximity of the entities.

9 . The system of claim 1 , wherein:

the processor is further configured to generate and store a zone object that specifies a zone within the workspace to be treated as empty; and

controlling operation of the machinery is further in accordance with the zone object.

10 . The system of claim 1 , the processor is further configured to:

generate a display of at least a portion of the workspace;

receive a command indicating entities in the workspace to be merged; and

merge objects associated with the entities indicated by the command into a single object.

11 . The system of claim 10 , wherein the processor is further configured to receive commands to change at least one attribute associated with the single object.

12 . The system of claim 1 wherein the processor is further configured to:

generate a display of at least a portion of the workspace;

receive commands indicating entities in the workspace associated with a merged object; and

generate and store separate objects corresponding to the indicated entities; and

delete the merged object.

13 . The system of claim 1 , wherein:

the processor is responsive to an external tracking system; and

the processor is further configured to identify at least the first or second non-machinery entity based on data from the external tracking system.

14 . The system of claim 13 , wherein the external tracking system includes at least one of (i) proximity sensors, (ii) IR break-beam sensors, or (iii) a 2D or 3D vision system.

15 . The system of claim 13 , wherein at least some of the first and second non-machinery entities include a feature to permit object and position tracking by the external tracking system.

16 . The system of claim 15 , wherein the feature is at least one of (i) an ultra-wideband transponder, (ii) an RFID tag, (iii) a barcode, or (iv) a QR code.

17 . The system of claim 13 , wherein the processor is further configured to make data available to a controller of the first or second non-machinery entity based on identifying the first or second non-machinery entity.

18 . A method of monitoring a workspace, the method comprising:

providing sensors configured to capture images of a workspace;

storing (i) captured images from the sensors and (ii) a model of machinery located within the workspace, wherein the model comprises movement capabilities of the machinery;

generating a spatial representation of the workspace based on the captured images;

identifying first and second non-machinery entities within the workspace;

generating a first object that specifies attributes of the first non-machinery entity comprising (i) a spatial extent of the first non-machinery entity and (ii) a protective separation distance for the first non-machinery entity;

generating a second object that specifies attributes of the second non-machinery entity comprising (i) a spatial extent of the second non-machinery entity and (ii) a protective separation distance for the second non-machinery entity;

recognizing an interaction between the first and second non-machinery entities in the workspace;

responsive to recognizing the interaction, merging the first and second objects into a merged object that specifies a protective separation distance for the merged object; and

controlling operation of the machinery in accordance with (i) a safety protocol and (ii) the protective separation distance for the merged object.

19 . The method claim 18 , further comprising:

recognizing dissociation between the first and second non-machinery entities;

generating and storing separate object data structures corresponding to the first and second non-machinery entities and specifying at least some different attributes.

20 . A system for monitoring a workspace, the system comprising:

sensors configured to capture images of a workspace;

memory configured to store (i) captured images from the sensors and (ii) a model of machinery located within the workspace, wherein the model comprises movement capabilities of the machinery; and

a processor configured to:

generate a spatial representation of the workspace based on the captured images;

identify first and second non-machinery entities within the workspace;

generate a first object that specifies attributes of the first non-machinery entity comprising (i) a spatial extent of the first non-machinery entity and (ii) a protective separation distance for the first non-machinery entity;

generate a second object that specifies attributes of the second non-machinery entity comprising (i) a spatial extent of the second non-machinery entity and (ii) a protective separation distance for the second non-machinery entity;

monitor a distance between the first and second non-machinery entities; and

control operation of the machinery in accordance with (i) the monitored distance,

(ii) the attributes of the first non-machinery entity, and (iii) the attributes of the second non-machinery entity.

Assignments (1)

Continuity (5)

References Cited (76)