Granted Patent B2

US 12,397,434 · App. 18/740,527 · Granted Aug 26, 2025

Inventors: Clara Vu (Cambridge, MA); Scott Denenberg (Newton, MA); Patrick Sobalvarro (Harvard, MA); Alberto Moel (Cambridge, MA)

Assignee: Symbotic LLC

B25J9/1694B25J9/1666B25J9/1676B25J9/1697G01S7/4808G01S17/04G01S17/87G01S17/89G06T17/10G01V8/20G05B2219/40202G06T17/05Y10S901/47Y10S901/49

Abstract

Claims (49)

1. A method of safely operating machinery in a workspace, the method comprising:

recording images of a portion of a workspace;

generating a 3D representation of the portion of the workspace based on the recorded images, wherein the 3D representation comprises one or more volumes that correspond to the portion of the workspace;

identifying one or more of the volumes as being either occupied or unoccupied; and

mapping one or more safe zones based on the one or more identified volumes, wherein the safe zones correspond to one or more regions within the portion of the workspace for safe operation of machinery.

2. The method of claim 1 , wherein a particular safe zone of the one or more safe zones is detected based on user input.

3. The method of claim 2 , wherein:

recording images of the portion of the workspace comprises using a sensor associated with a grid of pixels configured to capture the recorded images, wherein the portion of the workspace is within a field of view of the sensor; and

the particular safe zone is disposed outside the sensor field of view.

4. The method of claim 2 , wherein:

recording images of the portion of the workspace comprises using a sensor associated with a grid of pixels configured to capture the recorded images, wherein the portion of the workspace is within a field of view of the sensor; and

the user input overrides a mapping based on images recorded by the sensor.

5. The method of claim 1 , wherein:

recording images of the portion of the workspace comprises using a sensor associated with a grid of pixels configured to capture the recorded images, wherein the portion of the workspace is within a field of view of the sensor; and

identifying a particular volume of the one or more volumes as occupied comprises receiving light via the sensor, wherein the light is reflected from an object disposed in the particular volume, and an intensity of the reflected light is higher than a predetermined threshold.

6. The method of claim 1 , wherein:

recording images of the portion of the workspace comprises using a sensor associated with a grid of pixels configured to capture the recorded images, wherein the portion of the workspace is within a field of view of the sensor; and

one or more objects within the sensor field of view are insufficiently reflective to reflect to the sensor light with an intensity higher than a predetermined threshold.

7. The method of claim 1 , further comprising associating the one or more safe zones with a safety criterion.

8. The method of claim 7 , wherein the safety criterion corresponds to a minimum distance from a piece of the machinery.

9. The method of claim 1 , wherein the one or more safe zones include only volumes of the one or more volumes that have been identified as unoccupied.

10. The method of claim 1 , further comprising, based on the recorded images, identifying a workpiece or an element of the machinery using a neural network.

11. The method of claim 1 , wherein:

recording images of the portion of the workspace comprises using a sensor associated with a grid of pixels configured to capture the recorded images, wherein the portion of the workspace is within a field of view of the sensor; and

identifying one or more of the volumes as being occupied or unoccupied comprises (i) detecting an occlusion in the sensor field of view, (ii) classifying volumes associated with the occlusion as occupied volumes, and (iii) classifying volumes between the sensor and the occupied volumes as unoccupied volumes.

12. The method of claim 1 , wherein:

recording images of the portion of the workspace comprises using a sensor associated with a grid of pixels configured to capture the recorded images, wherein the portion of the workspace is within a field of view of the sensor; and

the sensor comprises at least one of (i) a time-of-flight sensor, (ii) a 3D LIDAR sensor, or (iii) a stereo vision camera.

13. The method of claim 1 , further comprising:

recording images of another portion of a workspace;

generating another 3D representation of the other portion of the workspace based on the recorded images, wherein the other 3D representation comprises one or more other volumes that correspond to the other portion of the workspace; and

identifying one or more of the other volumes as being either occupied or unoccupied.

14. The method of claim 1 , further comprising:

receiving a task specification; and

generating a motion plan comprising a trajectory of the machinery for performing a specified task within the one or more safe zones.

15. The method of claim 14 , further comprising:

recording images of another portion of a workspace;

generating another 3D representation of the other portion of the workspace based on the recorded images, wherein the other 3D representation comprises one or more other volumes that correspond to the other portion of the workspace; and

identifying one or more of the other volumes as being either occupied or unoccupied;

wherein mapping the one or more safe zones is further based on the one or more identified other volumes.

16. The method of claim 14 , further comprising operating the machinery in accordance with the motion plan.

17. The method of claim 14 , wherein:

the motion plan comprises a plurality of trajectories of the machinery; and

the method further comprises (i) assigning a respective cost value to each of the trajectories, (ii) selecting one of the trajectories based at least in part on the respective cost value assigned to each of the trajectories, and (iii) causing the machinery to execute the selected trajectory.

18. The method of claim 17 , wherein the respective cost value associated with each of the trajectories is assigned based at least in part on (i) a length of the trajectory, (ii) an operation time of the machinery on the trajectory, or (iii) allowed kinematics of the trajectory.

19. The method of claim 14 , wherein the motion plan precludes entry into a zone outside the one or more safe zones by the machinery and any workpiece associated with the machinery or the task.

20. The method of claim 14 , further comprising:

monitoring the workspace in real-time; and

altering the motion plan in response to a change in the one or more safe zones.

Assignments (2)

Continuity (7)

References Cited (100)