Dynamically determining and monitoring workspace safe zones using semantic representations of workpieces

1. A safety system for identifying safe regions in a three-dimensional (3D) workspace including machinery performing a planned activity, the system comprising:

a plurality of sensors distributed about the workspace, each of the sensors comprising a grid of pixels for recording images of a portion of the workspace within a sensor field of view, the workspace portions collectively covering the entire workspace;

a computer memory for storing (i) a plurality of images from the sensors, (ii) a model of the machinery and its permitted movements during performance of the activity, and (iii) a safety protocol specifying speed restrictions of the machinery in proximity to a human and a minimum separation distance between the machinery and a human; and

a processor configured to:

computationally generate, from the stored images, a 3D spatial representation of the workspace;

identify and monitor over time a representation of space occupied by the machinery within the workspace as a 3D machinery region and computationally define, around the machinery region, a 3D envelope region spanning all points reachable by the machinery only during performance of the planned activity in accordance with the stored model, the 3D envelope region encompassing less than all points that the machinery is capable of reaching;

recognize interaction between the machinery and a workpiece within the workspace;

in response to the recognized interaction, update the 3D envelope region to include the workpiece; and

computationally generate a 3D safe zone around the machinery region, the 3D safe zone including the 3D envelope region as updated, in accordance with the safety protocol.

2. The safety system of claim 1 , wherein the processor is further configured to:

identify a human-occupied region in the volume corresponding to space occupied or potentially occupied by a human within the workspace; and

restricting the machinery's activity in accordance with the safety protocol based on proximity between the machinery region and the human-occupied region.

3. The safety system of claim 2 , wherein the human-occupied region is augmented by a 3D envelope around the human corresponding to anticipated movements of the human within the workspace within a predetermined future time.

4. The safety system of claim 1 , wherein the processor is further configured to recognize, in the images, items in the workspace other than the machinery and the workpiece, the processor identifying, as human, detected items not part of the machinery or workpiece and not otherwise recognized.

5. The safety system of claim 4 , wherein the processor is configured to detect, in the images, items within the workspace and to receive externally provided identifications thereof, the processor identifying, as human, detected items not part of the machinery or workpiece and for which no externally provided identification has been received.

6. The safety system of claim 1 , wherein the workspace is computationally represented as a plurality of voxels.

7. The safety system of claim 1 , wherein the machinery is at least one robot and the current state corresponds to current joint positions and velocities of the robot.

8. The system of claim 1 , wherein the 3D envelope region is spatially smaller than and lies within a spatial region encompassing all points that the machinery is kinematically capable of reaching.

9. The system of claim 1 , wherein the 3D envelope region, after being updated to include the workpiece, comprises points extending beyond a spatial region encompassing all points that the machinery is kinematically capable of reaching prior to the recognized interaction with the workpiece.

10. The system of claim 1 , wherein a second 3D safe zone extends beyond the 3D envelope region as updated to define an outer safety zone, entry of the human-occupied region into the second 3D safe zone causing the processor to operate the machinery at a reduced speed.

11. A method of safely operating machinery in a three-dimensional (3D) workspace, the method comprising the steps of:

monitoring the workspace with a plurality of sensors distributed thereabout, each of the sensors comprising a grid of pixels for recording images of a portion of the workspace within a sensor field of view, the workspace portions partially overlapping with each other;

registering the sensors with respect to each other so that the images obtained by the sensors collectively represent the workspace;

storing, in a computer memory, (i) a plurality of images from the sensors, (ii) a model of the machinery and its permitted movements during performance of a planned activity, and (iii) a safety protocol specifying speed restrictions of the machinery in proximity to a human and a minimum separation distance between a machine and a human;

computationally generating, from the stored images, a 3D spatial representation of the workspace;

computationally identifying and monitoring over time a representation of space occupied by the machinery within the workspace as a 3D machinery region and computationally defining, around the machinery region, a 3D envelope region spanning all points reachable by the machinery only during performance of the planned activity in accordance with the stored model, the 3D envelope region encompassing less than all points that the machinery is capable of reaching;

recognizing interaction between the machinery and a workpiece within the workspace;

in response to the recognized interaction, computationally updating the 3D envelope region to include the workpiece; and

computationally generating a 3D safe zone around the machinery region, the 3D safe zone including the 3D envelope region as updated, in accordance with the safety protocol.

12. The method of claim 11 , further comprising the steps of:

identifying a human-occupied region in the volume corresponding to space occupied by a human within the workspace; and

restricting the machinery's activity in accordance with the safety protocol based on proximity between the machinery region and the human-occupied region.

13. The method of claim 12 , further comprising the step of augmenting the human-occupied region by a 3D envelope around the human corresponding to anticipated movements of the human within the workspace within a predetermined future time.

14. The method of claim 11 , further comprising the steps of (i) recognizing, in the images, items in the workspace other than the machinery and the workpiece, and (ii) identifying, as human, detected items not part of the machinery or workpiece and not otherwise recognized.

15. The method of claim 14 , further comprising the steps of (i) detecting, in the images, items within the workspace and receiving externally provided identifications thereof, and (ii) identifying, as human, detected items not part of the machinery or workpiece and for which no externally provided identification has been received.

16. The method of claim 11 , wherein the workspace is computationally represented as a plurality of voxels.

17. The method of claim 11 , wherein the machinery is at least one robot and the current state corresponds to current joint positions and velocities of the robot.

18. The method of claim 11 , wherein the 3D envelope region is spatially smaller than and lies within a spatial region encompassing all points that the machinery is kinematically capable of reaching.

19. The method of claim 11 , wherein the 3D envelope region, after being updated to include the workpiece, comprises points extending beyond a spatial region encompassing all points that the machinery is kinematically capable of reaching prior to the recognized interaction with the workpiece.

20. The method of claim 11 , wherein a second 3D safe zone extends beyond the 3D envelope region as updated to define an outer safety zone, entry of the human-occupied region into the second 3D safe zone causing the machinery to operate at a reduced speed.