Task impact zone and 4-dimensional safety envelope for robots
Disclosed herein are systems, devices, and methods of a safety system for analyzing and improving the safety of environments that may be shared between robots and humans. The safety system may receive a safety envelope that includes a reachable set of locations at an expected position of an object at a prediction time. The receiver may also receive a perception prediction that may be based on the safety envelope and may include environmental information associated with the object at the expected position at the prediction time. The safety system may also include a processor that generates an instruction to move a robot according to a safe movement instruction based on whether a perception check exceeds a predetermined threshold, wherein the perception check may be based on a difference between the perception prediction and sensor information indicative of the environment of the object at the prediction time.
1 . A device comprising:
a receiver configured to
receive a safety envelope for an object, wherein the safety envelope comprises a reachable set of locations that the object may reach at an expected position of the object at a prediction time,
receive a perception prediction for an environment of the object, wherein the perception prediction comprises a predicted value of sensor data with respect to the object in the environment at the prediction time, wherein the predicted value of the sensor data is based on the safety envelope and comprises environmental information associated with the object at the expected position at the prediction time, and
a processor configured to:
generate an instruction to move a robot according to a safe movement instruction based on whether a perception check exceeds a predetermined threshold, wherein the perception check is based on a difference between the perception prediction and sensor information indicative of the environment of the object at the prediction time, wherein the processor is configured to determine the prediction time based on a subinterval of an expected latency to move the robot according to the safe movement instruction; and
control the robot to move according to the safe movement instruction.
2 . The device of claim 1 , wherein the reachable set of locations is based on object status information associated with the object at a first location at a first time, wherein the prediction time is subsequent to the first time.
3 . The device of claim 2 , wherein the object status information comprises a velocity, a trajectory, and/or or an acceleration of the object.
4 . The device of claim 1 , wherein the safe movement instruction is associated with the prediction time and based on the safety envelope.
5 . The device of claim 1 , further comprising a sensor configured to provide the sensor information to the receiver, wherein the sensor comprises a depth sensor, a camera, a radar, a light ranging and detection sensor, and/or an ultrasonic sensor.
6 . The device of claim 1 , wherein the safety envelope comprises a plurality of reachable sets, wherein each reachable set of the plurality of reachable sets is for an associated prediction time of a plurality of prediction times, wherein each reachable set comprises a reachable set of locations that the object may reach at the associated prediction time.
7 . The device of claim 1 , wherein the safety envelope for the object comprises a plurality of safety envelopes for a plurality of objects, each safety envelope of the plurality of safety envelopes corresponding to an associated one of the plurality of objects.
8 . The device of claim 1 , wherein the processor is further configured to generate an instruction to modify the safe movement instruction of the robot if the perception check exceeds the predetermined threshold.
9 . The device of claim 1 , wherein the difference comprises an actual object position of the object that is different from the expected position.
10 . The device of claim 1 , wherein the receiver is further configured to receive the safety envelope at an update time, wherein if the perception check exceeds the predetermined threshold, the processor is further configured to generate an instruction to modify the safe movement instruction of the robot based on an elapsed time between the update time and the prediction time.
11 . The device of claim 1 , wherein processor is further configured to generate a subsequent safety envelope for a subsequent prediction time, wherein the subsequent safety envelope is based on an extrapolation of the safety envelope to the subsequent prediction time.
12 . A non-transitory computer readable medium, including instructions which, if executed, cause a receiver to:
receive a selected task from a plurality of possible tasks, wherein each possible task of the plurality of possible tasks is associated with a safety attribute of the possible task; and
receive a perception prediction for an environment of an object, wherein the perception prediction comprises a predicted value of sensor data with respect to the object in the environment at the prediction time, wherein the predicted value of the sensor data is based on the safety envelope and comprises environmental information associated with the object at an expected position at a prediction time,
wherein the instructions are further configured to cause a processor to:
generate an instruction to move a robot according to a safe movement instruction based on whether a perception check exceeds a predetermined threshold, wherein the perception check is based on a difference between the perception prediction and sensor information indicative of the environment of the object at the prediction time, wherein the instructions further cause the processor to determine the prediction time based on a subinterval of an expected latency to move the robot according to the safe movement instruction; and
control the robot to move according to the safe movement instruction.
13 . The non-transitory computer readable medium of claim 12 , wherein the safe movement instruction comprises a selected task that is selected from a plurality of possible tasks, wherein each possible task of the plurality of possible tasks is associated with a safety attribute of the possible task.
14 . The non-transitory computer readable medium of claim 13 , wherein the safety attribute comprises an impact zone of the possible task, an approved list of allowed objects allowed in the impact zone, a disapproved list of disallowed object prohibited from the impact zone, and/or a motion restriction to robot motions associated with the possible task.
15 . The non-transitory computer readable medium of claim 14 , wherein the impact zone comprises an area around the robot that may be impacted by robot motions associated with the possible task, wherein the impact zone is based on a current environment of the robot and a hazard rating of the possible task.
16 . The non-transitory computer readable medium of claim 14 , wherein the motion restriction comprises at least one of the following: a speed limit, a force limit, an acceleration limit, or a partial motion limit to the robot motions for the possible task.
17 . The non-transitory computer readable medium of claim 16 , wherein the safety attribute comprises a simulated safety attribute based on simulated movements of the robot in a simulated environment associated with the possible task.
18 . The non-transitory computer readable medium of claim 12 , wherein the receiver is further configured to receive sensor data indicative of a current environment of the robot, wherein the selected task is selected based on a comparison of the current environment to the simulated environment.
19 . The non-transitory computer readable medium of claim 13 , wherein the receiver is further configured to receive sensor data indicative of a current environment of the robot, wherein the processor is further configured to generate a risk mitigation instruction based on the current environment, wherein the risk mitigation instruction is configured to modify based on the current environment the safe movement instruction to comply with the motion restriction, to respond to an object in the current environment that is not on the approved list or is on the disapproved list, and/or to request a new task from the plurality of possible tasks.
20 . The non-transitory computer readable medium of claim 19 , wherein the risk mitigation instruction comprises at least one of the following: an instruction to reduce a speed of the robot, an instruction to change a trajectory of the robot, an instruction to interrupt the selected task, and/or an instruction to provide a warning message.
21 . An apparatus comprising:
a means for receiving a safety envelope for an object, wherein the safety envelope comprises a reachable set of locations that the object may reach at an expected position of the object at a prediction time,
a means for receiving a perception prediction for an environment of the object, wherein the perception prediction comprises a predicted value of sensor data with respect to the object in the environment at the prediction time, wherein the predicted value of the sensor data is based on the safety envelope and comprises environmental information associated with the object at the expected position at the prediction time, and
a means for generating an instruction to move a robot according to a safe movement instruction based whether a perception check exceeds a predetermined threshold, wherein the perception check is based on a difference between the perception prediction and sensor information indicative of the environment of the object at the prediction time; a means for determining the prediction time based on a subinterval of an expected latency to move the robot according to the safe movement instruction; and
a means for controlling the robot to move according to the safe movement instruction.
22 . The apparatus of claim 21 , wherein the reachable set of locations is based on object status information associated with the object at a first location at a first time, wherein the prediction time is subsequent to the first time.
23 . The apparatus of claim 22 , wherein the object status information comprises a velocity, a trajectory, and/or or an acceleration of the object.
24 . The apparatus of claim 21 , wherein the safety envelope for the object comprises a plurality of safety envelopes for a plurality of objects, each safety envelope of the plurality of safety envelopes corresponding to an associated one of the plurality of objects.
25 . The apparatus of claim 21 , the apparatus further comprising a means for generating an instruction to modify the safe movement instruction of the robot if the perception check exceeds the predetermined threshold.