Learning-embedded motion planning
Method of sequential handling of a plurality of freight units by an end effector of a robotic system comprises training a neural network operable to bias a plurality of samples of a configuration space of the end effector. Deployment of the trained neural network involves sensing an arrangement of freight units in a scene based on an image of the scene from a sensor. A freight unit is chosen based on the at least one trained neural network. A starting pose of the robotic end effector as a means to pick the chosen freight unit is chosen; and a goal pose of the robotic end effector is chosen as a means to place the chosen freight unit. A motion of the robotic end effector is planned from the starting pose to the goal pose based on a biased plurality of samples. Then motion on the end effector is controlled based on the planning.
1. A method of sequential handling of a plurality of freight units by an end effector of a robotic system, the method comprising:
training, by a programmed computer system, at least one neural network operable to bias a plurality of samples of a configuration space of the end effector; and
after training of the at least one neural network, deploying the neural network, wherein deploying the at least one neural network comprises:
sensing, by a computer system of the robotic system, an arrangement of freight units in a scene based on an image of the scene from a sensor;
choosing, by the computer system of the robotic system, a freight unit based on the at least one trained neural network;
choosing, by the computer system of the robotic system, a starting pose of the robotic end effector as a means to pick the chosen freight unit;
choosing, by the computer system of the robotic system, a goal pose of the robotic end effector as a means to place the chosen freight unit;
planning, by the computer system of the robotic system, a motion of the robotic end effector from the starting pose to the goal pose based on a biased plurality of samples; and
controlling, by the computer system of the robotic system, the motion on the end effector based on the planning.
2. The method of claim 1 , wherein the end effector comprises a robotic arm and the robotic system further comprises a mobile base.
3. The method of claim 1 , wherein the sensor of the robotic system comprises a camera.
4. The method of claim 1 , wherein the neural network comprises a feed-forward neural network.
5. The method of claim 1 , wherein the neural network comprises a 3D convolutional neural network.
6. The method of claim 1 , wherein the training of the neural network comprises:
simulating a random arrangement of freight units;
choosing a random freight unit to handle;
generating a plurality of samples of a configuration space of the robotic end effector;
measuring a stability of the random arrangement of freight units; and assigning a bias to each of the plurality of samples of a configuration space of the robotic end effector to minimize a cycle time of a path through the samples.
7. The method of claim 6 , further comprising assigning, by the programmed computer system, a bias to each of the plurality of samples of a configuration space of the end effector to maximize a smoothness of a path through the samples.
8. The method of claim 6 , wherein the path is selected from a random tree of the samples.
9. A system of sequential handling of a plurality of freight units by an end effector of a robotic system, the system comprising:
a first computer system for training a neural network operable to bias a plurality of samples of a configuration space of the end effector; and
the robotic system comprising a sensor, a robot computer system, and the end effector, wherein:
the sensor is for capturing an image of freight units in a scene; and
the robot computer system is configured to:
sense an arrangement of the freight units in the scene based on the image;
choose a freight unit based on the trained neural network;
choose a starting pose of the robotic end effector as a means to pick the chosen freight unit;
choose a goal pose of the robotic end effector as a means to place the chosen freight unit;
plan a motion of the end effector from the starting pose to the goal pose; and
control the motion on the robotic end effector based on the planning.
10. The system of claim 9 , wherein the end effector comprises a robotic arm.
11. The system of claim 9 , wherein the sensor of the robotic system comprises a camera.
12. The system of claim 9 , wherein the neural network comprises a feed-forward neural network.
13. The system of claim 9 , wherein the neural network comprises a 3D convolutional neural network.
14. The system of claim 9 , wherein the first computer system is for training the neural network by:
simulating a random arrangement of freight units;
choosing a random freight unit to handle;
generating a plurality of samples of a configuration space of the robotic end effector; and
measuring a stability of the random arrangement of freight units; and assigning a bias to each of the plurality of samples of a configuration space of the robotic end effector to minimize a cycle time of a path through the samples.
15. The system of claim 14 , wherein the first computer system is further configured to assign a bias to each of the plurality of samples of a configuration space of the end effector to maximize a smoothness of a path through the samples.
16. The system of claim 14 , wherein the path is selected from a random tree of the samples.