Systems and methods for robotic grasp verification

1. A computer-implemented method of controlling a robotic apparatus for manipulating objects, comprising:

at a time window at least partially overlapping with a first point in time, obtaining first sensor data indicating at least a portion of an environment where the robotic apparatus resides;

feeding the first sensor data to at least one convolutional neural network (CNN) to generate a first output that feeds into at least one long short-term memory (LSTM) network;

at a time window at least partially overlapping with a second point in time that succeeds the first point in time, obtaining second sensor data indicating at least a portion of the environment;

feeding the second sensor data to the at least one CNN to generate a second output that feeds into the at least one LSTM network, the LSTM network further receiving feedback to itself based on a state of the LSTM achieved from the first point in time;

determining a robotic action based, at least in part, on an output of the at least one LSTM network; and

causing the robotic apparatus to perform the robotic action.

2. The method of claim 1 , wherein the first sensor data includes a first image and the second sensor data includes a second image.

3. The method of claim 2 , wherein the first image and second image are consecutive images within a sequence of images.

4. The method of claim 1 , wherein the robotic apparatus includes a robotic end-effector used to grasp one or more objects.

5. The method of claim 4 , wherein a plurality of statuses of the robotic apparatus are defined in accordance with a quantity of objects grasped by the robotic end-effector.

6. The method of claim 1 , wherein determining the robotic action comprises applying at least one reinforcement learning policy.

7. The method of claim 6 , wherein the reinforcement learning policy is applied, at least in part, to information indicating the second point in time.

8. A non-transitory computer-readable medium storing contents that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

feeding first sensor data to at least one convolutional neural network (CNN) to generate a first output that feeds into at least one long short-term memory (LSTM) network, the first sensor data indicating at least a portion of an environment where a robotic device resides, and the first sensor data obtained at a time window at least partially overlapping with a first point in time;

feeding second sensor data to the at least one CNN to generate a second output that feeds into the at least one LSTM network, the LSTM network further receiving feedback to itself based on a state of the LSTM achieved from the first point in time, and the second sensor data obtained at a time window at least partially overlapping with a second point in time;

determining a robotic action based, at least in part, on an output of the at least one LSTM network; and

causing the robotic device to perform the robotic action.

9. The computer-readable medium of claim 8 , wherein the acts further comprise applying a classifier to at least a subset of the first sensor data to assess at least a status of the robotic device prior to the feeding of second sensor data to the at least one CNN.

10. The computer-readable medium of claim 9 , wherein the acts comprise updating the classifier based on at least a subset of the second sensor data.

11. The computer-readable medium of claim 10 , wherein the acts further comprise applying the updated classifier to at least a subset of third sensor data indicating at least a portion of the environment.

12. The computer-readable medium of claim 8 , wherein determining the robotic action is based further on at least one reinforcement learning policy in accordance with reward values assigned to different status-action pairs.

13. The computer-readable medium of claim 12 , wherein the reward values include positive and negative numbers.

14. A system, comprising:

one or more processors; and

memory storing contents that, when executed by the one or more processors, cause the system to:

feed first sensor data obtained at a time window at least partially overlapping with a first time to at least one convolutional neural network (CNN) to generate a first output that feeds into at least one long short-term memory (LSTM) network;

feed second sensor data obtained at a time window at least partially overlapping with a second time to the at least one CNN to generate a second output that feeds into the at least one LSTM network, the LSTM network further receiving feedback to itself based on a state of the LSTM achieved from the first time;

determine a robotic action based, at least in part, on an output of the at least one LSTM network; and

cause a robotic device including a robotic end-effector to perform the robotic action.

15. The system of claim 14 , wherein the robotic action includes at least one of (a) abort—stop a current grasp with the robotic end-effector, and retry, (b) continue—wait for third sensor data to be obtained, or (c) stow—stow at least one item grasped by the robotic end-effector.

16. The system of claim 14 , wherein first time precedes the second time.

17. The system of claim 16 , wherein the first time and the second time are two points within a time sequence of sensor data obtained.

18. The system of claim 14 , wherein determining the robotic action includes applying a trained reinforcement learning agent.

19. The system of claim 18 , wherein the reinforcement learning agent is trained independently from training of the CNN and LSTM.