Robotic control system

1. A system, comprising:

an articulated robot with a manipulator;

one or more processors; and

memory storing executable instructions that, as a result of being executed by the one or more processors, cause the system to at least:

identify a real-world object and a grasp pose with which to grasp the real-world object;

obtain a first image from a camera, the camera positioned to track an axis of the manipulator;

move the manipulator to a real-world pre-grasp position that matches a simulated pre-grasp position determined based at least in part on the grasp pose and an estimated pose of the real-world object determined using the first image;

provide a second image from the camera to a neural network to produce a value from a value function; and

move the manipulator based on the value to grasp the real-world object with the manipulator.

2. The system of claim 1 , wherein the neural network is trained using a Q-value loss and a set of auxiliary losses generated by the neural network.

3. The system of claim 1 , wherein the value is a Q-value.

4. The system of claim 1 , wherein the manipulator is a claw or mechanical hand.

5. The system of claim 1 , wherein:

the articulated robot includes a wrist member; and

the camera is attached to the wrist member of the articulated robot.

6. The system of claim 1 , wherein the executable instructions further cause the system to at least:

determine a 6-d pose of the real-world object;

determine a pre-grasp position for the manipulator based at least in part on the 6-d pose of the real-world object; and

move the manipulator to the pre-grasp position prior to obtaining the first image from the camera.

7. The system of claim 6 , wherein the executable instructions further cause the system to at least:

capture an image of the real-world object using a camera not mounted on the articulated robot; and

determine the 6-d pose of the real-world object using the image and a location of the camera not on the articulated robot.

8. A method, comprising: performing a grasp of a real-world object with a robot under control of one or more neural networks, at least in part, by:

identifying the real-world object and a grasp pose with which to grasp the real-world object;

obtaining a first image from a camera that tracks an axis of a manipulator of the robot;

moving the manipulator to a real-world pre-grasp position that matches a simulated pre-grasp position determined based at least in part on the grasp pose and an estimated pose of the real-world object determined using the first image;

providing a second image from the camera to a neural network to produce a value from a value function; and

moving the manipulator based on the value to grasp the real-world object with the manipulator.

9. The method of claim 8 , wherein the grasp is performed at least in part by:

determining a 6-d pose of the real-world object; and

grasping the real-world object in accordance with a specific grasp pose.

10. The method of claim 9 , wherein the grasp is performed at least in part by:

determining the specific grasp pose for the real-world object based at least in part on the 6-d pose of the real-world object; and

grasping the real-world object in accordance with the specific grasp pose.

11. The method of claim 8 , wherein the one or more neural networks are trained, at least in part, by:

generating a simulation of a simulated robot attempting to grasp a simulated object with the manipulator;

using the simulation to determine a result of an interaction between the simulated robot and the simulated object;

generating an image from a point of view of the manipulator; and

using the image and the result of the interaction to train the one or more neural networks.

12. The method of claim 8 , wherein an action for the robot is selected from a plurality of possible actions based at least in part on a plurality of Q-values that correspond to the plurality of possible actions.

13. The method of claim 8 , wherein:

the one or more neural networks produce a set of auxiliary losses;

the set of auxiliary losses includes at least one of distance of the manipulator from the real-world object, an indication of contact with the real-world object, and a force estimation of the manipulator; and

the one or more neural networks are trained, based at least in part, on the set of auxiliary losses.

14. The method of claim 11 , wherein:

the image is processed by a first neural network to produce an embedding; and

the embedding is processed using a second neural network to produce a Q-value associated with a proposed action.

15. A processor, comprising one or more arithmetic logic units (ALUs) to direct a robot to grasp a real-world object by at least:

using a neural network to generate an action from a first image captured by an in-hand camera on the robot;

moving the robot to a real-world pre-grasp position that matches a simulated pre-grasp position determined based at least in part on an estimated pose of the real-world object determined using the first image;

causing the robot to move in accordance with the action;

determining, based on a Q-value produced by the neural network using a second image from the in-hand camera, that the robot is properly positioned to grasp the real-world object; and

causing the robot to grasp the real-world object.

16. The processor of claim 15 , wherein the processor determines an action to be taken by the robot by at least:

providing the first image and a distribution of possible actions to the neural network to generate a set of Q-values that correspond to the possible actions; and

selecting the action from the possible actions based on the set of Q-values.

17. The processor of claim 15 , wherein the one or more arithmetic logic units further cause the robot to at least:

determine a position and an orientation of the real-world object; and

prior to capturing the first image, move a gripper to a pre-grasp position, the pre-grasp position determined from the position and the orientation of the real-world object.

18. The processor of claim 15 , wherein the neural network is trained using a double-deep reinforcement learning algorithm.

19. The processor of claim 15 , wherein:

the in-hand camera is mounted to a wrist element of an articulated robot; and

the in-hand camera is oriented substantially on a primary axis of a robotic gripper attached to the wrist element.

20. The processor of claim 15 , wherein the first image is downsampled from a raw image acquired from the in-hand camera.

21. The processor of claim 16 , wherein the action is a motion defined in a 4-dimensional action space centered around a robotic gripper of the robot.

22. The processor of claim 15 , wherein:

the robot includes a mechanical hand having a plurality of digits.

23. The processor of claim 15 , wherein:

the robot includes a manipulator mounted to a vehicle; and

the manipulator is positioned at least in part by moving the vehicle.