Machine learning control of object handovers

1. A processor, comprising one or more computers comprising one or more processors to:

obtain a point cloud that represents a hand holding an object;

determine, from a first portion of the point cloud, a pose of the object;

determine, from a second portion of the point cloud, a pose of the hand;

generate a set of grasp poses that allow a robot to grasp the object;

select, from the set of grasp poses, based at least in part on the pose of the hand, a target grasp pose that does not interfere with the hand; and

cause the robot to perform the target grasp pose.

2. The processor of claim 1 , wherein the pose of the hand identifies a plurality of segments and joint angles.

3. The processor of claim 1 , wherein the one or more processors:

obtain a three-dimensional image from a depth camera; and

produce the point cloud from the three-dimensional image.

4. The processor of claim 1 , wherein:

the set of grasp poses are poses for a robotic gripper of the robot; and

the robotic gripper has two opposed digits that perform the grasp.

5. The processor of claim 1 , wherein the pose of the object includes three angles that indicate an orientation of the object and information that identifies a position of the object.

6. The processor of claim 1 , wherein the robot takes the object from the hand.

7. A system, comprising:

one or more processors coupled to non-transitory computer-readable media;

the non-transitory computer-readable media storing executable instructions that, as a result of being executed by the one or more processors, cause the system to:

determine, from a three-dimensional image of an appendage holding an object, a pose of the appendage and a pose of the object;

determine a set of grasp poses that allow a robotic gripper to grasp the object;

select, from the set of grasp poses, a grasp pose that does not interfere with the appendage; and

perform the grasp pose.

8. The system of claim 7 , wherein the three dimensional image is generated from a depth camera, radar image, LIDAR image, or three dimensional medical imaging device.

9. The system of claim 7 , wherein the executable instructions, as a result of being executed by the one or more processors, further cause the system to:

generate a point cloud from the three-dimensional image;

identify a first portion of the point cloud that represents the appendage; and

identify a second portion of the point cloud that represents the object.

10. The system of claim 7 , wherein the executable instructions, as a result of being executed by the one or more processors, further cause the system to:

determine a type of hand pose from the three-dimensional image; and

determine a grasp pose based at least in part on the type of hand pose.

11. The system of claim 7 , wherein the executable instructions, as a result of being executed by the one or more processors, further cause the system to:

generate a point cloud from the three-dimensional image; and

provide the point cloud to a trained model that outputs the grasp pose.

12. The system of claim 11 , wherein the train model is trained by providing ground truth data that includes point cloud information and corresponding grasp poses.

13. The system of claim 7 , wherein the grasp pose is determined to interfere with the appendage when the robotic gripper is predicted to touch the appendage during performance of the grasp pose.

14. A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:

obtain a three-dimensional image of an appendage holding an object;

generate a 3D model from the three dimensional image;

provide the 3D model to a trained network, the trained network producing a grasp pose for a robotic gripper that is able to grasp the object without contacting the appendage; and

cause the robotic gripper to perform the grasp pose.

15. The non-transitory machine-readable medium of claim 14 , wherein the trained network is trained at least in part by providing, to a network, training data that includes a colored 3D model of an appendage holding an object, and a proposed grasp for the robotic gripper such that the robotic gripper can receive the object from the appendage without touching the appendage.

16. The non-transitory machine-readable storage medium of claim 15 , wherein training data is generated by at least:

generating a dataset of human object handovers; and

annotating the dataset with ground truth hand poses and ground truth object poses.

17. The non-transitory machine-readable medium of claim 14 , wherein the instructions further comprise instructions that, as a result of being executed by the one or more processors, cause the one or more processors to:

determine, from the 3D model, a pose of the appendage; and

determine, from the 3D model, a pose of the object.

18. The non-transitory machine-readable storage medium of claim 17 , wherein:

the grasp pose is based at least in part on the pose of the object and the pose of the appendage; and

the grasp pose is determined as a grasp that will successfully grasp the object while not touching the appendage.

19. The non-transitory machine-readable storage medium of claim 14 , wherein the network is trained using images of the appendage holding different object types.

20. The non-transitory machine-readable medium of claim 14 , wherein the instructions further comprise instructions that, as a result of being executed by the one or more processors, cause the one or more processors to determine the grasp pose as a result of determining that the pose of the appendage is a pose type that is offering the object to the robot.

21. A processor comprising: one or more arithmetic logic units (ALUs) to train one or more neural networks, at least in part, by providing, to a network, training data that includes a 3D model of a appendage holding an object, and a proposed grasp for the robotic gripper such that the robotic gripper can receive the object from the appendage without interfering with the appendage.

22. The processor of claim 21 , wherein training data is generated by at least:

generating a dataset of human object handovers; and

annotating the dataset with ground truth hand poses and ground truth object poses.

23. The processor of claim 21 , wherein the network:

determines, from the 3D model, a pose of the appendage; and

determines, from the 3D model, a pose of the object.

24. The processor of claim 21 , wherein the 3D model is generated from a three dimensional image of the appendage holding the object.

25. The processor of claim 21 , wherein the 3D model is a point cloud.

26. The processor of claim 21 , wherein the appendage is a human hand, a robotic gripper or hand, a part of an animal, or a human leg or arm.