Method for detecting grasping position of robot in grasping object

1. A method for detecting a grasping position of a robot in grasping a target object, comprising:

collecting at least one target RGB image and at least one target Depth image of the target object at different view angles, wherein pixel points in the at least one target RGB image have one-to-one corresponding pixel points in the at least one target Depth image;

inputting each of the at least one target RGB image to a target object segmentation network for calculation to obtain an RGB pixel region of the target object in the target RGB image and a Depth pixel region of the target object in the target Depth image;

inputting the RGB pixel region of the target object to an optimal grasping position generation network to obtain an optimal grasping position for grasping the target object;

inputting the Depth pixel region of the target object and the optimal grasping position to a grasping position quality evaluation network to calculate a score of the optimal grasping position; and

selecting an optimal grasping position corresponding to a highest score as a global optimal grasping position of the robot.

2. The method according to claim 1 , wherein

the target object segmentation network is a network trained based on a convolutional neural network model, wherein the target object segmentation network is trained by:

acquiring an RGB image containing the target object;

zooming the RGB image to a first predetermined resolution to obtain a first training set;

annotating a pixel region corresponding to the target object in the first training set;

inputting the first training set and the pixel region corresponding to the target object to the convolutional neural network model for training to obtain the target object segmentation network;

performing an overlap comparison between the RGB pixel region corresponding to the target object obtained by the target object segmentation network and the annotated pixel region corresponding to the target object; and

adjusting a weight of the target object segmentation network according to the overlap comparison result.

3. The method according to claim 1 , wherein the optimal grasping position generation network is a network trained based on a convolutional neural network model, and the optimal grasping position generation network is trained by:

zooming the RGB pixel region corresponding to the target object obtained based on the target object segmentation network to a second predetermined resolution to obtain a second training set;

marking optimal grasping position coordinates for an image in the second training set; and

taking the image in the second training set and the corresponding optimal grasping position coordinates as inputs, and training the inputs based on the convolutional neural network model to obtain the optimal grasping position generation network.

4. The method according to claim 1 , wherein the grasping position quality evaluation network is a network trained based on a convolutional neural network model, and the grasping position quality evaluation network is trained by:

acquiring a Depth image containing the target object;

zooming the Depth image to a third predetermined resolution to obtain a third training set;

randomly acquiring a pair of grasping positions on the Depth image in the third training set, and calculating a corresponding score by a predetermined scoring algorithm; and

taking the Depth image, the grasping positions and the score corresponding to the grasping positions as inputs, and training the inputs based on the convolutional neural network model to obtain the grasping position quality evaluation network.

5. The method according to claim 2 , wherein the first predetermined resolution is a 320×320 pixel.

6. The method according to claim 3 , wherein the second predetermined resolution is a 227×227 pixel.

7. The method according to claim 4 , wherein the third predetermined resolution is a 32×32 pixel.

8. A computing device, comprising: a processor, a memory, a communication interface and a communication bus; wherein the processor, the memory and the communication bus communicate with each other via the communication bus; and

the memory is configured to store at least one executable instruction, wherein the at least one executable instruction causes the processor to perform the steps of:

collecting at least one target RGB image and at least one target Depth image of the target object at different view angles, wherein pixel points in the at least one target RGB image have one-to-one corresponding pixel points in the at least one target Depth image;

inputting each of the at least one target RGB image to a target object segmentation network for calculation to obtain an RGB pixel region of the target object in the target RGB image and a Depth pixel region of the target object in the target Depth image;

inputting the RGB pixel region of the target object to an optimal grasping position generation network to obtain an optimal grasping position for grasping the target object;

inputting the Depth pixel region of the target object and the optimal grasping position to a grasping position quality evaluation network to calculate a score of the optimal grasping position; and

selecting an optimal grasping position corresponding to a highest score as a global optimal grasping position of the robot.

9. The computing device according to claim 8 , wherein the target object segmentation network is a network trained based on a convolutional neural network model, wherein the target object segmentation network is trained by:

acquiring an RGB image containing the target object;

zooming the RGB image to a first predetermined resolution to obtain a first training set;

annotating a pixel region corresponding to the target object in the first training set;

inputting the first training set and the pixel region corresponding to the target object to the convolutional neural network model for training to obtain the target object segmentation network;

performing an overlap comparison between the RGB pixel region corresponding to the target object obtained by the target object segmentation network and the annotated pixel region corresponding to the target object; and

adjusting a weight of the target object segmentation network according to the overlap comparison result.

10. The computing device according to claim 8 , wherein the optimal grasping position generation network is a network trained based on a convolutional neural network model, and the optimal grasping position generation network is trained by:

zooming the RGB pixel region corresponding to the target object obtained based on the target object segmentation network to a second predetermined resolution to obtain a second training set;

marking optimal grasping position coordinates for an image in the second training set; and

taking the image in the second training set and the corresponding optimal grasping position coordinates as inputs, and training the inputs based on the convolutional neural network model to obtain the optimal grasping position generation network.

11. The computing device according to claim 8 , wherein the grasping position quality evaluation network is a network trained based on a convolutional neural network model, and the grasping position quality evaluation network is trained by:

acquiring a Depth image containing the target object;

zooming the Depth image to a third predetermined resolution to obtain a third training set;

randomly acquiring a pair of grasping positions on the Depth image in the third training set, and calculating a corresponding score by a predetermined scoring algorithm; and

taking the Depth image, the grasping positions and the score corresponding to the grasping positions as inputs, and training the inputs based on the convolutional neural network model to obtain the grasping position quality evaluation network.

12. The computing device according to claim 9 , wherein the first predetermined resolution is a 320×320 pixel.

13. The computing device according to claim 10 , wherein the second predetermined resolution is a 227×227 pixel.

14. The computing device according to claim 11 , wherein the third predetermined resolution is a 32×32 pixel.

15. A computer-readable storage medium, the storage medium storing at least one executable instruction; wherein the at least one executable instruction, when being executed, causes the processor to perform the steps of:

collecting at least one target RGB image and at least one target Depth image of the target object at different view angles, wherein pixel points in the at least one target RGB image have one-to-one corresponding pixel points in the at least one target Depth image;

inputting each of the at least one target RGB image to a target object segmentation network for calculation to obtain an RGB pixel region of the target object in the target RGB image and a Depth pixel region of the target object in the target Depth image;

inputting the RGB pixel region of the target object to an optimal grasping position generation network to obtain an optimal grasping position for grasping the target object;

inputting the Depth pixel region of the target object and the optimal grasping position to a grasping position quality evaluation network to calculate a score of the optimal grasping position; and

selecting an optimal grasping position corresponding to a highest score as a global optimal grasping position of the robot.

16. The computer-readable storage medium according to claim 15 , wherein the target object segmentation network is a network trained based on a convolutional neural network model, wherein the target object segmentation network is trained by:

acquiring an RGB image containing the target object;

zooming the RGB image to a first predetermined resolution to obtain a first training set;

annotating a pixel region corresponding to the target object in the first training set;

inputting the first training set and the pixel region corresponding to the target object to the convolutional neural network model for training to obtain the target object segmentation network;

performing an overlap comparison between the RGB pixel region corresponding to the target object obtained by the target object segmentation network and the annotated pixel region corresponding to the target object; and

adjusting a weight of the target object segmentation network according to the overlap comparison result.

17. The computer-readable storage medium according to claim 15 , wherein the optimal grasping position generation network is a network trained based on a convolutional neural network model, and the optimal grasping position generation network is trained by:

zooming the RGB pixel region corresponding to the target object obtained by target object segmentation network to a second predetermined resolution to obtain a second training set;

marking optimal grasping position coordinates for an image in the second training set; and

taking the image in the second training set and the corresponding optimal grasping position coordinates as inputs, and training the inputs based on the convolutional neural network model to obtain the optimal grasping position generation network.

18. The computer-readable storage medium according to claim 15 , wherein the grasping position quality evaluation network is a network trained based on a convolutional neural network model, and the grasping position quality evaluation network is trained by:

acquiring a Depth image containing the target object;

zooming the Depth image to a third predetermined resolution to obtain a third training set;

randomly acquiring a pair of grasping positions on the Depth image in the third training set, and calculating a corresponding score by a predetermined scoring algorithm; and

taking the Depth image, the grasping positions and the score corresponding to the grasping positions as inputs, and training the inputs based on the convolutional neural network model to obtain the grasping position quality evaluation network.