Robotic wire insertion using connector segmentation

1 . A robotic system comprising:

an end effector;

a camera system connected to the end effector; and

a controller, wherein the controller is configured to:

generate an image of a connector using the camera system with the end effector at a connector pose facing a surface of the connector, the surface having holes for receiving wires, wherein the connector pose orients the end effector and the camera system such that the surface is perpendicular to an axis from the end effector and the camera system;

select a region in the image encompassing the connector;

create an initial mask using the image and the region, wherein the initial mask comprises a background region and a connector region;

create a segmentation mask using the initial mask and the image, wherein the segmentation mask includes edges for the connector; and

apply thresholding to the segmentation mask to recover pixels for the connector and generate a final mask;

wherein the controller is further configured to insert a set of wires into the connector using the final mask.

2 . The robotic system of claim 1 , wherein in inserting the set of wires, the controller is configured to:

position the end effector holding a wire at the connector pose; and

move the end effector at the connector pose to insert the wire into the connector using the final mask.

3 . The robotic system of claim 2 , wherein the controller is further configured to:

pick up a second wire using the end effector;

move the end effector back to the connector pose; and

move the end effector at the connector pose to insert the second wire into the connector using the final mask.

4 . The robotic system of claim 1 , wherein in creating the initial mask, the controller is configured to:

create the background region as a first portion of the image outside of the region; and

create the connector region as a second portion of the image defined by the region.

5 . The robotic system of claim 1 , wherein in creating the initial mask, the controller is further configured to:

create an inner connector region that is inside of the connector region, wherein the inner connector region is selected as having a higher probability of encompassing the connector than the connector region.

6 . The robotic system of claim 1 , wherein the connector region is defined by the region and by an inner connector region that is an inner portion of the connector region.

7 . The robotic system of claim 1 , wherein in selecting the region, the controller is configured to:

display the image on display system; and

receive a user input from an input system, wherein the user input comprises a selection of the region.

8 . The robotic system of claim 1 further comprising:

a robot, wherein the robot comprises the end effector, the camera system is connected to the end effector, and the end effector is connected to a platform.

9 . A method for inserting a wire, the method comprising:

generating an image of a connector using a camera system connected to an end effector, wherein the end effector is at a connector pose facing a surface of the connector, the surface having holes for receiving wires, wherein the connector pose orients the end effector and the camera system such that the surface is perpendicular to an axis from the end effector and the camera system;

selecting a region in the image encompassing the connector;

creating an initial mask using the image and the region, wherein the initial mask comprises a background region and a connector region;

creating a segmentation mask using the initial mask and the image, wherein the segmentation mask includes edges for the connector;

applying thresholding to the segmentation mask to recover pixels for the connector and generate a final mask, wherein the final mask is used to insert the wire into the connector; and

inserting a set of wires into the connector using the final mask.

10 . The method of claim 9 , wherein said inserting the set of wires comprises:

positioning the end effector holding a wire at the connector pose; and

moving the end effector at the connector pose to insert the wire into the connector using the final mask.

11 . The method of claim 10 further comprising:

picking up a second wire using the end effector;

moving the end effector back to the connector pose; and

moving the end effector at the connector pose to insert the second wire into the connector using the final mask.

12 . The method of claim 9 , wherein said creating the initial mask comprises:

creating the background region as a first portion of the image outside of the region; and

create the connector region as a second portion of the image defined by the region.

13 . The method of claim 9 , wherein said creating the initial mask comprises:

creating an inner connector region that is inside of the connector region, wherein the inner connector region is selected as having a higher probability of encompassing the connector than the connector region.

14 . The method of claim 9 , wherein the connector region is defined by the region and by an inner connector region that is an inner portion of the connector region.

15 . The method of claim 9 , wherein said selecting the region comprises:

displaying the image on a display system; and

receiving a user input from an input system, wherein the user input comprises a selection of the region.

16 . The method of claim 9 , wherein a robot comprises the end effector, the camera system is connected to the end effector, and the end effector is connected to a platform.

17 . A wire assembly system comprising:

a computer system; and

a controller in the computer system, wherein the controller is configured to:

generate an image of a connector using a camera system connected to an end effector at a connector pose facing a surface of the connector, the surface having holes for receiving wires, wherein the connector pose orients the end effector and the camera system such that the surface is perpendicular to an axis from the end effector and the camera system;

select a region in the image encompassing the connector;

create an initial mask using the image and the region, wherein the initial mask comprises a background region and a connector region;

create a segmentation mask using the initial mask and the image, wherein the segmentation mask includes edges for the connector;

apply thresholding to the segmentation mask to recover pixels for the connector and generate a final mask; and

control movement of the end effector to insert a wire into the connector using the final mask.

18 . A computer program product for wire assembly, the computer program product comprising a computer-readable storage media having program instructions embodied therewith, the program instructions executable by a computer system to cause the computer system to:

generate an image of a connector using a camera system connected to an end effector at a connector pose facing a surface of the connector, the surface having holes for receiving wires, wherein the connector pose orients the end effector and the camera system such that the surface is perpendicular to an axis from the end effector and the camera system;

select a region in the image encompassing the connector;

create an initial mask using the image and the region, wherein the initial mask comprises a background region and a connector region;

create a segmentation mask using the initial mask and the image, wherein the segmentation mask includes edges for the connector;

apply thresholding to the segmentation mask to recover pixels for the connector and generate a final mask; and

control movement of the end effector to insert a wire into the connector using the final mask.

19 . The wire assembly system of claim 17 , wherein in inserting the wire, the controller is configured to:

position the end effector holding the wire at the connector pose; and

move the end effector at the connector pose to insert the wire into the connector using the final mask.

20 . The wire assembly system of claim 19 , wherein the controller is further configured to:

pick up a second wire using the end effector;

move the end effector back to the connector pose; and

move the end effector at the connector pose to insert the second wire into the connector using the final mask.