Workpiece photographed image processing device, workpiece photographed image processing method, and processing system
A camera is located at a position deviated from directly above a center of a placement location of a workpiece. When a photographed image is generated by photographing, with the camera, the placement location at which the workpiece is placed, an image processing device specifies, by using a preliminarily determined conversion parameter, to which pixels in the photographed image two-dimensional coordinates within a photographed three-dimensional space correspond, aligns the specified pixels corresponding to the respective coordinates in a two-dimensional manner, and generates a corrected photographed image in which a trapezoidal distortion in the photographed image is corrected. A robot control device detects a position of the workpiece at the placement location based on the corrected photographed image, and controls a workpiece retention robot so as to retain the workpiece and transport the workpiece to a processing machine.
1. A workpiece photographed image processing device, comprising:
a first conversion parameter calculation unit configured to calculate, based on a first photographed image obtained by photographing a lattice pattern with a camera located at a position deviated from directly above a center of a placement location of a workpiece, a first conversion parameter for correcting an image distortion due to a lens of the camera;
a lens distortion correction unit configured to correct, by using the first conversion parameter, the image distortion due to the lens included in a second photographed image obtained by photographing a jig board with the camera;
a second conversion parameter calculation unit configured to extract a plurality of first reference points of the jig board within the second photographed image in which the image distortion is corrected by the lens distortion correction unit, and to calculate, based on actual first reference points of the jig board and the extracted plurality of first reference points within the second photographed image, a second parameter for converting two-dimensional coordinates within a three-dimensional space photographed by the camera into positions of pixels within the second photographed image;
a third conversion parameter calculation unit configured to extract, by using the second parameter, a region corresponding to an actual predetermined region of the jig board within the second photographed image in which the image distortion is corrected by the lens distortion correction unit, to extract a plurality of second reference points based on an image of the extracted region within the second photographed image, to add, by using the first conversion parameter, the image distortion due to the lens to the extracted plurality of second reference points, and to calculate, based on actual second reference points of the jig board and the extracted plurality of second reference points within the second photographed image to which the image distortion due to the lens is added, a third conversion parameter for converting the two-dimensional coordinates within the three-dimensional space into the positions of the pixels within the second photographed image; and
a trapezoidal distortion correction unit configured to specify, by using the third conversion parameter, to which pixels in a third photographed image the two-dimensional coordinates within the three-dimensional space photographed by the camera correspond when the third photographed image is generated by photographing, with the camera, the placement location at which the workpiece is placed, to align the specified pixels corresponding to respective coordinates in a two-dimensional manner, and to generate a fourth photographed image in which a trapezoidal distortion in the third photographed image is corrected.
2. The workpiece photographed image processing device according to claim 1 , wherein
a plurality of markers are provided on the jig board, and
the second conversion parameter calculation unit is configured to extract center points of the plurality of markers as the extracted plurality of first reference points.
3. The workpiece photographed image processing device according to claim 1 , wherein
a plurality of openings are formed on the jig board,
the second photographed image includes opening edge images corresponding to the plurality of openings, and the
third conversion parameter calculation unit is configured to
extract a region including the opening edge images within the second photographed image in which the image distortion is corrected, and
extract intersections between straight lines along the opening edge images as the extracted plurality of second reference points.
4. A workpiece photographed image processing method, comprising:
calculating, based on a first photographed image obtained by photographing a lattice pattern with a camera located at a position deviated from directly above a center of a placement location of a workpiece, a first conversion parameter for correcting an image distortion due to a lens of the camera;
correcting, by using the first conversion parameter, the image distortion due to the lens included in a second photographed image obtained by photographing a jig board with the camera;
extracting a plurality of first reference points of the jig board within the second photographed image in which the image distortion is corrected;
calculating, based on actual first reference points of the jig board and the extracted plurality of first reference points within the second photographed image, a second parameter for converting two-dimensional coordinates within a three-dimensional space photographed by the camera into positions of pixels within the second photographed image;
extracting, by using the second parameter, a region corresponding to an actual predetermined region of the jig board within the second photographed image in which the image distortion is corrected;
extracting a plurality of second reference points based on an image of the extracted region within the second photographed image;
adding, by using the first conversion parameter, the image distortion due to the lens to the extracted plurality of second reference points;
calculating, based on actual second reference points of the jig board and the extracted plurality of second reference points within the second photographed image to which the image distortion due to the lens is added, a third conversion parameter for converting the two-dimensional coordinates within the three-dimensional space into the positions of the pixels within the second photographed image; and
specifying, by using the third conversion parameter, to which pixels in a third photographed image the two-dimensional coordinates within the three-dimensional space photographed by the camera correspond when the third photographed image is generated by photographing, with the camera, the placement location at which the workpiece is placed, aligning the specified pixels corresponding to respective coordinates in a two-dimensional manner, and generating a fourth photographed image in which a trapezoidal distortion in the third photographed image is corrected.
5. The workpiece photographed image processing method according to claim 4 , wherein
a plurality of markers are provided on the jig board, and
the method further comprising extracting center points of the plurality of markers as the extracted plurality of first reference points.
6. The workpiece photographed image processing method according to claim 4 , wherein
a plurality of openings are formed on the jig board,
the second photographed image includes opening edge images corresponding to the plurality of openings, and
the method further comprising:
extracting a region including the opening edge images within the second photographed image in which the image distortion is corrected; and
extracting intersections between straight lines along the opening edge images as the extracted plurality of second reference points.