Image processing device, work robot, substrate inspection device, and specimen inspection device

1. An image processing device, comprising:

a storage section storing a three-dimensional shape model in which feature amounts and three-dimensional positional information for multiple feature points of a target object are associated;

an extraction process section configured to extract the feature amounts and two-dimensional positional information of the feature points from a two-dimensional image of the target object captured with a camera; and

a recognition process section configured to identify three-dimensional positional information of the feature points for the two-dimensional image and recognize position and orientation of the target object by

matching the feature points of the two-dimensional image with the feature points of the three-dimensional shape model using the feature amounts,

setting approximate positions of the feature points by the matching,

projecting an image of the three-dimensional shape model on the two-dimensional image based on the set approximate positions,

adjusting the approximate positions so as to increase a degree of overlap between the projected image and the two-dimensional image based on a brightness difference between a pixel of interest and an overlapping pixel, and

identifying the three-dimensional positional information of the feature points at a position of maximum overlap between the projected image and the two-dimensional image, and recognizing the position and orientation of the target object.

2. The image processing device of claim 1 , wherein

the extraction process section is provided with a creation process device configured to acquire multiple two-dimensional images of the target object captured with the camera at multiple viewpoints in a state in which the target object to be a reference is disposed at a predetermined position, extract feature amounts and two-dimensional positional information of the feature points from multiple two-dimensional images, create three-dimensional positional information by obtaining height information of the feature points from parallax between the two-dimensional images, create the three-dimensional shape model in which the feature amounts and the three-dimensional positional information are associated with the feature points, and store the three-dimensional shape model in the storage section.

3. The image processing device of claim 2 , wherein

the extraction process device acquires images, which are to become the two-dimensional images, captured by subsequently moving the camera from multiple viewpoints with respect to the predetermined position while the target object to be a reference is disposed at the predetermined position; and

the extraction process device comprises a notification section configured to send out a warning when the amount of positional deviation of the target object from the predetermined position is equal to or greater than a predetermined allowable amount.

4. The image processing device of claim 2 , wherein

the creation process section counts, for each of the feature points extracted from the multiple two-dimensional images, the number of times the feature points appear in the multiple two-dimensional images and creates the three-dimensional shape model using the feature points whose counted number of appearances are equal to or greater than a predetermined number.

5. The image processing device of claim 1 , wherein the extraction process section extracts the feature amounts by predetermined machine learning.

6. A work robot, comprising the image processing device of claim 1 and a camera configured to capture the two-dimensional image of the workpiece as the target object; wherein the work robot picks up the workpiece based on the workpiece recognition result on the workpiece recognized by the image processing device from the two-dimensional image captured with the camera and performs a predetermined operation.

7. A board inspection device, comprising: the image processing device of claim 1 and a camera configured to capture, as the target object, a two-dimensional image of a resulting object provided on a board by a predetermined operation on the board; wherein the board inspection device performs inspection of the board based on the recognition result of the resulting object recognized by the image processing device from the two-dimensional image captured with the camera.

8. A specimen inspection device comprising the image processing device of claim 1 , a microscope device configured to enlarge an image of a specimen which is the target object, a camera configured to capture a two-dimensional image of the specimen enlarged by the microscope device; and the specimen inspection device inspects the specimen based on the recognition result of the specimen recognized by the image processing device from the two-dimensional image captured with the camera.

9. An image processing device, comprising:

a storage section storing a three-dimensional shape model in which feature amounts and three-dimensional positional information for multiple feature points of a target object are associated;

an extraction process section configured to extract the feature amounts and two-dimensional positional information of the feature points from a two-dimensional image of the target object captured with a camera; and

a recognition process section configured to identify three-dimensional positional information of the feature points for the two-dimensional image and recognize position and orientation of the target object by matching the feature points of the two-dimensional image with the feature points of the three-dimensional shape model using the feature amounts,

wherein the extraction process section is provided with a creation process device configured to acquire multiple two-dimensional images of the target object captured with the camera at multiple viewpoints in a state in which the target object to be a reference is disposed at a predetermined position, extract feature amounts and two-dimensional positional information of the feature points from multiple two-dimensional images, create three-dimensional positional information by obtaining height information of the feature points from parallax between the two-dimensional images, create the three-dimensional shape model in which the feature amounts and the three-dimensional positional information are associated with the feature points, and store the three-dimensional shape model in the storage section,

wherein the extraction process device acquires images, which are to become the two-dimensional images, captured by subsequently moving the camera from multiple viewpoints with respect to the predetermined position while the target object to be a reference is disposed at the predetermined position, and

wherein the extraction process device comprises a notification section configured to send out a warning when the amount of positional deviation of the target object from the predetermined position is equal to or greater than a predetermined allowable amount.