Robot system, control method, image processing apparatus, image processing method, method of manufacturing products, and recording medium
A robot system includes a robot, an image capture apparatus, an image processing portion, and a control portion. The image processing portion is configured to specify in an image of a plurality of objects captured by the image capture apparatus, at least one area in which a predetermined object having a predetermined posture exists, and obtain information on position and/or posture of the predetermined object in the area. The control portion is configured to control the robot, based on the information on position and/or posture of the predetermined object, for the robot to hold the predetermined object.
1. A robot system comprising:
a robot;
an image capture apparatus;
an image processing portion configured to specify, in an image of a plurality of objects captured by the image capture apparatus, at least one area in which a predetermined object having a predetermined posture exists, and obtain information on position and/or posture of the predetermined object in the at least one area; and
a control portion configured to control the robot, based on the information on position and/or posture of the predetermined object, for the robot to hold the predetermined object,
wherein the image processing portion is configured to perform:
a first capturing process for capturing the plurality of objects and specifying the at least one area in which the predetermined object having the predetermined posture exists, and
a second capturing process to obtain information about the position and/or posture of the predetermined object in the at least one area by capturing a capturing area smaller than a capturing area captured in the first capturing process.
2. The robot system according to claim 1 , wherein the image processing portion is configured to determine a posture of a model, based on the predetermined posture, the model being used for obtaining the information on position and/or posture of the predetermined object and configured to indicate a shape of the predetermined object.
3. The robot system according to claim 1 , wherein the image processing portion is configured to specify the at least one area by using a learned model on which machine learning has been performed.
4. The robot system according to claim 1 , wherein the predetermined posture is a first posture in which an image of a first surface of the predetermined object is captured, or a second posture in which an image of a second surface of the predetermined object different from the first surface is captured.
5. The robot system according to claim 3 , wherein the image processing portion is configured to obtain the learned model by using an image obtained by capturing a first surface of an object and/or an image obtained by capturing a second surface of an object different from the first surface of the object.
6. The robot system according to claim 1 , wherein the image processing portion is configured to display the at least one area on a display device.
7. The robot system according to claim 1 , wherein the image processing portion is configured to display information on a state of the predetermined object on a display device.
8. The robot system according to claim 1 , wherein the at least one area is two or more areas, and
wherein the image processing portion is configured to specify the two or more areas and obtain priorities for predetermined objects corresponding to the two or more areas.
9. The robot system according to claim 8 , wherein the image processing portion is configured to display the priorities on a display device.
10. The robot system according to claim 8 , wherein the image processing portion is configured to obtain the priorities for the predetermined objects corresponding to the two or more areas by using a plurality of factors.
11. The robot system according to claim 10 , wherein one of the plurality of factors is a success rate obtained when the robot tried to hold a workpiece in past.
12. The robot system according to claim 10 , wherein the plurality of factors includes at least one of an exposure degree of the predetermined objects, a scattering degree of the predetermined objects, and a height of the predetermined objects stacked in bulk.
13. The robot system according to claim 1 , wherein the image processing portion is configured to obtain the position and/or the posture of the predetermined object by performing pattern matching on the at least one area.
14. The robot system according to claim 13 , wherein the image processing portion is configured to perform the pattern matching on the at least one area and not to perform the pattern matching on another area other than the at least one area.
15. The robot system according to claim 1 , wherein the control portion is configured to cause the robot to hold the predetermined object at first that corresponds to the at least one area and that is of the plurality of objects.
16. The robot system according to claim 1 , wherein the image processing portion is configured to obtain the information on position and/or posture of the predetermined object by performing pattern matching on an image of the predetermined object captured by the second image-capture unit.
17. The robot system according to claim 1 , wherein the second image-capture unit is disposed on the robot.
18. The robot system according to claim 1 , wherein the image processing portion is configured to specify the at least one area based on a contour of the predetermined object.
19. The robot system according to claim 18 , wherein a plurality of hold positions is set in the predetermined object for the robot to hold the predetermined object, and
wherein the image processing portion is configured to obtain priorities for the plurality of hold positions based on a state of the predetermined object.
20. The robot system according to claim 19 , wherein the image processing portion is configured to obtain the priorities for the plurality of hold positions based on at least one of a height of each of the plurality of hold positions of the predetermined object stacked in bulk, a center of gravity of the predetermined object, and a time necessary for the robot to approach each of the plurality of hold positions of the predetermined object.
21. The robot system according to claim 1 , wherein a plurality of hold positions is set in the predetermined object for the robot to hold the object, and
wherein the image processing portion is configured to specify a hold position of the plurality of hold positions that causes the robot to interfere with another object other than the predetermined object when the robot holds the predetermined object.
22. The robot system according to claim 21 , wherein the image processing portion is configured to determine whether the robot will interfere with the other object other than the predetermined object based on a height of the predetermined object, a height of the other object other than the predetermined object, and/or a contour of the predetermined object and a contour of the other object other than the predetermined object.
23. A control method of a robot system that includes a robot and an image capture apparatus, the method comprising:
specifying, by an image processing portion, in an image of a plurality of objects captured by the image capture apparatus, at least one area in which a predetermined object having a predetermined posture exists, and obtaining, by the image processing portion, information on position and/or posture of the predetermined object in the at least one area; and
controlling, by a control portion, the robot based on the information on position and/or posture of the object, for the robot to hold the predetermined object,
wherein the image processing portion performs:
a first capturing process of capturing the plurality of objects and specifying the at least one area in which the predetermined object having the predetermined posture exists, and
a second capturing process of obtaining information about the position and/or posture of the predetermined object in the at least one area by capturing a capturing area smaller than a capturing area captured in the first capturing process.
24. An image processing apparatus comprising:
an image processing portion configured to specify in a captured image of a plurality of objects, at least one area in which a predetermined object having a predetermined posture exists, and obtain information on position and/or posture of the predetermined object in the at least one area,
the image processing portion is configured to perform:
a first capturing process for capturing the plurality of objects and specifying the at least one area in which the predetermined object having the predetermined posture exists, and
a second capturing process to obtain information about the position and/or posture of the predetermined object in the at least one area by capturing a capturing area smaller than a capturing area captured in the first capturing process.
25. An image processing method comprising:
specifying, by an image processing portion, in a captured image of a plurality of objects, at least one area in which a predetermined object having a predetermined posture exists; and
obtaining, by the image processing portion, information on position and/or posture of the predetermined object in the area,
wherein the image processing portion performs:
a first capturing process for capturing the plurality of objects and specifying the at least one area in which the predetermined object having the predetermined posture exists, and
a second capturing process to obtain information about the position and/or posture of the predetermined object in the at least one area by capturing a capturing area smaller than a capturing area captured in the first capturing process.
26. A method of manufacturing products by using the robot system according to claim 1 .
27. A computer-readable non-transitory recording medium storing a program that causes a computer to perform the control method according to claim 23 .
28. A computer-readable non-transitory recording medium storing a program that causes a computer to perform the image processing method according to claim 25 .