Control device, control system, robot system, and control method
A control device includes: first circuitry that generates a command to cause a robot to autonomously grind a grinding target portion; second circuitry that generates a command to cause the robot to grind a grinding target portion according to manipulation information from an operation device; third circuitry that controls operation of the robot according to the command; storage that stores image data of a grinding target portion and operation data of the robot corresponding to the command; and forth circuitry that performs machine learning by using image data of a grinding target portion and the operation data for the grinding target portion, receives the image data as input data, and outputs an operation correspondence command corresponding to the operation data as output data. The first circuitry generates the command, based on the operation correspondence command.
1. A control device for a robot that performs grinding work by using a grinder, the control device comprising:
autonomous command generation circuitry that generates an autonomous operation command for causing the robot to autonomously grind a grinding target portion;
manual command generation circuitry that generates a manual operation command for causing the robot to grind a grinding target portion according to manipulation information outputted from an user interface for operating the robot;
operation control circuitry that controls operation of the grinding work of the robot according to the operation command;
storage that stores image data of a grinding target portion and operation data of the robot corresponding to the operation command generated for the grinding target portion in a state of the image data; and
learning circuitry that performs machine learning by using image information that is information regarding image data of a grinding target portion and the operation data of the robot for the grinding target portion in a state of the image information, receives the image information of a grinding target portion as input data, and outputs an operation correspondence command corresponding to the operation data of the robot for the grinding target portion in a state of the image information as output data, wherein;
the operation correspondence command is a command including:
a force command that commands magnitude and a direction of force that is applied to a grinding target portion from the grinder by the robot, and
a position command that commands a three-dimensional position and three-dimensional posture of the grinder which are realized by the robot, and
the autonomous command generation circuitry generates the autonomous operation command, based on the operation correspondence command of the learning circuitry.
2. The control device according to claim 1 , wherein:
the learning circuitry performs machine learning by using the image information of a grinding target portion and the operation data of the robot corresponding to the manual operation command generated for the grinding target portion in a state of the image information.
3. The control device according to claim 1 , wherein;
the learning circuitry performs machine learning by using the image information of a grinding target portion and the operation data of the robot corresponding to the autonomous operation command and the manual operation command that are generated for the grinding target portion in a state of the image information.
4. The control device according to claim 1 , wherein:
during operation of the robot according to the autonomous operation command, the manual command generation circuitry receives input of correction manipulation information that is the manipulation information for correcting the operation of the robot, via the user interface, and generates a correction manual operation command that is the manual operation command, according to the correction manipulation information, and
the learning circuitry performs machine learning by using the image information of a grinding target portion and the operation data of the robot corresponding to the autonomous operation command and the manual operation command that are generated for the grinding target portion in a state of the image information.
5. The control device according to claim 1 , wherein:
the autonomous command generation circuitry generates:
a first autonomous operation command that is the autonomous operation command for first grinding work in which the robot autonomously grinds a grinding target portion according to a predetermined operation procedure, and
a second autonomous operation command that is the autonomous operation command for second grinding work in which the robot autonomously grinds the grinding target portion, based on the operation correspondence command of the learning circuitry after the first grinding work, and
the autonomous command generation circuitry generates the second autonomous operation command, based on the operation correspondence command outputted by the learning circuitry with the image information of the grinding target portion as input data.
6. The control device according to claim 1 , wherein;
the autonomous command generation circuitry generates a third autonomous operation command that is the autonomous operation command for third grinding work in which an unground grinding target portion is autonomously ground, and
the autonomous command generation circuitry generates the third autonomous operation command, based on the operation correspondence command outputted by the learning circuitry with the image information of the grinding target portion as input data.
7. The control device according to claim 1 , wherein the learning circuitry performs machine learning and input/output by using a learning model including a neural network.
8. The control device according to claim 1 , wherein the learning circuitry uses data representing three-dimensional positions of various portions on a grinding target portion, as the image information.
9. The control device according to claim 8 , further comprising:
first image processing circuitry that detects three-dimensional positions of various portions on a grinding target portion by processing an image of the grinding target portion, and generates image data representing the three-dimensional positions of said various portions.
10. The control device according to claim 1 , further comprising:
first image processing circuitry that, based on at least either one of a shade or a hue represented in an image of a grinding target portion taken by using an imager including a non-reflective light source and a camera, detects a distance from the camera to various positions on the grinding target portion, and generates image data representing the distance to said various positions.
11. The control device according to claim 1 , further comprising:
imaging control circuitry that controls operation of an imager that takes an image of a grinding target portion, and acquires image data of the grinding target portion,
wherein:
the imaging control circuitry causes the imager to take an image of a grinding target portion at a timing before grinding work on the grinding target portion and a predetermined timing during the grinding work, and
the learning circuitry outputs the operation correspondence command with the image information regarding the image data acquired by the imaging control circuitry as input data.
12. The control device according to claim 11 , wherein:
the operation control circuitry causes the robot to grind all grinding target portions of a grinding target group including the grinding target portions according to the autonomous operation command,
the imaging control circuitry causes the imager to take images of said all grinding target portions after grinding work according to the autonomous operation command on said all grinding target portions is completed, and
the control device causes a display to present the images of said all grinding target portions.
13. The control device according to claim 11 , further comprising:
second image processing circuitry,
wherein the second image processing circuitry processes image data of a grinding target portion taken by the imager, generates image data representing a three-dimensional shape of the grinding target portion by using at least either one of a color, a pattern, or a shade, and causes the display to present the image data.
14. The control device according to claim 1 , further comprising:
movement control circuitry that controls operation of a moving device that moves the robot,
wherein the movement control circuitry causes the moving device to move the robot such that the robot grinds grinding target sections into which grinding target portions are divided, in a predetermined order.
15. The control device according to claim 1 , wherein the control device performs to:
receive, from a force sensor that detects a grinding reaction force that is a reaction force received by the grinder, information on the grinding reaction force; and
cause the user interface to generate, as a manipulation reaction force that is a reaction force with respect to a manipulation inputted to the user interface, the manipulation reaction force corresponding to the grinding reaction force.
16. The control device according to claim 15 , wherein the control device causes the user interface to generate the manipulation reaction force corresponding to the grinding reaction force acquired via a filter that reduces a high-frequency component of the grinding reaction force detected by the force sensor.
17. The control device according to claim 1 , wherein;
the manipulation information includes commands of a position, a posture, and a force inputted to the user interface, and
the manual command generation circuitry generates the manual operation command including commands of a position, a posture, and an action force of the grinder that are increased from the commands of the position, the posture, and the force included in the manipulation information.
18. The control device according to claim 1 , wherein;
the control device performs to:
determine a first coordinate system relationship that is a relationship between a robot coordinate system set for the robot and an object coordinate system set for the grinding target portion; and
determine a second coordinate system relationship that is a relationship between a manipulation coordinate system set for the user interface and the object coordinate system, and
the manual command generation circuitry generates the manual operation command according to the manipulation information, the first coordinate system relationship, and the second coordinate system relationship, and
when the object coordinate system is moved, the control device newly determines the first coordinate system relationship after movement, based on the object coordinate system after movement and the robot coordinate system, and determines a relationship that is the same as the second coordinate system relationship before movement, as the second coordinate system relationship after movement between the object coordinate system after movement and the manipulation coordinate system.
19. A control system comprising:
the control device according to claim 1 ; and
the user interface for operating the robot.
20. A robot system comprising:
the control device according to claim 1 ;
the robot;
the user interface for operating the robot; and
an imager,
wherein the imager outputs image data obtained by taking an image of a grinding target portion, to the control device.
21. The robot system according to claim 20 , further comprising a display,
wherein the display presents at least either an image taken by the imager or an image outputted from the control device, to a user of the robot system.
22. A control method comprising:
causing a learning model to receive image data of a grinding target portion as input data and to output an operation correspondence command for a robot as output data;
generating an autonomous operation command for causing the robot to autonomously grind the grinding target portion, based on the operation correspondence command;
generating a manual operation command for causing the robot to grind the grinding target portion according to manipulation information outputted from an user interface for operating the robot;
controlling operation of grinding work of the robot according to the operation command; and
causing the learning model to perform machine learning by using the image data of the grinding target portion and operation data of the robot corresponding to the operation command generated for the grinding target portion in a state of the image data,
wherein:
the operation correspondence command outputted by the learning model is a command corresponding to the operation data of the robot; and
the operation correspondence command outputted by the learning model is a command including:
a force command that commands magnitude and a direction of force that is applied to a grinding target portion from the grinder by the robot and
a position command that commands a three-dimensional position and three-dimensional posture of the grinder which are realized by the robot.