Endoscopic image learning device, endoscopic image learning method, endoscopic image learning program, and endoscopic image recognition device
An object is to provide an endoscopic image learning device, an endoscopic image learning method, an endoscopic image learning program, and an endoscopic image recognition device that appropriately learn a learning model for image recognition for recognizing an endoscopic image in which a treatment tool for an endoscope appears. The object is achieved by an endoscopic image learning device including an image generation unit and a machine learning unit. The image generation unit generates a superimposed image where a foreground image in which a treatment tool for an endoscope is extracted is superimposed on a background-endoscopic image serving as a background of the foreground image, and the machine learning unit performs the learning of a learning model for image recognition using the superimposed image.
1. A method of producing a learned model, the method comprising:
performing generation of a plurality of superimposed images in which first images are superimposed respectively on second images, each of the first images includes an image of an endoscopic treatment tool, and each of the second images includes an image of a living body; and
performing a machine learning process to produce the learned model through utilizing the plurality of superimposed images,
wherein the learned model is configured to, in response to an input of an endoscopic image, output a recognition result including at least one of followings: presentation of the endoscopic treatment tool in the endoscopic image, a type of the endoscopic treatment tool in the endoscopic image, and an area of the endoscopic treatment tool in the endoscopic image.
2. The method according to claim 1 , wherein the generation of the plurality of superimposed images includes generating the superimposed images by superimposing the first images respectively on the second images while moving the image of the endoscopic treatment tool to different positions in the first images.
3. The method according to claim 1 , wherein the machine learning process is performed through utilizing the superimposed images in which positions of the image of the endoscopic treatment tool are different from each other in the first images.
4. The method according to claim 1 , wherein the generation of the plurality of superimposed images includes generating the superimposed images by superimposing at least one of the first images respectively on the second images while changing combinations of the at least one of the first images and the second images different from each other.
5. The method according to claim 1 , wherein the machine learning process is performed through utilizing the superimposed images in which the first images are respectively processed by different transformations.
6. The method according to claim 5 , wherein the transformations include at least one of an affine transformation, a color conversion, and a noise application.
7. The method according to claim 1 , wherein:
the machine learning process is performed through utilizing the superimposed images in which the first images respectively including images of endoscopic treatment tools of types different from each other; and
the learned model is configured to recognize the endoscopic treatment tools of the types different from each other.
8. The method according to claim 7 , wherein each of the second images includes the image of the living body captured through an endoscope.
9. The method according to claim 8 , wherein the second images do not include the images of the endoscopic treatment tools.
10. The method according to claim 8 , wherein the endoscope includes:
an insertion part having a bendable part;
a distal end part provided on a distal end of the insertion part; and
an imager provided on the distal end.
11. The method according to claim 7 , wherein each of the first images is captured while the endoscopic treatment tool is used with an endoscope.
12. The method according to claim 7 , wherein the machine learning process utilizes a convolution neural network.
13. An endoscopic image learning method comprising:
performing generation of a plurality of superimposed images in which first images are superimposed respectively on second images, each of the first images includes an image of an endoscopic treatment tool, and each of the second images includes an image of a living body; and
performing a machine learning process to produce a learned model for image recognition through utilizing the superimposed images,
wherein the learned model is configured to distinguish an area of the endoscopic treatment tool and an area other than the area of the endoscopic treatment tool.