Image processing device, image processing method, and image processing program for detecting specific region from image captured by endoscope designated as detection target image in response to determining that operator's action in not predetermined action
An endoscope system includes an endoscope and an image processing device attached to one another. The image processing device includes at least one processor configured to perform operations of determining an operator's action based on an action signal from an endoscope inserted into a subject body, deciding whether an image is set as a detection target image based on the operator's action and detecting a specific region from the detection target image. The processor performs an operation of determining whether the operator's action at a time of capturing the image is a treatment action to give the subject body a treatment. Furthermore, the processor detects, from the image, a region, which exhibits a specular reflection and whose time change in area and position is large, as a washed region and then determine the operator's action at the time of capturing the image is the treatment action when the washed region is detected.
1. An image processing device comprising:
at least one processor configured to:
determine whether an operator's action in operating an endoscope inserted into a subject body is a predetermined action based on an image captured by the endoscope inserted into the subject body;
in response to determining that the operator's action is not the predetermined action, designate the image captured by the endoscope inserted into the subject body as a detection target image; and
detect a specific region from the detection target image,
wherein the predetermined action is a treatment action to give the subject body a treatment, and
wherein in determining whether the operator's action is the treatment action, the processor is configured to:
detect, from the image, a region, which exhibits a specular reflection and whose time change in area and position is larger than a threshold value, as a washed region; and
in response to detecting the washed region, determine that the operator's action is the treatment action.
2. The image processing device according to claim 1 ,
wherein the predetermined action is a differential action to differentiate any portion in the image.
3. The image processing device according to claim 2 ,
wherein the at least one processor is configured to, in determining whether the operator's action is the predetermined action:
determine whether the image is a dyed image obtained by capturing a dyed subject; and
in response to determining that the image is not the dyed image, determine that the operator's action is not the differential action.
4. The image processing device according to claim 2 ,
wherein the at least one processor is configured to, in determining whether the operator's action is the predetermined action:
determine whether the image is a special-light image captured using special light; and
in response to determining that the image is not the special-light image, determine that the operator's action is not the differential action.
5. The image processing device according to claim 2 ,
wherein the at least one processor is configured to, in determining whether the operator's action is the predetermined action:
determine whether the image is a front-view image obtained by capturing a subject from a front view; and
in response to determining that the image is not the front-view image, determine that the operator's action is not the differential action.
6. The image processing device according to claim 1 ,
wherein the predetermined action is an attentive action to pay close attention to a subject.
7. The image processing device according to claim 6 ,
wherein the at least one processor is configured to, in determining whether the operator's action is the predetermined action:
determine, from the image, whether a tube-shaped region having a color different from that of a mucosal region, and a bleeding region is detected; and
in response to determining that the tube-shaped region is not detected, determine that the operator's action is not the attentive action.
8. The image processing device according to claim 6 ,
wherein the at least one processor is configured to, in determining whether the operator's action is the predetermined action:
determine, whether the image is a front-view image obtained by capturing a subject from a front view; and
in response to determining that the image is not the front-view image, determine that the operator's action is not the attentive action.
9. The image processing device according to claim 6 ,
wherein the at least one processor is configured to, in determining whether the operator's action is the predetermined action:
determine, whether the image is a close-up image to capture a region whose distance from a distal end of the endoscope is less than or equal to a predetermined value; and
in response to determining that the image is not the close-up image, determine that the operator's action is not the attentive action.
10. The image processing device according to claim 6 ,
wherein the at least one processor is configured to, in determining whether the operator's action is the predetermined action:
determine, whether the image is an image with less movement, in which an amount representing movement of a structure corresponding to that of an image preceding by a predetermined number of frames is less than or equal to a threshold value, and the image with less movement is captured continuously during a certain fixed period or more prior to the image; and
in response to determining that the image is not the image with less movement, determine that the operator's action is not the attentive action.
11. The image processing device according to claim 6 ,
wherein the at least one processor is configured to, in determining whether the operator's action is the predetermined action:
determine, from the image, whether an endoscope region in which part of the endoscope comes out is detected; and
in response to determining that the endoscope region is not detected, determine that the operator's action is not the attentive action.
12. The image processing device according to claim 1 ,
wherein the predetermined action is an insertion action to insert the endoscope into the subject body.
13. The image processing device according to claim 12 ,
wherein the at least one processor is configured to, in determining whether the operator's action is the insertion action:
determine, from the image, whether the endoscope inserted from an anus of the subject body is moving in a forward path from a large intestine toward an appendix of the subject body, is detected; and
in response to determining that the endoscope is not moving in the forward path, determine that the operator's action is not the insertion action.
14. The image processing device according to claim 12 ,
wherein the at least one processor is configured to, in determining whether the operator's action is the insertion action:
determine, from the image, in which direction the endoscope is moving, to a distal end or a base end of the endoscope, and
in response to determining that the endoscope is moving to the base end, determine that the operator's action is not the insertion action.
15. An image processing device comprising:
at least one processor configured to:
determine whether an operator's action in operating an endoscope inserted into a subject body is a predetermined action based on an image captured by the endoscope inserted into the subject body;
in response to determining that the operator's action is not the predetermined action, designate the image captured by the endoscope inserted into the subject body as a detection target image;
classify the detection target image based on predetermined conditions;
set a detection target region in the detection target image according to a result of classifying the detection target image; and
detect a specific region from the detection target region in the detection target image set.
16. The image processing device according to claim 15 ,
wherein the predetermined conditions include whether contrast of frequency components in the detection target image falls within a range of tolerance levels, and
wherein the at least one processor is configured to set a region in the detection target image having the contrast of frequency components that falls out of the range of tolerance levels as the detection target region.
17. The image processing device according to claim 15 ,
wherein the at least one processor is configured to:
detect a non-mucosal region from the detection target image;
set a region in the detection target image other than the non-mucosal region in the detection target image as a detection target region; and
detect the specific region from the detection target region in the detection target image set.
18. The image processing device according to claim 15 ,
wherein the at least one processor is configured to:
classify a change in the detection target image as being large or small according to a magnitude of a change in the detection target image from an image preceding by a predetermined number of frames;
in response to classifying the change in the detection target image as being large, set a whole of the detection target image as the detection target region; and
in response to classifying the change in the detection target image as being small, set a new region appearing in the detection target image without being present in the image preceding by the predetermined number of frames as the detection target region.
19. The image processing device according to claim 15 ,
wherein the at least one processor is configured to:
set a detection parameter used to detect the specific region from the detection target region according to the result of classifying the detection target image.
20. The image processing device according to claim 15 ,
wherein the predetermined action is at least one of a treatment action, a differential action, an attentive action and insertion action.
21. The image processing device according to claim 15 ,
wherein the predetermined action is at least one of:
a treatment action that gives a medical treatment to the subject body;
a differential action that differentiates any portion in the image;
an attentive action that pays close attention to the subject body; and
an insertion action that inserts the endoscope into the subject body.
22. The image processing device according to claim 21 ,
wherein the predetermined action is the treatment action.
23. The image processing device according to claim 22 ,
wherein the at least one processor is configured to determine that the operator's action is the treatment action when a treatment tool region is detected in the image.
24. The image processing device according to claim 22 ,
wherein the at least one processor is configured to determine that the operator's action is the treatment action when a washed region is detected in the image.
25. The image processing device according to claim 15 ,
wherein the at least one processor is configured to:
in response to determining that the operator's action is the predetermined action, not designate the image as the detection target image.
26. The image processing device according to claim 15 ,
wherein the at least one processor is configured to:
receive information input by a user; and
set a type of the predetermined action based on the information input by the user.