Image processing apparatus, method for operating image processing apparatus, and endoscope system for indicating an imaged area and a not-yet-imaged area of an image
The present invention enables to report insufficient observation at an appropriate timing. A medical image processing apparatus according to an aspect of the present invention is a medical image processing apparatus including a memory that stores area information indicating predetermined areas to be imaged in a photographic subject, a processor, and a reporting unit. The processor is configured to acquire a medical image of the photographic subject, perform recognition of an area of the photographic subject in the medical image, compare the recognized area with an area indicated by the area information and make a determination of a not-yet-imaged area among the areas to be imaged, and perform, using the reporting unit, reporting of a result of the determination at an expected end timing at which acquisition of the medical image of the areas to be imaged is expected to have been ended.
1 . A medical image processing apparatus comprising:
a memory that stores area information indicating a plurality of areas to be imaged in a photographic subject, in advance; and
a processor, wherein
the processor is configured to
acquire a medical image of the photographic subject,
perform image recognition on the medical image using a trained model,
in a case where at least one of following conditions (a), (b), or (c):
(a) a specific subject continuously appears in the medical image for more than a predetermined time,
(b) a specific subject appears in a predetermined area of the medical image,
(c) a specific subject appears in the medical image at a size equal to or larger than a predetermined size
is met, recognize an area of the subject in the medical image as an imaged area based on result of the image recognition,
record information of the imaged area on a memory,
compare the information of the imaged area with the area information indicating the plurality of areas to be imaged, and
control to output information discriminating between the information of the imaged area and information indicating a not-yet-imaged area at an expected end timing that is automatically determined by the processor based on a change in an organ to be observed or a detection of a predetermined anatomical landmark in the medical image.
2 . The medical image processing apparatus according to claim 1 , wherein the processor is configured to store information of an area which is recognized as the imaged area on the memory, in association with the medical image.
3 . The medical image processing apparatus according to claim 1 , wherein the processor is configured to cause a display to display the information indicating the imaged area, together with the information indicating the not-yet-imaged area.
4 . The medical image processing apparatus according to claim 1 , wherein the processor is configured to cause the information indicating the imaged area and the information indicating the not-yet-imaged area to be displayed on a display screen the same as the medical image, using a schematic diagram.
5 . The medical image processing apparatus according to claim 4 , wherein the processor is configured to cause the information indicating the imaged area and the information indicating the not-yet-imaged area, to be displayed with a display mode different from each other in the schematic diagram.
6 . The medical image processing apparatus according to claim 1 , wherein the processer is configured to perform the image recognition using a neural network.
7 . The medical image processing apparatus according to claim 1 , wherein the predetermined area is an esophagogastric junction.
8 . The medical image processing apparatus according to claim 1 , wherein the predetermined area is a pharynx.
9 . The medical image processing apparatus according to claim 1 , wherein the processor is configured to make the determination at a timing at which the medical image of the predetermined area is acquired.
10 . The medical image processing apparatus according to claim 1 , wherein the processor is configured to reduce an intensity of the output, upon elapse of a predesignated time after starting the outputting.
11 . The medical image processing apparatus according to claim 1 , wherein the processor is configured to cause a display which displays of information in a screen and/or a speaker which outputs a sound, to perform the outputting.
12 . The medical image processing apparatus according to claim 11 , wherein the processor is configured to control to output by changing a display mode of the information that is already being displayed in the screen on the display and/or an output mode of the sound that is already being output from the speaker.
13 . The medical image processing apparatus according to claim 11 , wherein, in controlling the outputting, the processor causes the display to newly perform display of information that is not displayed in the screen before the outputting is performed, and/or causes the speaker to newly output a sound that is not output before the outputting starts.
14 . The medical image processing apparatus according to claim 11 , wherein the processor is configured to increase or decrease an intensity of display in the screen by the display.
15 . An endoscope system comprising:
the medical image processing apparatus according to claim 1 ; and
an endoscope configured to be inserted into a subject as the photographic subject and capture the medical image, wherein
the processor is configured to acquire the medical image captured by the endoscope.
16 . The endoscope system according to claim 15 , wherein the processor is configured to
estimate a movement direction of the endoscope, and
cause a result of the determination to be output at a timing when the estimated movement direction is changed to a backward direction or later.
17 . A method for operating a medical image processing apparatus comprising a memory that stores area information indicating a plurality of predetermined areas to be imaged in a photographic subject, and a processor, the method comprising:
acquiring a medical image of the photographic subject;
performing image recognition on the medical image using a trained model;
in a case where at least one of following conditions (a), (b), or (c):
(a) a specific subject continuously appears in the medical image for more than a predetermined time,
(b) a specific subject appears in a predetermined area of the medical image,
(c) a specific subject appears in the medical image at a size equal to or larger than a predetermined size
is met, recognizing an area of the subject in the medical image as an imaged area based on result of the image recognition;
recording information of the imaged area on a memory;
comparing the information of the imaged area with the area information indicating the plurality of areas to be imaged; and
controlling to output information discriminating between the information of the imaged area and information indicating a not-yet-imaged area at an expected end timing that is automatically determined by the processor based on a change in an organ to be observed or a detection of a predetermined anatomical landmark in the medical image.
18 . The method for operating a medical image processing apparatus according to claim 17 , wherein the image recognition is performed using a neural network.