Image processing device, medical diagnosis device, endoscope device, and image processing method
A processor of an image processing device acquires a medical image including a lesion region, and causes a determiner to perform a determination process of determining a type of a lesion on the basis of the medical image. A first determination process is a process of determining whether the type belongs to a first group including a first type classified as a first category and a second type classified as a second category or a second group including a third type classified as the second category. A second determination process is a process of determining whether the type is the first type or the second type. The processor outputs a first signal for specifying whether the type is classified as the first category or the second category on the basis of a determination result of the first determination process and a determination result of the second determination process.
1 . An image processing device comprising:
a processor,
wherein the processor is configured to:
acquire a medical image including a lesion region showing a lesion, and
cause a determiner to perform a determination process of determining a type of the lesion on the basis of the medical image,
the determiner includes a first determiner and a second determiner,
the determiner is a neural network,
the first determiner and the second determiner share a plurality of layers after an input layer,
the determination process includes a first determination process of the first determiner and a second determination process of the second determiner,
the first determination process is a process of determining whether the type belongs to a first group including a first type classified as a first category and a second type classified as a second category or a second group including a third type classified as the second category,
the second determination process is a process of determining whether the type is the first type or the second type, and is a process using an intermediate feature amount of the first determiner,
the processor is configured to perform at least one of a first output process of outputting a first signal for specifying whether the type is classified as the first category or the second category on the basis of a determination result of the first determination process and a determination result of the second determination process or a second output process of outputting a second signal for specifying whether the type is the first type or the second type on the basis of the determination result of the first determination process and the determination result of the second determination process,
the first output process includes a process of outputting a signal for generating a position specifying image based on a feature amount map obtained from the determiner,
the processor is configured to perform a display process of causing a display device to display the position specifying image and the medical image in a contrastable manner, and
results of the determination process are used for selecting a treatment for the lesion.
2 . The image processing device according to claim 1 ,
wherein, in a case where it is determined that the type belongs to the first group through the first determination process and it is determined that the type is the second type through the second determination process, the processor outputs, as the first signal, a second category signal for specifying that the type is classified as the second category.
3 . The image processing device according to claim 1 ,
wherein, in a case where it is determined that the type belongs to the first group through the first determination process and it is determined that the type is the first type through the second determination process, the processor outputs, as the first signal, a first category signal for specifying that the type is classified as the first category.
4 . The image processing device according to claim 1 ,
wherein the processor performs at least one of a display process of displaying first category information indicating that the type is classified as the first category and second category information indicating that the type is classified as the second category on a display device in a distinguishable display mode on the basis of the determination result of the first determination process and the determination result of the second determination process or a storage process of storing the first category information and the second category information in a unit of the medical image in a distinguishable manner on the basis of the determination result of the first determination process and the determination result of the second determination process.
5 . The image processing device according to claim 4 ,
wherein the display mode includes a first mode of displaying the medical image and the first category information on the display device in a contrastable manner and a second mode of displaying the medical image and the second category information on the display device in a contrastable manner.
6 . The image processing device according to claim 4 ,
wherein at least one of the first category information or the second category information is information based on a feature amount map obtained from the determiner.
7 . The image processing device according to claim 1 ,
wherein the first determiner is a model that has been trained with a first lesion image in which a first lesion corresponding to the first type is captured and a second lesion image in which a second lesion corresponding to the second type is captured as images corresponding to the first group, and a third lesion image in which a third lesion corresponding to the third type is captured as an image corresponding to the second group.
8 . The image processing device according to claim 1 ,
wherein the processor
acquires the medical image in a time series over a plurality of frames,
determines which of a plurality of lesion types including the first type and the second type is the type for each of the frames on the basis of the determination result of the first determination process and the determination result of the second determination process,
outputs a signal corresponding to a second type determination signal indicating that the type is determined as being the second type in a case where it is determined that the type is the second type for a first frame among the plurality of frames, and
outputs a signal corresponding to the second type determination signal in a case where it is determined that the type is a lesion type different from the second type for second frames within a range from the first frame to a predetermined number of frames among the plurality of frames.
9 . The image processing device according to claim 1 ,
wherein the first category is a category showing non-neoplasticity, and
the second category is a category showing neoplasticity.
10 . The image processing device according to claim 1 ,
wherein the first type is a non-neoplastic lesion.
11 . The image processing device according to claim 1 ,
wherein the second type is a serrated lesion.
12 . The image processing device according to claim 1 ,
wherein the third type is a neoplastic lesion different from a serrated lesion.
13 . An image processing method comprising:
acquiring a medical image including a lesion region showing a lesion; and
causing a determiner to perform a determination process of determining a type of the lesion on the basis of the medical image,
wherein the determiner includes a first determiner and a second determiner,
the determiner is a neural network,
the first determiner and the second determiner share a plurality of layers after an input layer,
the determination process includes a first determination process of the first determiner and a second determination process of the second determiner,
the first determination process is a process of determining whether the type belongs to a first group including a first type classified as a first category and a second type classified as a second category or a second group including a third type classified as the second category,
the second determination process is a process of determining whether the type is the first type or the second type, and is a process using an intermediate feature amount of the first determiner,
the image processing method further includes performing at least one of a first output process of outputting a first signal for specifying whether the type is classified as the first category or the second category on the basis of a determination result of the first determination process and a determination result of the second determination process or a second output process of outputting a second signal for specifying whether the type is the first type or the second type on the basis of the determination result of the first determination process and the determination result of the second determination process,
the first output process includes a process of outputting a signal for generating a position specifying image based on a feature amount map obtained from the determiner,
the image processing method further includes performing a display process of causing a display device to display the position specifying image and the medical image in a contrastable manner, and
results of the determination process are used for selecting a treatment for the lesion.
14 . An image processing device comprising:
a processor,
wherein the processor is configured to:
acquire a medical image including a lesion region showing a lesion over a plurality of frames in a time series, and
cause a determiner to perform a determination process of determining which of a plurality of lesion types including a first type and a second type is a type of the lesion for each of the frames on the basis of the medical image,
the determiner is a neural network including a first determiner and a second determiner,
the first determiner and the second determiner share a plurality of layers after an input layer,
the determination process includes a first determination process of the first determiner and a second determination process of the second determiner,
the second determination process is a process using an intermediate feature amount of the first determiner,
the processor is configured to output, in a case where it is determined that the type is the second type for a first frame among the plurality of frames, a signal corresponding to a second type determination signal indicating that the type is determined as being the second type for second frames within a range from the first frame to a predetermined number of frames among the plurality of frames,
the processor is configured to output a determination result signal corresponding to a determination result of the type for the second frames in a case where it is determined that the type is the first type for the first frame,
the signal corresponding to the second type determination signal includes a signal for generating a position specifying image based on a feature amount map obtained from the determiner,
the processor is configured to perform a display process of causing a display device to display the position specifying image and the medical image in a contrastable manner, and
results of the determination process are used for selecting a treatment for the lesion.
15 . The image processing device according to claim 14 ,
wherein a category to which the first type belongs is a category showing non-neoplasticity, and
a category to which the second type belongs is a category showing neoplasticity.
16 . The image processing device according to claim 14 ,
wherein the first type is a type showing non-neoplasticity, and
the second type is a type showing a serrated lesion.
17 . The image processing device according to claim 14 ,
wherein the processor performs at least one of a display process of displaying information based on the signal corresponding to the second type determination signal and information based on the determination result signal on a display device in units of the frames or a storage process of storing the information based on the signal corresponding to the second type determination signal and the information based on the determination result signal in the units of the frames.
18 . A medical diagnosis device comprising:
the image processing device according to claim 1 ; and
an imaging device that acquires an image showing an observation target region of a subject as the medical image by imaging the observation target region.
19 . An endoscope device comprising:
the image processing device according to claim 1 ; and
an endoscope that is inserted into a body and acquires an image showing an observation target region in the body as the medical image by imaging the observation target region.