Computer program, information processing method, and information processing device
A non-transitory computer-readable medium (CRM) storing computer program code executed by a computer processor that executes a process of acquiring a medical image generated based on a signal detected by a catheter inserted to a lumen organ, estimating a position of an object at least included in the acquired medical image by inputting the medical image to a first learning model for estimating a position of an object included in the medical image, extracting from the medical image an image portion by using the estimated position of the object as a reference, and recognizing the object included in the extracted image portion by inputting the image portion to a second learning model for recognizing an object included in the image portion.
1 . A non-transitory computer-readable medium (CRM) storing computer program code executed by a computer processor that executes a process comprising:
acquiring a medical image generated based on a signal detected by an imaging catheter inserted to a lumen organ;
estimating a position of an object at least included in the acquired medical image by inputting the medical image into a first learning model configured to estimate the position of the object included in the medical image;
extracting, from the medical image, an image portion based on the estimated position of the object included in the medical image; and
recognizing the object included in the extracted image portion by inputting the extracted image portion into a second learning model configured to recognize the object included in the image portion.
2 . The computer-readable medium according to claim 1 , further comprising:
acquiring a plurality of medical images generated in chronological order;
estimating a position of the object included in the medical image by inputting an acquired first medical image into the first learning model;
extracting an image portion from a second medical image acquired after the first medical image based on the position of the object estimated based on the first medical image; and
recognizing the object included in the image portion extracted from the second medical image by inputting the image portion into the second learning model.
3 . The computer-readable medium according to claim 1 , wherein the lumen organ is a blood vessel, further comprising:
acquiring a medical image of the blood vessel generated based on a signal detected by the imaging catheter.
4 . The computer-readable medium according to claim 1 , wherein the first learning model is trained to estimate a position of at least one of a blood vessel lumen boundary, a blood vessel wall portion, and a medical instrument existing in a blood vessel, further comprising:
extracting an image portion centered on the estimated position of the object.
5 . The computer-readable medium according to claim 1 , wherein the first learning model is trained to estimate a position of at least one of a guide wire inserted to a blood vessel lumen and a calcified portion in a blood vessel, further comprising:
extracting an image portion taking the estimated position of the object as a top portion or an end portion.
6 . The computer-readable medium according to claim 1 , further comprising:
superimposing an image indicating an object recognized using the second learning model on a medical image based on information on a position of an object estimated using the first learning model and displaying the superimposed image.
7 . The computer-readable medium according to claim 6 , further comprising:
recognizing a plurality of types of objects using the second learning model;
receiving a selection of a type of an object to be displayed; and
superimposing an image indicating an object of the received type on an acquired medical image and displaying the superimposed image.
8 . An information processing method for causing a computer to execute processes comprising:
acquiring a medical image generated based on a signal detected by an imaging catheter inserted to a lumen organ;
estimating a position of an object at least included in the acquired medical image by inputting the medical image into a first learning model configured to estimate the position of the object included in the medical image;
extracting, from the medical image, an image portion based on the estimated position of the object included in the medical image; and
recognizing the object included in the extracted image portion by inputting the extracted image portion into a second learning model configured to recognize the object included in the image portion.
9 . The method according to claim 8 , further comprising:
acquiring a plurality of medical images generated in chronological order;
estimating a position of the object included in the medical image by inputting an acquired first medical image into the first learning model;
extracting an image portion from a second medical image acquired after the first medical image based on the position of the object estimated based on the first medical image; and
recognizing the object included in the image portion extracted from the second medical image by inputting the image portion into the second learning model.
10 . The method according to claim 8 , wherein the lumen organ is a blood vessel, further comprising:
acquiring a medical image of the blood vessel generated based on a signal detected by the imaging catheter.
11 . The method according to claim 8 , wherein the first learning model is trained to estimate a position of at least one of a blood vessel lumen boundary, a blood vessel wall portion, and a medical instrument existing in a blood vessel, further comprising:
extracting an image portion centered on the estimated position of the object.
12 . The method according to claim 8 , wherein the first learning model is trained to estimate a position of at least one of a guide wire inserted to a blood vessel lumen and a calcified portion in a blood vessel, further comprising:
extracting an image portion taking the estimated position of the object as a top portion or an end portion.
13 . The method according to claim 8 , further comprising:
superimposing an image indicating an object recognized using the second learning model on a medical image based on information on a position of an object estimated using the first learning model and displaying the superimposed image.
14 . The method according to claim 13 , further comprising:
recognizing a plurality of types of objects using the second learning model;
receiving a selection of a type of an object to be displayed; and
superimposing an image indicating an object of the received type on an acquired medical image and displaying the superimposed image.
15 . An information processing device, comprising:
a processor configured to;
acquire a medical image generated based on a signal detected by an imaging catheter inserted to a lumen organ;
input the acquired medical image into a first learning model configured to output information indicating an estimated position of an object at least included in the medical image;
receive, an image portion extracted from the medical image based on the estimated position of the object included in the medical image; and
input the extracted image portion into a second learning model configured to output information indicating the object included in the image portion.
16 . The information processing device according to claim 15 , wherein the processor is configured to:
acquire a plurality of medical images generated in chronological order;
input an acquired first medical image into the first learning model to estimate a position of the object included in the medical image
extract an image portion from a second medical image acquired after the first medical image based on the position of the object estimated based on the first medical image; and
input the extracted image portion into the second learning model to recognize the object included in the image portion extracted from the second medical image.
17 . The information processing device according to claim 15 , wherein
the lumen organ is a blood vessel; and
the processor is configured to acquire a medical image of the blood vessel generated based on a signal detected by the imaging catheter.
18 . The information processing device according to claim 15 , wherein
the first learning model is trained to estimate a position of at least one of a blood vessel lumen boundary, a blood vessel wall portion, and a medical instrument existing in a blood vessel; and
the processor is configured to extract an image portion centered on the estimated position of the object.
19 . The information processing device according to claim 15 , wherein
the first learning model is trained to estimate a position of at least one of a guide wire inserted to a blood vessel lumen and a calcified portion in a blood vessel; and
the processor is configured to extract an image portion taking the estimated position of the object as a top portion or an end portion.
20 . The information processing device according to claim 15 , wherein
the second learning model is configured to superimpose an image indicating an object recognized on a medical image based on information on a position of an object estimated using the first learning model and to display the superimposed image.