Medical-image-based lesion analysis method

1 . A method for analyzing a lesion based on a medical image performed by a computing device including at least one processor, the method comprising:

generating, input image data comprising 2D medical images and a 3D medical image from the medical image;

generating, from the 2D medical images, a first probability value regarding a presence of a nodule in at least one region of interest and first location information identifying the at least one region of interest,

estimating a second probability value regarding the at least one region of interest based on the 3D medical image and the first location information; and

determining, second location information for the nodule from the first location information, based on the first probability value and the second probability value.

2 . The method of claim 1 , wherein the generating of the input image data includes:

calculating a Hounsfield unit value based on a 3D medical image; and

generating the 2D medical images from the 3D medical image in which the Hounsfield unit value is calculated.

3 . The method of claim 1 , wherein the generating of the first probability value and the first location information includes:

generating first feature maps having a plurality of sizes based on the 2D medical images by using a first neural network module;

generating second feature maps by concatenating at least some of the first feature maps based on the sizes of the first feature maps by using a second neural network module; and

generating the first probability value and the first location information regarding the at least one region of interest by matching the second feature maps with a predetermined anchor box by using a third neural network module.

4 . The method of claim 3 , wherein the generating of the first probability value and the first location information further includes:

clustering, when there is a plurality of regions of interest, at least some of the regions of interest based on a ratio of overlapping regions between the plurality of regions of interest; and

correcting a coordinate included in the first location information.

5 . The method of claim 1 , wherein the estimating of the second probability value includes:

generating at least one third feature map by performing encoding based on a patch extracted from the 3D medical image based on the first location information by using a fourth neural network module;

generating at least one fourth feature map by performing decoding based on the third feature map by using a fifth neural network module; and

generating the second probability value regarding the at least one region of interest based on a feature map generated by combining the third feature map and the fourth feature map by using a sixth neural network module.

6 . The method of claim 1 , wherein the estimating of the second probability value is performed by a neural network that is pre-trained by performing,

a first operation of training the neural network based on a randomly sampled training image, and

a second operation of training the neural network based on a training image selected based on recall and precision.

7 . The method of claim 1 , wherein the determining of the second location information for the nodule includes:

comparing a third probability value regarding the presence of the nodule generated through a weighted sum of the first probability value and the second probability value, and a threshold value; and

determining the first location information of the at least one region of interest corresponding to the third probability value regarding the presence of the nodule selected as a result of the comparison as the second location information for the nodule.

8 . The method of claim 1 , further comprising:

generating a mask for the nodule based on a patch of the medical image corresponding to the second location information by using a pre-trained measurement module; and

generating numerical information including at least one of a diameter and a volume of the nodule based on the mask for the nodule.

9 . The method of claim 8 , wherein the mask for the suspicious nodule includes:

a first mask for an entire region of the nodule generated based on a 3D patch corresponding to the second location information; and

a second mask for a region representing a specific attribute of the nodule generated based on the 3D patch corresponding to the second location information.

10 . The method of claim 8 , further comprising:

classifying a class for a state of the nodule based on the patch of the medical image and the mask for the nodule by using a pre-trained classification module.

11 . The method of claim 10 , wherein the classifying of the class for the state of the nodule includes:

determining at least one of a type for an attribute of the nodule, whether the nodule is spiculated, or whether the nodule is calcified, based on the patch and the mask by using different sub modules included in the classification module.

12 . The method of claim 10 , further comprising:

calculating an assessment score of the nodule based on the numerical information and the class for the state of the nodule based on an auxiliary index of lung cancer diagnosis; and

modifying, when a subject of the input image data corresponds to a subject of a pre-analyzed image, an assessment score of the medical image or an assessment score of the pre-analyzed image based on capturing time points of the input image data and the pre-analyzed image by using a pre-trained tracking module.

13 . The method of claim 12 , further comprising:

generating a user interface based on at least one of the second location information, the mask, the class, the numerical information, or the assessment score for the nodule.

14 . The method of claim 10 , further comprising:

estimating malignancy of the nodule by inputting the second location information, the class for the state, and the numerical information of the nodule into a pre-trained machine learning module.

15 . The method of claim 14 , further comprising:

generating a user interface based on at least one of the second location information, the mask, the class, the numerical information, or the malignancy for the nodule.

16 . The method of claim 10 , further comprising:

estimating malignancy of the nodule by inputting the patch of the medical image and the mask for the nodule into a pre-trained machine learning module.

17 . A computer program stored in a non-transitory computer-readable storage medium, the computer program executing following operations for analyzing a lesion based on a medical image when the computer program is executed by one or more processors, the operations comprising:

an operation of generating, input image data comprising 2D medical images and a 3D medical image from the medical image;

an operation of generating, from the 2D medical images a first probability value regarding a presence of a nodule in at least one region of interest and first location information identifying the at least one region of interest;

an operation of estimating a second probability value regarding the at least one region of interest based on the 3D medical image and the first location information; and

an operation of determining, second location information for the nodule from the first location information, based on the first probability value and the second probability value.

18 . A computing device for analyzing a lesion based on a medical image, comprising:

a processor including at least one core;

a memory including program codes executable in the processor; and

a network unit receiving a medical image,

wherein the processor,

generates, input image data comprising 2D medical images and a 3D medical image from the medical image,

generates, from the 2D medical images, a first probability value regarding a presence of a nodule in at least one region of interest and first location information identifying the at least one region of interest,

estimates a second probability value regarding the at least one region of interest based on the 3D medical image and the first location information, and

determines, second location information for the nodule from the first location information, based on the first probability value and the second probability value.