Method and system for outputting a slice corresponding to a target item for 3D image including a plurality of slices
Provided is a method for providing annotation information for a 3D image, which may include outputting a representative image for the 3D image including a plurality of slices, selecting at least one pixel associated with a target item from among a plurality of pixels included in the representative image, outputting, among the plurality of slices, a slice associated with the selected at least one pixel, and receiving an annotation for a partial region of the output slice.
1. A method, performed by one or more computing devices, for providing analysis results for an artificial neural network model for a 3D image, the method comprising:
outputting a representative image for the 3D image including a plurality of slices;
selecting at least one pixel associated with a target item from among a plurality of pixels included in the representative image; and
outputting, among the plurality of slices, at least one slice associated with the selected at least one pixel based on a prediction score for the target item,
wherein the prediction score for the target item is output by inputting each of the plurality of slices to a target item detection model, and
wherein the outputting, among the plurality of slices, the at least one slice associated with the selected at least one pixel includes outputting, based on the prediction score, a slice where a visibility of the target item is highest among the plurality of slices.
2. The method according to claim 1 , wherein:
the prediction score for the target item includes a plurality of prediction scores for a plurality of pixels included in each of the plurality of slices; and
the target item detection model is trained to output the prediction score for the target item for each of the plurality of pixels included in a 2D image.
3. The method according to claim 1 , wherein the outputting, among the plurality of slices, the at least one slice associated with the selected at least one pixel includes outputting, among the plurality of slices, a slice that has a highest prediction score for the target item in the at least one pixel.
4. The method according to claim 1 , wherein the outputting, among the plurality of slices, the at least one slice associated with the selected at least one pixel comprises:
outputting one or more slices that have prediction scores for the target item in the at selected least one pixel equal to or greater than a predetermined threshold.
5. The method according to claim 4 , wherein:
the plurality of slices include information on a sequence of the plurality of slices;
the one or more slices that have the prediction scores equal to or greater than the predetermined threshold include a plurality of slices that have prediction scores equal to or greater than the predetermined threshold; and
the outputting, among the plurality of slices, the at least one slice associated with the selected at least one pixel includes outputting at least one of a first slice or a last slice in the plurality of slices that have the prediction scores equal to or greater than the predetermined threshold.
6. The method according to claim 1 , wherein the outputting the at least one slice associated with the selected at least one pixel includes:
applying a prediction result generated based on the prediction score for the target item to the at least one slice associated with the at least one pixel; and
outputting the applied slice.
7. The method according to claim 1 , wherein:
the target item detection model is trained to further infer additional information on the target item included in the 3D image; and
the outputting the representative image for the 3D image including the plurality of slices includes outputting additional information including at least one of a calcification, a soft tissue lesion, a BI-RADS or a density of the target item, associated with the representative image.
8. A computer program stored in a computer-readable recording medium, for executing the method according to claim 1 on a computer.
9. A user terminal, comprising:
a memory storing one or more instructions; and
a processor configured to execute the stored one or more instructions to:
output a representative image for a 3D image including a plurality of slices;
select at least one pixel associated with a target item from among a plurality of pixels included in the representative image; and
output, among the plurality of slices, at least one slice associated with the selected at least one pixel based on a prediction score for the target item;
wherein the prediction score for the target item is output by inputting each of the plurality of slices to a target item detection model, and
wherein the processor is further configured to output, based on the prediction score, a slice where a visibility of the target item is highest among the plurality of slices.
10. The user terminal according to claim 9 , wherein:
the prediction score for the target item includes a plurality of prediction scores for a plurality of pixels included in each of the plurality of slices; and
the target item detection model is trained to output the prediction score for the target item for each of the plurality of pixels included in a 2D image.
11. The user terminal according to claim 9 , wherein the processor is further configured to output, among the plurality of slices, a slice that has a highest prediction score for the target item in the at least one pixel.
12. The user terminal according to claim 9 , further configured to output one or more slices that have prediction scores for the target item in the selected at least one pixel equal to or greater than a predetermined threshold.
13. The method according to claim 10 , wherein the outputting the representative image includes outputting the representative image with the prediction result, the prediction result being applied in a form of at least one of a heatmap or contour to the representative image.
14. The method according to claim 1 , wherein the representative image is one of an image generated by compositing multiple slices among the plurality of slices to come out the target item, one of the plurality of slices included in the 3D image, or an image provided by a 3D image capturing system.
15. The user terminal according to claim 9 , wherein the processor is further configured to:
apply a prediction result generated based on the prediction score for the target item to the at least one slice associated with the at least one pixel; and
output the applied at least one slice.
16. The user terminal according to claim 15 , wherein the processor is further configured to output the representative image with the prediction result, the prediction result being applied in a form of at least one of a heatmap or contour to the representative image.
17. The user terminal according to claim 9 , wherein:
the target item detection model is trained to further infer additional information on the target item included in the 3D image; and
the processor is further configured to output additional information including at least one of a calcification, a soft tissue lesion, a BI-RADS or a density of the target item, associated with the representative image.
18. The user terminal according to claim 9 , wherein the representative image is one of an image generated by compositing multiple slices among the plurality of slices to come out the target item, one of the plurality of slices included in the 3D image, or an image provided by a 3D image capturing system.