AI-assisted generation of annotated medical images
A set of pre-annotated medical images is received, and the received set is processed by automatically: training an AI-based uncertainty model using the received set as training data; processing medical images of the received set by determining classified segments and/or uncertainty regions in the medical images using the trained AI-based uncertainty model; selecting at least a part of the processed medical images including classified segments and/or uncertainty regions based on the processing result; and presenting the selected part of processed medical images to a human expert. Furthermore, a modified received set including additional annotations created by the human expert is received.
1 . A method for AI-assisted generation of annotated medical images, the method comprising:
receiving a set of pre-annotated medical images, a subset of the set of pre-annotated medical images being coarsely annotated by a first human expert;
processing the set of pre-annotated medical images by automatically training an AI-based uncertainty model using the set of pre-annotated medical images as training data,
processing medical images of the set of pre-annotated medical images by determining at least one of classified segments or uncertainty regions in the medical images using the trained AI-based uncertainty model,
selecting at least a part of the processed medical images including the at least one of the classified segments or the uncertainty regions based on the processing, and
presenting the part of the processed medical images to a second human expert; and
receiving a modified set of pre-annotated medical images including additional annotations created by the second human expert, wherein the additional annotations are related to the part of the processed medical images.
2 . The method according to claim 1 , wherein at least one of:
the medical images include whole slide images,
the AI-based uncertainty model includes an AI-based Bayesian segmentation model, or
the selecting of the at least the part of the processed medical images is performed based on a ranking of the at least one of the classified segments or the uncertainty regions.
3 . The method according to claim 1 , wherein at least the part of the set of pre-annotated medical images are coarsely segmented based on at least one of a line or a closed curve within the set of pre-annotated medical images.
4 . The method according to claim 3 , wherein the at least one of the line or the closed curve is at least one of piecewise linear or piecewise polynomial.
5 . The method according to claim 1 , further comprising:
determining the classified segments, the determining of the classified segments including refining a segmentation of coarsely annotated medical images by the first human expert.
6 . The method according to claim 1 , wherein the processing the set of pre-annotated medical images and the receiving the modified set of pre-annotated medical images are repeated iteratively using the modified set of pre-annotated medical images as the training data in the training of the AI-based uncertainty model.
7 . The method according to claim 6 , wherein the training of the AI-based uncertainty model, the processing of the medical images of the set of pre-annotated medical images, and the selecting at least the part of the processed medical images are repeated iteratively, until a quality criteria for the annotated medical images is achieved.
8 . The method according to claim 1 , further comprising:
determining the uncertainty regions using the trained AI-based uncertainty model; and wherein
the determining of the at least one of the classified segments or the uncertainty regions includes
generating a simplified graphical representation of the uncertainty regions, and
extracting the simplified graphical representation for presentation to the second human expert.
9 . The method according to claim 8 , wherein the simplified graphical representation includes at least one of:
a point set based representation, or
a parameter based representation.
10 . The method according to claim 2 , wherein the selecting of at least the part of the processed medical images is based on a ranking of the at least one of the classified segments or the uncertainty regions, and wherein the ranking is based on a determination of a preferred type of the at least one of the classified segments or the uncertainty regions.
11 . The method according to claim 10 , wherein the preferred type includes at least one of:
an image structure systematically neglected by a coarse annotation,
a high uncertainty,
assigned metadata revealing a type of rarely collected medical images, or
image features revealing dissimilar medical images.
12 . The method according to claim 1 , wherein the AI-based uncertainty model comprises at least one of:
a specialized spatial segmentation loss algorithm,
a feature pyramid pooling algorithm,
a Monte Carlo Dropout algorithm,
a Monte Carlo Depth algorithm, or
a Deep Ensemble algorithm.
13 . The method according to claim 1 , wherein at least one of the AI-based uncertainty model or an AI-based segmentation model is trained based on the modified set of pre-annotated medical images.
14 . A computer-implemented method for providing a segmented medical image, computer-implemented method comprising:
receiving a generated medical image;
determining a segmented medical image by processing the generated medical image by a trained AI-based segmentation model provided by the method according to claim 13 ; and
providing the segmented medical image.
15 . A non-transitory computer-readable medium storing computer-executable instructions that, when executed at a computer, cause the computer to perform the method of claim 14 .
16 . A non-transitory computer-readable medium storing computer-executable instructions that, when executed at a computer, cause the computer to perform the method of claim 1 .
17 . The method according to claim 4 , wherein the processing the set of pre-annotated medical images and the receiving a modified set of pre-annotated medical images are repeated iteratively using the modified set of pre-annotated medical images as the training data in the training of the AI-based uncertainty model.
18 . The method of claim 7 , wherein the quality criteria is met when a first modified set of pre-annotated medical images from a first iteration is less than a threshold difference from a second modified set of pre-annotated medical images from a second iteration after the first iteration.
19 . An annotation assistance device, comprising:
a communication interface configured to receive a set of pre-annotated medical images;
a model training unit configured to train an AI-based uncertainty model using the set of pre-annotated medical images as training data, a subset of the set of pre-annotated medical images being coarsely annotated by a first human expert;
a processing unit configured to determine at least one of classified segments or uncertainty regions in medical images of the set of pre-annotated medical images using the trained AI-based uncertainty model; and
a selection unit configured to select at least a part of the medical images including the at least one of the classified segments or the uncertainty regions,
wherein the communication interface is configured to
present the part of the medical images to a second human expert, and
receive a modified set of pre-annotated medical images including additional annotations created by the second human expert, the additional annotations being related to the part of the medical images.
20 . An annotation assistance device, comprising:
at least one processor configured to execute computer-executable instructions to cause the annotation assistance device to
train an AI-based uncertainty model using a set of pre-annotated medical images as training data, a subset of the set of pre-annotated medical images being coarsely annotated by a first human expert,
determine at least one of classified segments or uncertainty regions in medical images of the set of pre-annotated medical images using the trained AI-based uncertainty model,
select at least a part of the medical images including the at least one of the classified segments or the uncertainty regions,
present the part of the medical images to a second human expert, and
receive a modified set of pre-annotated medical images including additional annotations created by the second human expert, the additional annotations being related to the part of the medical images.