Systems and methods of automatic medical image labeling

1. A computer-implemented method of automatically labeling medical images, the method comprising:

clustering training images and training labels of the training images into a plurality of clusters, wherein the training images include anatomical structures of interest, and wherein the training labels include masks that annotate the anatomical structures of interest in the training images, each cluster including a representative template having a representative image of the cluster and a representative label of the representative image, wherein clustering training images and training labels comprises:

for each of the training images and training labels,

registering each of the training images and training labels to a representative template of a corresponding cluster for each of the training images and training labels to derive transformed training images and transformed training labels; and

registering the transformed training images and transformed training labels using parameters estimated during registering the representative image to an unlabeled test image;

training a neural network model with a training dataset, wherein the training dataset includes the training images and the training labels, and target outputs of the neural network model are labels of the medical images;

generating a suboptimal label corresponding to the unlabeled test image using the trained neural network model;

generating an optimal label corresponding to the unlabeled test image using the suboptimal label and representative templates of the clusters, wherein generating the optimal label comprises:

for each of the registered transformed training images, computing a similarity metric between each of the registered transformed training images and the unlabeled test image; and

computing the optimal label based on the computed similarity metrics and the registered transformed training labels;

updating the training dataset using the test image and the optimal label;

retraining the neural network model with the updated training dataset;

generating a label of an unlabeled image using the retrained neural network model; and

outputting the generated label of the unlabeled image.

2. The method of claim 1 , wherein clustering training images and training labels further comprises:

for each of the training images and training labels, registering each of the training images and training labels to a representative template of a corresponding cluster for each of the training images and training labels.

3. The method of claim 2 , wherein registering further comprises registering a region of interest of the training image with a region of interest of the representative image.

4. The method of claim 1 , wherein generating an optimal label further comprises:

for each of the plurality of clusters,

registering the representative image to the unlabeled test image; and

computing a similarity metric between the registered representative image and the unlabeled test image; and

computing the optimal label based on the computed similarity metric.

5. The method of claim 4 , wherein registering the representative image further comprises:

registering a region of interest of the representative image to a region of interest of the unlabeled test image.

6. The method of claim 4 , wherein computing the optimal label comprises:

assigning the optimal label as a registered representative label of a registered representative image that has the highest similarity metric, wherein the registered representative label is generated by transforming the representative label with parameters estimated during registering the representative image to the unlabeled test image.

7. The method of claim 4 , wherein computing the optimal label comprises:

assigning the optimal label as a combined label of a plurality of registered representative labels for registered representative images that have similarity metrics meeting a predetermined criterion, wherein the plurality of registered representative labels are generated by transforming the representative labels with parameters estimated during registering the representative images to the unlabeled test image.

8. The method of claim 1 , wherein the anatomical structures of interest are first anatomical structures of interest, the method further comprising:

for each of the plurality of clusters,

updating the representative template by updating the representative label of the representative image, wherein the updated representative label includes the first anatomical structures of interest and second anatomical structures of interest in the representative image; and

updating training labels in the cluster by transforming the training labels using the updated representative label to derive updated labels.

9. The method of claim 8 , wherein updating the training dataset further comprises replacing the training labels in the training dataset with the updated labels.

10. A computer-implemented method of automatically labeling medical images, the method comprising:

clustering training images and training labels into a plurality of clusters, wherein the training images include anatomical structures of interest, and wherein the training labels include masks that annotate the anatomical structures of interest in the training images, each cluster including a representative template having a representative image of the cluster and a representative label of the representative image, wherein clustering training images and training labels comprises:

for each of the training images and training labels,

registering each of the training images and training labels to a representative template of a corresponding cluster for each of the training images and training labels to derive transformed training images and transformed training labels; and

registering the transformed training images and transformed training labels using parameters estimated during registering the representative image to an unlabeled test image;

training a neural network model with a training dataset, wherein the training dataset includes the training images and training labels, and target outputs of the neural network model are labels of the medical images;

generating a suboptimal label corresponding to the unlabeled test image using the trained neural network model; and

generating an optimal label corresponding to the unlabeled test image using the suboptimal label and representative templates of the clusters, wherein generating the optimal label comprises:

for each of the registered transformed training images, computing a similarity metric between each of the registered transformed training images and the unlabeled test image; and

computing the optimal label based on the computed similarity metrics and the registered transformed training labels.

11. The method of claim 10 , wherein clustering training images and training labels further comprises:

for each of the training images and training labels, registering each of the training images and training labels to a representative template of a corresponding cluster for each of the training images and training labels.

12. The method of claim 11 , wherein registering further comprises registering a region of interest of the training image with a region of interest of the representative image.

13. The method of claim 10 , wherein generating an optimal label further comprises:

for each of the plurality of clusters,

registering the representative image to the unlabeled test image; and

computing a similarity metric between the registered representative image and the unlabeled test image; and

computing the optimal label based on the computed similarity metric.

14. The method of claim 13 , wherein registering the representative image further comprises:

registering a region of interest of the representative image to a region of interest of the unlabeled test image.

15. The method of claim 13 , wherein computing the optimal label comprises:

assigning the optimal label as a registered representative label of a registered representative image having the highest similarity metric, wherein the registered representative label is generated by transforming the representative label with parameters estimated during registering the representative image to the unlabeled test image.

16. The method of claim 10 , wherein the anatomical structures of interest are first anatomical structures of interest, the method further comprising:

for each of the plurality of clusters,

updating the representative template by updating the representative label of the representative image, wherein the updated representative label includes the first anatomical structures of interest and second anatomical structures of interest in the representative image; and

updating training labels in the cluster by transforming the training labels using the updated representative label to derive updated labels.

17. The method of claim 16 , wherein updating the training dataset further comprises replacing the training labels in the training dataset with the updated labels.

18. A computer-implemented method of automatically labeling an image, the method comprising:

clustering training images and training labels into a plurality of clusters, wherein the training images include structures of interest, and wherein the training labels include masks that annotate structures of interest in the training images, each cluster including a representative template having a representative image of the cluster and a representative label of the representative image, wherein clustering training images and training labels comprises:

for each of the training images and training labels,

registering each of the training images and training labels to a representative template of a corresponding cluster for each of the training images and training labels to derive transformed training images and transformed training labels; and

registering the transformed training images and transformed training labels using parameters estimated during registering the representative image to an unlabeled test image;

generating a suboptimal label corresponding to an unlabeled image using a segmentation model;

generating an optimal label corresponding to the unlabeled image using the suboptimal label and representative templates of the clusters, wherein generating the optimal label comprises:

for each of the registered transformed training images, computing a similarity metric between each of the registered transformed training images and the unlabeled test image; and

computing the optimal label based on the computed similarity metrics and the registered transformed training labels; and

outputting the generated optimal label of the unlabeled image.