Jointly trained machine learning models for automatic contouring in radiotherapy applications
Joint training techniques to train multiple models across clinical datasets for automatic contouring. Rather than using separate deep neural networks that are trained independently for each different dataset (e.g., a different image contrast or anatomy), joint training can be used to train multiple models simultaneously across clinical datasets for automatic contouring. By taking advantage of commonalities between two or more datasets, the techniques effectively take advantage of data that would otherwise be considered irrelevant to the task—allowing the user to train more performant models while requiring less training data per dataset.
1 . A computer-implemented method for generating a radiotherapy treatment plan for a subject including training processing circuitry to automatically contour an image depicting an anatomy of the subject, the method comprising:
jointly training a first machine learning model and a second machine learning model by using images from a first training dataset and a second training dataset independent from the first training dataset, wherein the first machine learning model and the second machine learning model are configured to share at least one learned model parameter, and wherein, when trained, the machine learning models are configured to automatically contour images for radiotherapy treatment.
2 . The method of claim 1 , wherein jointly training the at least two machine learning models includes:
inputting the first training dataset of a first type to train a first one of the machine learning models, wherein each training sample from the first training dataset is a single image, wherein the first machine learning model is configured to use at least one first parameter and the shared learned model parameter, the at least one first parameter being a learned parameter specific to the first machine learning model and not trained by a second one of the machine learning models; and
inputting a second training dataset of a second type to the second one of the machine learning models, wherein each training sample from the second training dataset comprises a single image, wherein the first type is different than the second type, wherein the second machine learning model is configured to use at least one second parameter and the shared learned model parameter, the at least one second parameter being a learned parameter specific to the second machine learning model and not trained by the first one of the machine learning models, and wherein the second machine learning model is coupled to the first machine learning model to share the parameter.
3 . The method of claim 2 , wherein the first machine learning model and the second machine learning model form part of a convolutional block of a convolutional neural network.
4 . The method of claim 3 , wherein the convolutional block includes a first normalization layer configured to receive data from the first training dataset and a second normalization layer configured to receive data from the second training dataset.
5 . The method of claim 2 , wherein the first type includes a T1-weighted image, and wherein the second type includes a T2-weighted image.
6 . The method of claim 2 , wherein the first type includes either a T1-weighted image or a T2-weighted image having a first scan time, and wherein the second type includes either a T1-weighted image or a T2-weighted image having a different time greater than the first scan time.
7 . The method of claim 2 , wherein the first type includes an anatomy of a male subject and the second type includes an anatomy of a female subject.
8 . The method of claim 7 , wherein the first type includes a male pelvic region and the second type includes female pelvic region.
9 . The method of claim 2 , wherein the first type includes a female subject having had an ablative surgery and the second type includes a female subject not having had the ablative surgery.
10 . The method of claim 2 , wherein the first type includes a male subject having had an ablative surgery and the second type includes a male subject not having had the ablative surgery.
11 . A computer-implemented method for generating a radiotherapy treatment plan for a subject including automatically contouring an image depicting an anatomy of the subject, the method comprising:
obtaining the image of the subject;
selecting, based on an image parameter, a previously trained first machine learning model, wherein the previously trained first machine learning model was jointly trained with a second machine learning model by using images from a first training dataset and a second training dataset independent from the first training dataset;
applying the first previously trained machine learning model to the image of the subject to generate a machine learning model output, wherein applying includes applying a shared learned parameter learned during joint training with the second machine learning model, and a learned parameter specific to the first machine learning model and not shared with the second machine learning model;
contouring, without user intervention and based on the first machine learning model output, one or more anatomical structures of the image; and
processing the one or more contoured anatomical structures to generate the radiotherapy treatment plan.
12 . The method of claim 11 , wherein the previously trained first machine learning model was trained using anatomies of male subjects and female subjects.
13 . The method of claim 12 , wherein the anatomies of male subjects and female subjects include anatomies of male pelvic regions and female pelvic regions.
14 . The method of claim 11 , wherein the previously trained first machine learning model was trained using anatomies of female subjects having had an ablative surgery and female subjects not having had the ablative surgery.
15 . The method of claim 11 , wherein the previously trained first machine learning model was trained using anatomies of male subjects having had an ablative surgery and male subjects not having had the ablative surgery.
16 . The method of claim 11 wherein the previously trained first machine learning model was trained using images having different imaging contrasts.
17 . The method of claim 11 , wherein the previously trained first machine learning model was trained using T2-weighted images having a first scan time and T2-weighted images having a second scan time greater than the first time.
18 . The method of claim 11 , wherein the first machine learning model is configured to generate a different set of structures than the second machine learning model.
19 . A radiotherapy system for generating a radiotherapy treatment plan for a subject including automatically contouring an image depicting an anatomy of the subject, the radiotherapy system comprising:
a radiation therapy device configured to deliver a dose of radiation to an anatomical region of interest; and
a processor configured to:
obtain the image of the subject;
select, based on an image parameter, a previously trained first machine learning model, wherein the previously trained first machine learning model was jointly trained with a second machine learning model by using images from a first training dataset and a second training dataset independent from the first training dataset;
apply the first previously trained machine learning model to the image of the subject to generate a machine learning model output, wherein applying includes applying a shared learned parameter learned during joint training with the second machine learning model, and a learned parameter specific to the first machine learning model and not shared with the second machine learning model;
contour, without user intervention and based on the first machine learning model output, one or more anatomical structures of the image; and
process the one or more contoured anatomical structures to generate the radiotherapy treatment plan.
20 . The radiotherapy system of claim 19 , wherein the previously trained first machine learning model was trained using images having different imaging contrasts.
21 . The radiotherapy system of claim 19 , wherein the previously trained first machine learning model was trained using T2-weighted images having a first scan time and T2-weighted images having a second scan time greater than the first scan time.
22 . The radiotherapy system of claim 19 , wherein the previously trained first machine learning model was trained using anatomies of male subjects and female subjects.
23 . The radiotherapy system of claim 19 , wherein the first machine learning model is configured to generate a different set of structures than the second machine learning model.