Aggregation of artificial intelligence (AI) engines

1. A method for a computer system to generate an aggregated artificial intelligence (AI) engine for radiotherapy treatment planning, wherein the method comprises:

obtaining multiple AI engines associated with respective multiple treatment planners, wherein the multiple AI engines are trained to perform a particular treatment planning step, and each of the multiple AI engines is trained to emulate one of the multiple treatment planners performing the particular treatment planning step;

generating multiple sets of output data using the multiple AI engines associated with the respective multiple treatment planners;

comparing the multiple AI engines associated with the respective multiple treatment planners based on the multiple sets of output data; and

based on the comparison, aggregating at least some of the multiple AI engines to generate the aggregated AI engine for performing the particular treatment planning step.

2. The method of claim 1 , wherein comparing the multiple AI engines comprises:

assigning a weight to each of the multiple AI engines based on the multiple sets of output data and a performance target.

3. The method of claim 2 , wherein comparing the multiple AI engines comprises:

extracting parameter data specifying one or more measurable features associated with the multiple sets of output data; and

assigning the weight to each of the multiple AI engines based on the comparison between the parameter data and the performance target.

4. The method of claim 3 , wherein comparing the multiple AI engines comprises:

determining the performance target based on the parameter data, wherein the performance target includes one or more of the following: a mean value associated with the parameter data; a weighted average value associated with the parameter data; a mean value or median value associated with a unimodal distribution of the parameter data; and one of multiple mean values associated with a multimodal distribution of the parameter data.

5. The method of claim 2 , wherein aggregating at least some of the multiple AI engines comprises:

based on the weight assigned to each of the multiple AI engines, generating aggregated training data to train the aggregated AI engine to perform the particular treatment planning step.

6. The method of claim 1 , wherein the method comprises:

performing radiotherapy treatment planning using an inferential chain that includes the aggregated AI engine and at least one other AI engine, wherein the aggregated AI engine is trained independently from the at least one other AI engine.

7. The method of claim 1 , wherein obtaining the multiple AI engines comprises:

training each of the multiple AI engines to perform at least one of the following treatment planning steps: segmentation to generate output data in the form of structure data based on image data; three-dimensional (3D) dose prediction to generate output data in the form of 3D dose data based on structure data; structure projection estimation to generate output data in the form of structure projection data to deliver 3D dose data; clinical goal estimation to generate output data in the form of clinical goal data; patient outcome prediction to generate output data in the form of patient outcome data; proton dose estimation to generate output data in the form of proton dose data; proton layer spot data estimation to generate output data in the form of proton layer spot data; and

treatment plan generation to generate output data in the form of a treatment plan for treatment delivery using a treatment system.

8. A non-transitory computer-readable storage medium that includes a set of instructions which, in response to execution by a processor of a computer system, cause the processor to perform a method of generating an aggregated artificial intelligence (AI) engine for radiotherapy treatment planning, the method comprising:

obtaining multiple AI engines associated with respective multiple treatment planners, wherein the multiple AI engines are trained to perform a particular treatment planning step, and each of the multiple AI engines is trained to emulate one of the multiple treatment planners performing the particular treatment planning step;

generating multiple sets of output data using the multiple AI engines associated with the respective multiple treatment planners;

comparing the multiple AI engines associated with the respective multiple treatment planners based on the multiple sets of output data; and

based on the comparison, aggregating at least some of the multiple AI engines to generate the aggregated AI engine for performing the particular treatment planning step.

9. The non-transitory computer-readable storage medium of claim 8 , wherein comparing the multiple AI engines comprises:

assigning a weight to each of the multiple AI engines based on the multiple sets of output data and a performance target.

10. The non-transitory computer-readable storage medium of claim 9 , wherein comparing the multiple AI engines comprises:

extracting parameter data specifying one or more measureable features associated with the multiple sets of output data; and

assigning the weight to each of the multiple AI engines based on the comparison between the parameter data and the performance target.

11. The non-transitory computer-readable storage medium of claim 10 , wherein comparing the multiple AI engines comprises:

determining the performance target based on the parameter data, wherein the performance target includes one or more of the following: a mean value associated with the parameter data; a weighted average value associated with the parameter data; a mean value or median value associated with a unimodal distribution of the parameter data; and one of multiple mean values associated with a multimodal distribution of the parameter data.

12. The non-transitory computer-readable storage medium of claim 9 , wherein aggregating at least some of the multiple AI engines comprises:

based on the weight assigned to each of the multiple AI engines, generating aggregated training data to train the aggregated AI engine to perform the particular treatment planning step.

13. The non-transitory computer-readable storage medium of claim 8 , wherein the method comprises:

performing radiotherapy treatment planning using an inferential chain that includes the aggregated AI engine and at least one other AI engine, wherein the aggregated AI engine is trained independently from the at least one other AI engine.

14. The non-transitory computer-readable storage medium of claim 8 , wherein obtaining the multiple AI engines comprises:

training each of the multiple AI engines to perform at least one of the following treatment planning steps: segmentation to generate output data in the form of structure data based on image data; three-dimensional (3D) dose prediction to generate output data in the form of 3D dose data based on structure data; structure projection estimation to generate output data in the form of structure projection data to deliver 3D dose data; clinical goal estimation to generate output data in the form of clinical goal data; patient outcome prediction to generate output data in the form of patient outcome data; proton dose estimation to generate output data in the form of proton dose data; proton layer spot data estimation to generate output data in the form of proton layer spot data; and treatment plan generation to generate output data in the form of a treatment plan for treatment delivery using a treatment system.

15. A computer system configured to generate an aggregated artificial intelligence (AI) engine for radiotherapy treatment planning, the computer system comprising:

a processor; and

a non-transitory computer-readable medium having stored thereon instructions that, when executed by the processor, cause the processor to:

obtain multiple AI engines associated with respective multiple treatment planners, wherein the multiple AI engines are trained to perform a particular treatment planning step, and each of the multiple AI engines is trained to emulate one of the multiple treatment planners performing the particular treatment planning step;

generate multiple sets of output data using the multiple AI engines associated with the respective multiple treatment planners;

compare the multiple AI engines associated with the respective multiple treatment planners based on the multiple sets of output data; and

based on the comparison, aggregate at least some of the multiple AI engines to generate the aggregated AI engine for performing the particular treatment planning step.

16. The computer system of claim 15 , wherein the instructions for comparing the multiple AI engines cause the processor to:

assign a weight to each of the multiple AI engines based on the multiple sets of output data and a performance target.

17. The computer system of claim 16 , wherein the instructions for comparing the multiple AI engines cause the processor to:

extract parameter data specifying one or more measureable features associated with the multiple sets of output data; and

assign the weight to each of the multiple AI engines based on the comparison between the parameter data and the performance target.

18. The computer system of claim 17 , wherein the instructions for comparing the multiple AI engines cause the processor to:

determine the performance target based on the parameter data, wherein the performance target includes one or more of the following: a mean value associated with the parameter data; a weighted average value associated with the parameter data; a mean value or median value associated with a unimodal distribution of the parameter data; and one of multiple mean values associated with a multimodal distribution of the parameter data.

19. The computer system of claim 16 , wherein the instructions for aggregating at least some of the multiple AI engines cause the processor to:

based on the weight assigned to each of the multiple AI engines, generate aggregated training data to train the aggregated AI engine to perform the particular treatment planning step.

20. The computer system of claim 15 , wherein the instructions further cause the processor to:

perform radiotherapy treatment planning using an inferential chain that includes the aggregated AI engine and at least one other AI engine, wherein the aggregated AI engine is trained independently from the at least one other AI engine.

21. The computer system of claim 15 , wherein the instructions for obtaining the multiple AI engines cause the processor to:

train each of the multiple AI engines to perform at least one of the following treatment planning steps: segmentation to generate output data in the form of structure data based on image data; three-dimensional (3D) dose prediction to generate output data in the form of 3D dose data based on structure data; structure projection estimation to generate output data in the form of structure projection data to deliver 3D dose data; clinical goal estimation to generate output data in the form of clinical goal data; patient outcome prediction to generate output data in the form of patient outcome data; proton dose estimation to generate output data in the form of proton dose data; proton layer spot data estimation to generate output data in the form of proton layer spot data; and treatment plan generation to generate output data in the form of a treatment plan for treatment delivery using a treatment system.