Predicting response to immunotherapy treatment using deep learning analysis of imaging and clinical data

1. A method, comprising:

uniquely training a plurality of deep learning models using sets of training data to predict immunotherapy treatment responses indicative of pseudo-progression based on a change in volume of a lesion of a patient, wherein each of the sets of training data are indicative of unique diagnostic imaging scans at baselines, follow-up intervals, and temporary changes in volume, wherein the pseudo-progression is characterized by increases in the volume of the lesion of the patient that are responsive to immunotherapy treatment rather than a progression of a disease;

providing a single pre-treatment image of a target lesion of a target patient to the plurality of deep learning models that are uniquely trained using the sets of training data to generate the immunotherapy treatment responses;

averaging, by a processing device, the immunotherapy treatment responses of the plurality of deep learning models that are uniquely trained using the sets of training data to generate a predicted treatment response score; and

providing, based on the predicted treatment response score, a recommended treatment plan for the target lesion of the target patient.

2. The method of claim 1 , further comprising:

receiving an intra-treatment follow-up image;

providing the intra-treatment follow-up image to the at least one deep learning model;

generating an updated predicted treatment response score; and

providing, based on the updated predicted treatment response score, an updated recommended treatment plan.

3. The method of claim 1 , wherein the pre-treatment image comprises a plurality of imaging features.

4. The method of claim 1 , wherein the at least one deep learning model comprises a convolutional neural network.

5. The method of claim 1 , wherein the treatment is a PD-[L]1 immune checkpoint inhibitor treatment.

6. The method of claim 1 , wherein the treatment is a PD-[L]1 or CTLA-4-immune checkpoint inhibitor treatment.

7. The method of claim 1 , wherein the treatment is a PD-[L]1-based treatment or CTLA-4-based treatment combined with chemotherapy treatment.

8. The method of claim 1 , wherein the treatment is a PD-[L]1-based treatment or CTLA-4-based treatment combined with radiotherapy treatment.

9. The method of claim 1 , wherein the pre-treatment image is one of: a three-dimensional anatomical image or a four-dimensional anatomical image.

10. The method of claim 1 , wherein the at least one deep learning model use a population of training subjects and a plurality of images associated with each of a plurality of training subjects as training data.

11. The method of claim 1 , wherein the predicted treatment response score indicates a prediction of a response to a predefined pharmaceutical product.

12. The method of claim 1 , wherein the predicted treatment response score indicates a prediction of a progression-free survival at patient-level and lesion-level on a predefined pharmaceutical product.

13. The method of claim 1 , wherein the predicted treatment response score indicates a prediction of an overall survival at a patient-level and lesion-level on a predefined pharmaceutical product.

14. The method of claim 1 , wherein the predicted treatment response score indicates a prediction of hyper-progression at a patient-level and lesion-level on a predefined pharmaceutical product.

15. The method of claim 1 , wherein the predicted treatment response score indicates a prediction of one or more immune-related adverse events associated with the treatment.

16. The method of claim 1 , further comprising:

providing, to the at least one deep learning model, one or more non-imaging features associated with the target subject, wherein the predicted treatment response score to the treatment is generated based on the single pre-treatment image, the one or more non-imaging features, and the at least one deep learning model.

17. A treatment analysis system comprising:

a memory to store a pre-treatment image of a target subject; and

a processing device, operatively coupled to the memory, the processing device to:

uniquely train a plurality of deep learning models using sets of training data to predict immunotherapy treatment responses indicative of pseudo-progression based on a change in volume of a lesion of a patient, wherein each of the sets of training data are indicative of unique diagnostic imaging scans at baselines, follow-up intervals, and temporary changes in volume, wherein the pseudo-progression is characterized by increases in the volume of the lesion of the patient that are responsive to immunotherapy treatment rather than a progression of a disease;

provide a single pre-treatment image of a target lesion of a target patient to the plurality of deep learning models that are uniquely trained using sets of training data to generate the immunotherapy treatment responses;

average the immunotherapy treatment responses of the plurality of deep learning models that are uniquely trained using the sets of training data to generate a predicted treatment response; and

provide, based on the predicted treatment response score, a recommended treatment plan for the target lesion of the target patient.

18. The treatment analysis system of claim 17 , wherein the at least one deep learning model comprises a convolutional neural network.

19. The treatment analysis system of claim 17 , wherein the treatment is a PD-[L]1 immune checkpoint inhibitor treatment.

20. The treatment analysis system of claim 17 , wherein the treatment is a PD-[L]1 or CTLA-4-immune checkpoint inhibitor treatment.

21. The treatment analysis system of claim 17 , wherein the treatment is a PD-[L]1-based treatment or CTLA-4-based treatment combined with chemotherapy treatment.

22. The treatment analysis system of claim 17 , wherein the predicted treatment response score indicates a prediction of a response to a predefined pharmaceutical product.

23. The treatment analysis system of claim 17 , wherein the predicted treatment response score indicates a prediction of hyper-progression at a patient-level and lesion-level on a predefined pharmaceutical product.

24. A non-transitory computer-readable storage medium comprising instructions, which when executed by a processing device, cause the processing device to:

uniquely train a plurality of deep learning models using sets of training data to predict immunotherapy treatment responses indicative of pseudo-progression based on a change in volume of a lesion of a patient, wherein each of the sets of training data are indicative of unique diagnostic imaging scans at baselines, follow-up intervals, and temporary changes in volume, wherein the pseudo-progression is characterized by increases in the volume of the lesion of the patient that are responsive to immunotherapy treatment rather than a progression of a disease;

provide a single pre-treatment image of a target lesion of a target patient to the plurality of deep learning models that are uniquely trained using sets of training data to generate the immunotherapy treatment responses;

average, by the processing device, the immunotherapy treatment responses of the plurality of deep learning models that are uniquely trained using the sets of training data to generate a predicted treatment response score; and

provide, based on the predicted treatment response score, a recommended treatment plan for the target lesion of the target patient.

25. The non-transitory computer-readable storage medium of claim 24 , wherein the at least one deep learning model comprises a convolutional neural network.

26. The non-transitory computer-readable storage medium of claim 24 , wherein the treatment is a PD-[L]1 immune checkpoint inhibitor treatment.

27. The non-transitory computer-readable storage medium of claim 24 , wherein the treatment is a PD-[L]1 or CTLA-4-immune checkpoint inhibitor treatment.

28. The non-transitory computer-readable storage medium of claim 24 , wherein the treatment is a PD-[L]1-based treatment or CTLA-4-based treatment combined with chemotherapy treatment.

29. The non-transitory computer-readable storage medium of claim 24 , wherein the predicted treatment response score indicates a prediction of hyper-progression at a patient-level and lesion-level on a predefined pharmaceutical product.

30. The method of claim 1 , wherein the sets of training data are different from each other.

31. The method of claim 1 , wherein each deep learning model of the plurality of deep learning models is further uniquely trained using a unique set of hyperparameters indicative of at least one of a model architecture type, a model size, or a weight initialization.