Systems and methods for processing digital images for radiation therapy

1. A computer-implemented method for predicting a benefit of radiation therapy, the method comprising: receiving one or more digital images of a pathology specimen; receiving additional information about a patient and/or a disease associated with the pathology specimen and transforming the additional information into a vector representation;

identifying a target region of one or more digital images for analysis, the target region including at least a portion of a detected tumor and tissue surrounding the detected tumor;

removing a non-relevant region of the one or more digital images:

applying a machine learning system to the one or more digital images to predict a benefit of radiation therapy for the target region by integrating the vector representation into a neural network of the machine learning system, the machine learning system having been trained by processing a plurality of training images to predict the benefit of radiation therapy for the target region, wherein the predicted benefit of radiation therapy is based on an intrinsic sensitivity of the detected tumor to radiation and/or a probability of radiotherapy-induced toxicity in the tissue surrounding the detected tumor; and

outputting the predicted benefit of radiation therapy corresponding to the target region to an electronic storage device.

2. The computer-implemented method of claim 1 , further comprising alerting a user with a visual indicator when the benefit of radiation therapy is available.

3. The computer-implemented method of claim 1 , further comprising:

detecting one or more tumors in the received one or more digital images, wherein identifying the target region of the one or more digital images for analysis is done by using the machine learning system and is based on the detected one or more tumors;

identifying an additional target region of the one or more digital images for analysis using the machine learning system and based on the detected one or more tumors, and

applying the machine learning system to the one or more digital images to predict an additional benefit of radiation therapy for the additional target region.

4. The computer-implemented method of claim 1 , wherein identifying the target region of the one or more digital images for analysis is done manually.

5. The computer-implemented method of claim 1 , wherein the target region comprises an entire slide image of the one or more digital images.

6. The computer-implemented method of claim 1 , wherein no target region is identified and an entire one or more digital slides is analyzed.

7. The computer-implemented method of claim 1 , wherein using a plurality of training images to predict the benefit of radiation therapy comprises:

receiving one or more digital images of a pathology specimen;

receiving additional information about an associated patient or an associated disease and transforming the additional information into a vector representation;

identifying a region of the one or more digital images to analyze and removing a non-relevant region;

applying a trained machine learning system to the one or more digital images to predict the benefit of radiation therapy, the trained machine learning system having been trained using a learned set of parameters to predict the benefit of radiation therapy; and

outputting the benefit of radiation therapy to an electronic storage device.

8. The computer-implemented method of claim 7 , wherein using a learned set of parameters to predict the benefit of radiation therapy comprises:

receiving one or more digital images of a pathology specimen with an associated benefit of radiation therapy value;

receiving additional information about an associated patient and/or an associate disease and transforming the additional information into a vector representation;

identifying a region of the one or more digital images to analyze and removing a non-relevant region of the one or more digital images;

training a machine learning system to predict the benefit of radiation therapy with an input set of learned parameters; and

saving an output set of learned parameters to a digital storage device.

9. The computer-implemented method of claim 8 , wherein the associated benefit of radiation therapy value is:

a continuous number, and training the machine learning system is done with a regression loss; or

a binary number, and training the machine learning system is done with a classification loss.

10. The method of claim 1 , wherein:

the plurality of training images included a plurality of images of pathology specimens associated with a plurality of patients,

the machine learning system was trained using additional information about the plurality of patients and/or one or more diseases associated with the plurality of patients, and

wherein training the machine learning system using the plurality of training images includes transforming the additional information into a vector representation and integrating the vector representation into the prediction by:

concatenating the vector representation into representations within the neural network system to modulate outputs by adopting a probabilistic approach, or

modulating processing of the neural network system using conditional batch normalization.

11. The method of claim 1 , wherein training the machine learning system includes:

receiving the plurality of training images;

receiving additional information;

transforming the additional information into a vector representation; and

concatenating the vector representation into representations within the neural network system to modulate outputs by adopting a probabilistic approach.

12. A system for predicting a benefit of radiation therapy, comprising:

at least one memory storing instructions; and

at least one processor configured to execute the instructions to perform operations comprising:

receiving one or more digital images of a pathology specimen;

receiving additional information about a patient and/or a disease associated with the pathology specimen and transforming the additional information into a vector representation;

identifying a target region of one or more digital images for analysis;

applying a machine learning system to the one or more digital images to predict a benefit of radiation therapy for the target region by integrating the vector representation into a neural network of the machine learning system, the machine learning system having been trained by processing a plurality of training images to predict the benefit of radiation therapy for the target region; and

outputting the predicted benefit of radiation therapy corresponding to the target region to an electronic storage device.

13. The system of claim 12 , wherein the operations further comprise alerting a user with a visual indicator when the benefit of radiation therapy is available.

14. The system of claim 12 , wherein identifying the target region of the one or more digital images for analysis is done by using a machine learning system.

15. The system of claim 12 , wherein identifying the target region of the one or more digital images for analysis is done manually.

16. The system of claim 12 , wherein the operations further comprise:

removing a non-relevant region of the one or more digital images, the target region including at least a portion of a detected tumor and tissue surrounding the detected tumor, and

wherein the benefit of radiation therapy is based on an intrinsic sensitivity of the detected tumor to radiation and/or a probability of radiotherapy-induced toxicity in the tissue surrounding the detected tumor.

17. The system of claim 12 , wherein no target region is identified and an entire one or more digital slides is analyzed.

18. The system of claim 12 , wherein using a plurality of training images to predict the benefit of radiation therapy comprises:

receiving one or more digital images of a pathology specimen;

receiving additional information about an associated patient or an associated disease and transforming the additional information into a vector representation;

identifying a region of the one or more digital images to analyze and removing a non-relevant region;

applying a trained machine learning system to the one or more digital images to predict the benefit of radiation therapy, the trained machine learning system having been trained using a learned set of parameters to predict the benefit of radiation therapy; and

outputting the benefit of radiation therapy to an electronic storage device.

19. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations for predict a benefit of radiation therapy, the operations comprising: receiving one or more digital images of a pathology specimen; receiving additional information about a patient and/or a disease associated with the pathology specimen and transforming the additional information into a vector representation; identifying a target region of one or more digital images for analysis; the target region including at least a portion of a detected tumor and tissue surrounding the detected tumor;

removing a non-relevant region of the one or more digital images;

applying a machine learning system to the one or more digital images to predict a benefit of radiation therapy for the target region by integrating the vector representation into a neural network of the machine learning system, the machine learning system having been trained by processing a plurality of training images to predict the benefit of radiation therapy for the target region; wherein the predicted benefit of radiation therapy is based on an intrinsic sensitivity of the detected tumor to radiation and/or a probability of radiotherapy-induced toxicity in the tissue surrounding the detected tumor; and outputting the predicted benefit of radiation therapy corresponding to the target region to an electronic storage device.