SYSTEMS AND METHODS TO PROCESS ELECTRONIC IMAGES FOR DETERMINING TREATMENT

1 . A computer-implemented method for processing digital pathology images to determine a treatment for one or more patients, comprising:

receiving a plurality of medical images of at least one pathology specimen, the pathology specimen being associated with a patient;

receiving metadata corresponding to the plurality of medical images, the metadata comprising data regarding previous medical treatment of the patient;

providing the medical images and metadata as input to a machine learning system, the machine learning system having been trained by receiving as input historical treatment information and digital images labeled with a predicted treatment regimen; and

outputting, by the machine learning system, a treatment effectiveness assessment.

2 . The method of claim 1 , further comprising:

providing the plurality of medical images and metadata to a trained embedding system capable of outputting a single embedding that may be received by the machine learning system.

3 . The method of claim 2 , the trained embedding system performing steps comprising inferring one or more missing data points to construct a universal embedding, the universal embedding being received by the trained machine learning system.

4 . The method of claim 1 , wherein the trained system outputs the plurality of medical images with marking to display the predicted effects of the treatment effectiveness assessment.

5 . The method of claim 1 , wherein the metadata may further include information describing a tissue type of the pathology specimen for the medical specimen.

6 . The method of claim 1 , further comprising:

inputting the received medical images that correspond to previously treated medical specimen into a second trained system; and

determining a score to measure the effectiveness of past treatment, wherein the score defines a damage of previously healthy slides and additional damage to previously cancerous regions of the inputted slides.

7 . The method of claim 1 , wherein the treatment effectiveness assessment comprises a treatment type and a treatment dosage for the patient.

8 . A system for processing electronic medical images, the system comprising:

at least one memory storing instructions; and

at least one processor configured to execute the instructions to perform operations comprising:

receiving a plurality of medical images of at least one pathology specimen, the pathology specimen being associated with a patient;

receiving metadata corresponding to the plurality of medical images, the metadata comprising data regarding previous medical treatment of the patient;

providing the medical images and metadata as input to a machine learning system, the machine learning system having been trained by receiving as input historical treatment information and digital images labeled with a predicted treatment regimen; and

outputting, by the machine learning system, a treatment effectiveness assessment.

9 . The system of claim 8 , further comprising:

providing the plurality of medical images and metadata to a trained embedding system capable of outputting a single embedding that may be received by the machine learning system.

10 . The system of claim 9 , the trained embedding system performing steps comprising inferring one or more missing data points to construct a universal embedding, the universal embedding being received by the trained machine learning system.

11 . The system of claim 8 , wherein the trained system outputs the plurality of medical images with marking to display the predicted effects of the treatment effectiveness assessment.

12 . The system of claim 8 , wherein the metadata may further include information describing a tissue type of the pathology specimen for the medical specimen.

13 . The system of claim 8 , further comprising:

inputting the received medical images that correspond to previously treated medical specimen into a second trained system; and

determining a score to measure the effectiveness of past treatment, wherein the score defines a damage of previously healthy slides and additional damage to previously cancerous regions of the inputted slides.

14 . The system of claim 8 , wherein the treatment effectiveness assessment comprises a treatment type and a treatment dosage for the patient.

15 . A non-transitory computer-readable medium storing instructions that, when executed by a processor, perform operations processing electronic medical images, the operations comprising:

receiving a plurality of medical images of at least one pathology specimen, the pathology specimen being associated with a patient;

receiving metadata corresponding to the plurality of medical images, the metadata comprising data regarding previous medical treatment of the patient;

providing the medical images and metadata as input to a machine learning system, the machine learning system having been trained by receiving as input historical treatment information and digital images labeled with a predicted treatment regimen; and

outputting, by the machine learning system, a treatment effectiveness assessment.

16 . The computer-readable medium of claim 15 , further comprising:

providing the plurality of medical images and metadata to a trained embedding system capable of outputting a single embedding that may be received by the machine learning system.

17 . The computer-readable medium of claim 16 , the trained embedding system performing steps comprising inferring one or more missing data points to construct a universal embedding, the universal embedding being received by the trained machine learning system.

18 . The computer-readable medium of claim 15 , wherein the trained system outputs the plurality of medical images with marking to display the predicted effects of the treatment effectiveness assessment.

19 . The computer-readable medium of claim 15 , wherein the metadata may further include information describing a tissue type of the pathology specimen for the medical specimen.

20 . The computer-readable medium of claim 15 , further comprising:

inputting the received medical images that correspond to previously treated medical specimen into a second trained system; and

determining a score to measure the effectiveness of past treatment, wherein the score defines a damage of previously healthy slides and additional damage to previously cancerous regions of the inputted slides.