System and method for detecting recurrence of a disease

1 . A method for determining a recurrence of a disease in a patient, the method comprising:

generating a plurality of medical images of an organ of the patient from at least a first modality and a second modality, and wherein the plurality of medical images comprises X-ray images, Hematoxylin and Eosin (H&E) biopsy sample images, molecular images, Positron emission tomography (PET) scans images, ultrasound images, Magnetic resonance imaging (MRI) scan images or combinations thereof;

determining a plurality of recurrence probabilities from the plurality of medical images; and

determining a recurrence of the disease based on the plurality of recurrence probabilities and clinicopathological data of the patient using a Bayesian network,

wherein the plurality of recurrence probabilities includes a first recurrence probability determined by a pathomics learning model and a second recurrence probability determined by a radiomics learning model, and

wherein determining the first recurrence probability comprises extracting fixed image patches of predefined pixels from H&E biopsy sample images and automatically mapping each of the fixed image patches to an initial latent space, and providing the initial latent space to a dense network that classifies the initial latent space into aggressive patches and non-aggressive patches, and wherein the first recurrence probability is based on the aggressive and non-aggressive patches, and

wherein determining the second recurrence probability comprises extracting and analyzing a plurality of radiomics features from an invasive edge surrounding the disease observed in routine mammogram images.

2 . The method of claim 1 , wherein the plurality of recurrence probabilities includes a third recurrence probability.

3 . The method of claim 2 , wherein the third recurrence probability is determined based on in situ imaging on a small set of tissue images of the patient.

4 . The method of claim 1 , wherein determining the first recurrence probability further comprises refining the latent space by encoding patches generated by a generative adversarial network (GAN) model that captures features of aggressive cancers.

5 . The method of claim 4 , wherein determining the first recurrence probability further comprises using a deep learning (DL) network to predict the first recurrence probability based on the refined latent space.

6 . The method of claim 1 , wherein the clinicopathological data of the patient includes age, size, location, laterality and Lymph node positivity of the disease.

7 . The method of claim 6 , wherein the plurality of recurrence probabilities and clinicopathological data values represent a plurality of nodes in the Bayesian network and the Bayesian network determines the disease recurrence, depending on the node probability values and conditional probabilities between the nodes.

8 . A system comprising:

a memory;

a display device; and

a processor communicably coupled to the memory and configured to:

generate a plurality of medical images of an organ of the patient from at least a first modality and a second modality, and wherein the plurality of medical images comprises X-ray images, Hematoxylin and Eosin (H&E) biopsy sample images, molecular images, Positron emission tomography (PET) scans images, ultrasound images, Magnetic resonance imaging (MRI) scan images or combinations thereof;

determine a plurality of recurrence probabilities from the plurality of medical images; and

determine a recurrence of the disease based on the plurality of recurrence probabilities and clinicopathological data of the patient using a Bayesian network,

wherein the plurality of recurrence probabilities includes a first recurrence probability determined by a pathomics learning model and a second recurrence probability determined by a radiomics learning model, and

wherein determining the first recurrence probability comprises extracting fixed image patches of predefined pixels from H&E biopsy sample images and automatically mapping each of the fixed image patches to an initial latent space, and providing the initial latent space to a dense network that classifies the initial latent space into aggressive patches and non-aggressive patches, and wherein the first recurrence probability is based on the aggressive and non-aggressive patches, and

wherein determining the second recurrence probability comprises extracting and analyzing a plurality of radiomics features from an invasive edge surrounding the disease observed in routine mammogram images.

9 . The system of claim 8 , wherein the pathomics model further includes a generative adversarial network (GAN) model that generates synthetic image samples to further refine the latent space.

10 . The system of claim 9 , wherein the pathomics model comprises a deep learning (DL) network to predict the first recurrence probability based on the refined latent space.

11 . The system of claim 8 , wherein the plurality of recurrence probabilities and clinicopathological data values represent a plurality of nodes in the Bayesian network and the Bayesian network determines the disease recurrence, depending on the node probability values and conditional probabilities between the nodes.