AI-driven system and methods for 3D medical image disease detection
The disclosure provides a unified system and method for analyzing 3D medical images through probabilistic masking, multimodal embedding generation, anomaly detection, and reinforcement learning-based refinement. The methods comprise applying ailment-aware probabilistic masking to 3D medical scans using prior distributions of known abnormalities to retain diagnostically relevant voxel regions. The masked scans are encoded into latent vectors using a vision transformer encoder, while associated textual medical reports are encoded using a language model; both are mapped into a shared embedding space. Anomaly detection is performed through (1) a temporal comparison method based on differences in patient embeddings across time, and (2) a similarity-based method comparing current embeddings to a database of known abnormal cases using cosine similarity. Reinforcement learning is applied to refine embeddings using human expert feedback, including corrected annotations and textual clarifications, allowing dynamic adjustment of thresholds and encoder parameters. This adaptive framework supports efficient, scalable, and accurate medical diagnosis across a variety of imaging modalities.
1 . A method for masking 3D medical images, comprising:
receiving a plurality of 3D medical image volumes formatted in DICOM, NIfTI, or equivalent formats;
computing a voxel-wise prior probability distribution over anatomical regions by aggregating voxel intensities from the plurality of image volumes;
generating a spatial mask that selectively retains voxels having prior probability values P(v) greater than a threshold, such voxels representing regions with elevated likelihood of containing pathological features as determined from aggregated historical imaging data; and
applying the spatial mask to each new 3D image to reduce dimensionality and focus downstream processing on regions associated with pathological abnormalities including tumors, lesions, fractures, or structural deviations.
2 . The method of claim 1 , wherein computing the voxel-wise prior probability distribution comprises:
performing an element-wise summation of voxel intensities across a dataset of spatially aligned 3D medical image volumes, followed by normalization of the summed voxel intensities to produce a probability density function whose total voxel intensity values sum to one.
3 . The method of claim 1 , wherein applying the spatial mask comprises:
applying the binary mask to each new 3D medical image via element-wise multiplication to preserve voxel intensities in said regions associated with pathological abnormalities.
4 . The method of claim 1 , wherein the spatial mask is ailment-aware and reflects prior knowledge of anatomical regions affected by specific medical conditions, such that applying the spatial mask to each new 3D image functions as a three-dimensional attention mechanism directing input emphasis toward said regions associated with pathological abnormalities when the masked image is processed by a downstream machine learning model.
5 . A method for encoding medical data into a shared latent space, comprising:
encoding a masked 3D medical image prepared according to the method of claim 1 into a first embedding vector using a vision transformer-based encoder;
encoding an associated textual medical report into a second embedding vector using a language model-based encoder;
projecting both the first and second embedding vectors into a shared latent vector space; and
storing a resulting multimodal embedding in a vector database for downstream comparison and retrieval.
6 . The method of claim 5 , wherein the vision transformer-based encoder is trained jointly with a decoder network to reconstruct the 3D medical image from said first embedding vector, wherein training includes masked input images generated by ailment-aware spatial masking, and the decoder is used only during training.
7 . The method of claim 5 , wherein the encoder is trained using a composite loss function comprising:
a reconstruction loss that minimizes a difference between an original 3D medical image and an image decoded from the first embedding vector;
a Kullback-Leibler divergence term that regularizes a latent space distribution; and
an edge-preservation loss that applies a 3D Sobel operator to emphasize regions where voxel intensity exhibits gradient changes as measured using a 3D Sobel edge detection operator.
8 . The method of claim 5 , further comprising projecting the textual embedding and the 3D image embedding into a common latent space using modality-specific projection weights, and combining the projected textual embedding and the projected 3D image using a weighted summation to produce a final multimodal embedding vector.
9 . The method of claim 5 , comprising:
a. receiving human-annotated segmentation masks indicating corrected regions of interest in a 3D medical image;
b. receiving textual corrections to associated medical reports, the corrections comprising diagnostic descriptions;
c. computing reward signals based on overlap between predicted segmentations and corrected masks, and based on semantic similarity between original and corrected textual descriptions;
d. updating encoder parameters or embedding vector representations based on computed reward signals using a reinforcement learning update rule; and
repeating steps a-d to incrementally refine a latent representation space over time with continued expert interaction.
10 . The method of claim 9 , wherein the reward signal for textual feedback is computed using term frequency-inverse document frequency (TF-IDF) weighted cosine similarity between system-generated descriptions and human-provided corrected descriptions, such that a higher similarity corresponds to a higher reward.
11 . The method of claim 9 , wherein the reward signal for image-based feedback is computed by measuring spatial overlap between a predicted segmentation mask and a human-annotated segmentation mask using a similarity function selected from a group consisting of Dice coefficient, Intersection over Union, and binary overlap.
12 . The method of claim 9 , wherein the multimodal embedding is updated using a reinforcement learning update rule that adjusts the multimodal embedding via a gradient ascent step on an expected reward, and
wherein the reward signal is computed as a weighted combination of textual similarity and segmentation overlap based on human-provided feedback, such that both clinical descriptions and image annotations contribute to the embedding refinement.