MULTI-RESOLUTION FOUNDATION MODEL FOR PATHOLOGY

1 . A method for training a foundation model for use in pathology, the method comprising:

using a computer hardware processor to perform:

providing an input dataset representing a plurality of pathology images as input to a backbone of the foundation model, wherein the plurality of pathology images comprises patches having different levels of pixel resolution;

producing, with the backbone of the foundation model, a plurality of vector embeddings based on the input dataset;

adjusting weights associated with the backbone of the foundation model based on the plurality of vector embeddings by using a Fourier reconstruction loss function configured to separate portions of the patches in accordance with a high-frequency band and a low-frequency band; and

storing the foundation model on at least one storage device.

2 . The method of claim 1 , wherein the plurality of pathology images are unlabeled such that training the foundation model is performed in an unsupervised fashion.

3 . The method of claim 1 , wherein the patches have at least first and second levels of pixel resolution, wherein:

the first level of pixel resolution is between 0.25 microns per pixel (mpp) and 1 mpp, and

the second level of pixel resolution is between 1 mpp and 2 mpp.

4 . The method of claim 1 , wherein the plurality of pathology images comprises images of multiple different organs.

5 . The method of claim 1 , wherein the plurality of pathology images comprises images associated with multiple different diseases.

6 . The method of claim 1 , wherein the plurality of pathology images comprises images having different types of stains.

7 . The method of claim 1 , wherein the plurality of pathology images comprises images produced with different types of scanners.

8 . The method of claim 1 , wherein the plurality of pathology images comprises images produced with different levels of objective magnification.

9 . The method of claim 1 , wherein the backbone of the foundation model comprises a Flexible Vision Transformer (FlexiViT) backbone.

10 . The method of claim 9 , wherein producing the plurality of vector embeddings comprises training the FlexiVit backbone in accordance with a DINOv2 framework.

11 . The method of claim 1 , wherein the plurality of pathology images comprises at least one pathology image having a first patch having a first level of pixel resolution and a second patch having a second level of pixel resolution different from the first level of pixel resolution.

12 . The method of claim 1 , wherein the plurality of pathology images comprises a first pathology image having at least one patch having a first level of pixel resolution and a second pathology image having at least one patch having a second level of pixel resolution different from the first level of pixel resolution.

13 . The method of claim 1 , wherein the input dataset comprises a plurality of images comprising cropped portions of pathology images, the cropped portions of pathology images comprising cropped portions of a first size and cropped portions of a second size, smaller than the first size.

14 . The method of claim 13 , further comprising:

applying masks to images of the input dataset;

passing a first plurality of masked images of the input dataset to a first encoder of the backbone;

passing a second plurality of masked images of the input data set to a second encoder of the backbone, the second plurality of masked images being smaller than the images of the first plurality of masked images, wherein producing the plurality of vector embeddings comprises producing vector embeddings using the first and second pluralities of masked images using the first and second encoders;

reconstructing, based on the vector embeddings produced by the first and second encoders, masked portions of masked images of the input data set; and

adjusting the weights associated with the backbone of the foundation model based on a loss function determined from the reconstructed masked portions.

15 . The method of claim 14 , wherein:

the reconstructing comprises generating reconstructed pathology images; and

the Fourier loss function is based on patches in the reconstructed pathology images.

16 . The method of claim 1 , further comprising:

fine-tuning an adaptation head of the foundational model, to perform one or more of:

slide-level identification of biological features, tissue-level identification of biological feature, cellular-level identification of biological features, and/or subcellular-level identification of biological features, the fine-tuning comprising:

inputting a fine-tuning dataset comprising a plurality of pathology images to the backbone; and

fine-tuning the adaptation head using vector embeddings generated by the backbone using the fine-tuning dataset.

17 . A method for performing pathology using a foundation model having a backbone and an adaptation head, the method comprising:

using a computer hardware processor to perform:

obtaining one of more input pathology images;

providing the one or more input pathology images to the backbone of the foundation model;

obtaining, from the backbone of the foundation model, a plurality of vector embeddings generated from the one or more input pathology images, wherein the backbone of the foundation model is pre-trained with an input dataset representing a plurality of pathology images, the plurality of pathology images comprising patches having different levels of pixel resolution; and

providing the plurality of vector embeddings of the one or more input pathology images as input to the adaptation head of the foundation model; and

using the foundation model to perform a pathology-related task based on at least a subset of the plurality of vector embeddings.

18 . The method of claim 17 , wherein the plurality of vector embeddings represent portions of the one or more input pathology images having different levels of pixel resolution.

19 . The method of claim 17 , wherein the adaptation head of the foundation model is trained using data representing image annotations obtained from pathologists.

20 . The method of claim 17 , wherein the adaptation head of the foundation model comprises a Multiple Instance Learning (MIL) model.

21 . The method of claim 20 , wherein the adaptation head of the foundation model comprises an Additive MIL classifier.

22 . The method of claim 17 , wherein the one or more input pathology images comprise IHC-stained breast cancer slides, and perform the pathology-related task comprises performing quantification of an HER2 biomarker in the IHC-stained breast cancer slides.

23 . The method of claim 17 , wherein the one or more input pathology images comprise non-small cell lung carcinoma (NSCLC) H&E-stained WSIs, and perform the pathology-related task comprises performing prediction of either Adenocarcinoma or Squamous cell carcinoma in the NSCLC H&E-stained WSIs.