Systems and methods for multiple instance learning for classification and localization in biomedical imaging
The present disclosure is directed to systems and methods for classifying biomedical images. A feature classifier may generate a plurality of tiles from a biomedical image. Each tile may correspond to a portion of the biomedical image. The feature classifier may select a subset of tiles from the plurality of tiles by applying an inference model. The subset of tiles may have highest scores. Each score may indicate a likelihood that the corresponding tile includes a feature indicative of the presence of the condition. The feature classifier may determine a classification result for the biomedical image by applying an aggregation model. The classification result may indicate whether the biomedical includes the presence or lack of the condition.
1. A method, comprising:
identifying, by a computing system, from at least one biomedical image, a first plurality of tiles obtained at a first magnification factor and a second plurality of tiles obtained at a second magnification factor, the biomedical image having at least one feature indicative of a presence or an absence of a condition;
applying, by the computing system, a first machine learning (ML) model to each tile of the first plurality of tiles and the second plurality of tiles to generate a score indicating a likelihood of one of the presence or the absence of the condition in a corresponding tile;
selecting, by the computing system, a first subset of tiles from the first plurality of tiles and a second subset of tiles from the second plurality of tiles based on the score of each tile of the first plurality of tiles and the second plurality of tiles; and
applying, by the computing system, a second machine learning (ML) model to each of the first subset of tiles and the second subset of tiles to determine a classification result for the biomedical image as one of having the presence or the absence of the condition; and
storing, by the computing system, the classification result for the biomedical image.
2. The method of claim 1 , wherein identifying further comprises identifying the first plurality of tiles from a first biomedical image at the first magnification factor and the second plurality of tiles from a second biomedical image at the second magnification factor.
3. The method of claim 1 , wherein applying the first ML model further comprises:
applying a first feature extractor for the first magnification factor to at least one first tile of the first plurality of tiles to generate a corresponding first score for the at least one first tile; and
applying, in conjunction with applying the first feature extractor, a second feature extractor for the second magnification factor to at least one second tile of the second plurality of tiles to generate a corresponding second score for the at least one second tile.
4. The method of claim 1 , wherein applying the second ML model further comprises applying the second ML model to determine, for the biomedical image, a plurality of classification results for a corresponding plurality of conditions.
5. The method of claim 1 , wherein identifying further comprises generating, from the biomedical image, at least one of the first plurality of tiles and the second plurality of tiles in accordance with an overlap ratio.
6. The method of claim 1 , further comprising receiving, by the computing system via an imaging device, the biomedical image of a histological section of a sample from a subject, the sample having at least one portion corresponding to the feature.
7. The method of claim 1 , further comprising providing, by the computing system, for presentation, information based at least on the classification result and the biomedical image.
8. A method, comprising:
identifying, by a computing system, a training dataset comprising a plurality of examples, each example of the plurality of examples identifying: (i) a first plurality of tiles obtained at a first magnification factor and a second plurality of tiles obtained at a second magnification factor from at least one respective biomedical image and (ii) a label indicating a presence or an absence of a condition in the biomedical image;
applying, by the computing system, a first machine learning (ML) model to each tile of the first plurality of tiles and the second plurality of tiles in each example of the training dataset to generate a score indicating a likelihood of one of the presence or the absence of the condition in a corresponding tile;
selecting, by the computing system, a first subset of tiles from the first plurality of tiles and a second subset of tiles from the second plurality of tiles based on the score of each tile of the first plurality of tiles and the second plurality of tiles from each example;
applying, by the computing system, a second machine learning (ML) model to each of the first subset of tiles and the second subset of tiles to determine a classification result for the biomedical image in each example as one of having the presence or the absence of the condition;
comparing, by the computing system, for each example of the plurality of the examples, the classification result for the respective biomedical image with the label indicating the present or the absence of the condition within the biomedical image of a corresponding example;
updating, by the computing system, at least one parameter of the first ML model or the second ML model based at least on the comparison.
9. The method of claim 8 , further comprising:
determining, by the computing system, for each example of the plurality of examples, an error metric between (i) the score of each tile of the first plurality of tiles and the second plurality of tiles and (ii) a baseline value for the corresponding example; and
updating, by the computing system, the at least one parameter of the first ML model based at least on the error metric.
10. The method of claim 8 , further comprising:
determining, by the computing system, for at least one example of the plurality of examples, that the score of each tile of the first subset of tiles or the second subset of tiles satisfies a threshold value identified by the label of the at least one example; and
maintaining, by the computing system, one or more weights of the first ML model responsive to determining that the score satisfies the threshold value.
11. The method of claim 8 , wherein applying the first ML model further comprises, for at least one example of the plurality of examples:
applying a first feature extractor for the first magnification factor to at least one first tile of the first plurality of tiles to generate a corresponding first score for the at least one first tile; and
applying, in conjunction with applying the first feature extractor, a second feature extractor for the second magnification factor to at least one second tile of the second plurality of tiles to generate a corresponding second score for the at least one second tile.
12. The method of claim 8 , wherein applying the second ML model further comprises applying the second ML model to determine, for the respective biomedical image of each example of the plurality of examples, a plurality of classification results for a corresponding plurality of conditions.
13. The method of claim 8 , wherein selecting further comprises identifying a first predefined number of tiles having highest first corresponding scores from the first plurality of tiles and a second predefined number of tiles having highest second corresponding scores from the second plurality of tiles in each example.
14. The method of claim 8 , further comprising removing, by the computing system, for at least one example of the plurality of examples, negative space corresponding to lack of any features within the respective biomedical image, prior to applying the first ML model.
15. A system, comprising:
a computing system having one or more processors coupled with memory, configured to:
identify, from at least one biomedical image, a first plurality of tiles obtained at a first magnification factor and a second plurality of tiles obtained at a second magnification factor, the biomedical image having at least one feature indicative of a presence or an absence of a condition;
apply a first machine learning (ML) model to each tile of the first plurality of tiles and the second plurality of tiles to generate a score indicating a likelihood of one of the presence or the absence of the condition in a corresponding tile, and select a first subset of tiles from the first plurality of tiles and a second subset of tiles from the second plurality of tiles based on the score of each tile of the first plurality of tiles and the second plurality of tiles; and
apply a second machine learning (ML) model to each of the first subset of tiles and the second subset of tiles to determine a classification result for the biomedical image as one of having the presence or the absence of the condition; and
store the classification result for the biomedical image.
16. The system of claim 15 , wherein the computing system is configured to identify the first plurality of tiles from a first biomedical image at the first magnification factor and the second plurality of tiles from a second biomedical image at the second magnification factor.
17. The system of claim 15 , wherein the computing system is configured to apply the first ML model by:
applying a first feature extractor for the first magnification factor to at least one first tile of the first plurality of tiles to generate a corresponding first score for the at least one first tile; and
applying, in conjunction with applying the first feature extractor, a second feature extractor for the second magnification factor to at least one second tile of the second plurality of tiles to generate a corresponding second score for the at least one second tile.
18. The system of claim 15 , wherein the computing system is configured to apply the second ML model to determine, for the biomedical image, a plurality of classification results for a corresponding plurality of conditions.
19. The system of claim 15 , wherein the computing system is configured to receive, via an imaging device, the biomedical image of a histological section of a sample from a subject, the sample having at least one object corresponding to the feature.
20. The system of claim 15 , wherein the computing system is configured to provide, for presentation, information based at least on the classification result and the biomedical image.