IP Library Granted Patent US 11,538,155
Granted Patent B2
US 11,538,155 · App. 17/074,293 · Granted Dec 27, 2022

Systems and methods for multiple instance learning for classification and localization in biomedical imaging

Inventors: Thomas Fuchs (New York, NY); Gabriele Campanella (New York, NY)
Assignee: MEMORIAL SLOAN KETTERING CANCER CENTER
G06T7/0012G06K9/623G06K9/628G06K9/6262G06K9/6277G06N20/00G06V10/98G16H30/40G16H50/70G06T2207/10056G06T2207/20076G06T2207/20081G06T2207/20084G06T2207/30096G06V2201/03
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,538,155
App. No.
17/074,293
Granted
Dec 27, 2022
Kind
B2
Abstract

The present disclosure is directed to systems and methods for classifying biomedical images. A feature classifier may generate a plurality of tiles from a biomedical image. Each tile may correspond to a portion of the biomedical image. The feature classifier may select a subset of tiles from the plurality of tiles by applying an inference model. The subset of tiles may have highest scores. Each score may indicate a likelihood that the corresponding tile includes a feature indicative of the presence of the condition. The feature classifier may determine a classification result for the biomedical image by applying an aggregation model. The classification result may indicate whether the biomedical includes the presence or lack of the condition.

Claims (32)

1. A method of selecting tiles from biomedical images, comprising:

identifying, by a computing system, a plurality of tiles from a biomedical image subsequent to removal of negative space corresponding to lack of any features within the biomedical image, the biomedical image having at least one feature indicative of a presence or an absence of a condition within one or more portions;

applying, by the computing system, a trained inference model having a plurality of parameters to the plurality of tiles to determine a score for each tile of the plurality of tiles of the biomedical image, the score indicating a likelihood of one of the presence or the absence of the condition within a corresponding tile;

selecting, by the computing system, a subset of tiles from the plurality of tiles based on the score determined for each tile using the trained inference model; and

providing, by the computing system, the subset of tiles corresponding to the one or more portions of the biomedical image having the at least one feature.

2. The method of claim 1 , wherein selecting further comprises selecting, from the plurality of tiles, a first subset of tiles corresponding to a presence of a first condition and a second subset of tiles corresponding to a presence of a second condition.

3. The method of claim 1 , wherein selecting further comprises identifying a predefined number of tiles from the plurality of tiles, the predefined number of tiles having highest corresponding scores among a plurality of scores determined for the corresponding plurality of tiles.

4. The method of claim 1 , wherein applying further comprises applying the trained in ference model to the plurality of tiles, the trained inference model lacking an internal state memory to process the plurality of tiles without any sequence dependence in processing the plurality of tiles.

5. The method of claim 1 , wherein providing further comprises providing the subset of tiles to an aggregation model to determine a classification result for the biomedical image from which the subset of tiles are selected, the classification result indicating one of the biomedical image having the at least one feature corresponding to the presence of the condition or the biomedical image lacking any features corresponding to the presence of the condition.

6. The method of claim 1 , further comprising establishing, by the computing system, the trained inference model using a sample biomedical image and a label indicating one of the presence or the absence of the condition within one or more second portions within the sample biomedical image.

7. A method of comprising:

identifying, by a computing system, a plurality of tiles of a biomedical image subsequent to removal of negative space corresponding to lack of any features within the biomedical image, the biomedical image having one of a presence or an absence of a condition within one or more portions;

applying, by the computing system, a trained aggregation model having a plurality of parameters to the plurality of tiles in sequence;

determining, by the computing system, from applying the trained aggregation model to the plurality of tiles in sequence, a classification result indicating one of the biomedical image having at least one feature corresponding to the presence of the condition or the biomedical image corresponding to the absence of the condition; and

providing, by the computing system, the classification result for the biomedical image.

8. The method of claim 7 , wherein identifying further comprises identifying the plurality of tiles selected by an inference model based on a score for each tile of the plurality of tiles from the biomedical image, the score indicating a likelihood of one of the presence or the absence of the condition.

9. The method of claim 7 , wherein identifying further comprises identifying the plurality of tiles including sets of tiles from the biomedical image at a corresponding plurality of magnification factors.

10. The method of claim 7 , wherein applying further comprises applying the aggregation model to the plurality of tiles in accordance with sequence, the sequence defining an order of feeding the plurality of tiles based on a coordinate for each tile in the plurality of tiles.

11. The method of claim 7 , wherein determining further comprises determining a plurality of classification results for a corresponding plurality of conditions, each of the plurality of classification results indicating one of the biomedical image having the at least one feature corresponding to the presence of a corresponding condition of the plurality of conditions or the biomedical image lacking the presence of the corresponding condition.

12. The method of claim 7 , wherein providing further comprises providing, for display, the biomedical image and at least one visual indicator identifying the one or more portions within the biomedical image indicative of the classification result.

13. The method of claim 7 , further comprising establishing, by the computing system, the trained aggregation model using a sample biomedical image and a label indicating one of the presence or the absence of the condition within second one or more portions within the sample biomedical image.

14. A system for classifying biomedical images, comprising:

a computing system having one or more processors coupled with memory, configured to:

identify a plurality of tiles of a biomedical image subsequent to removal of negative space corresponding to lack of any features within the biomedical image, the biomedical image having one of a presence or an absence of a condition within one or more portions;

apply a trained aggregation model having a plurality of parameters to the plurality of tiles in sequence;

determine, from applying the trained aggregation model to the plurality of tiles in sequence, a classification result indicating one of the biomedical image having at least one feature corresponding to the presence of the condition or the biomedical image corresponding to the absence of the condition; and

provide the classification result for the biomedical image.

15. The system of claim 14 , wherein the computing system is further configured to identify the plurality of tiles selected by an inference model based on a score for each tile of the plurality of tiles from the biomedical image, the score indicating a likelihood of one of the presence or the absence of the condition.

16. The system of claim 14 , wherein the computing system is further configured to apply the aggregation model to the plurality of tiles in accordance with the sequence, the sequence defining an order of feeding the plurality of tiles based on a coordinate for each tile in the plurality of tiles.

17. The system of claim 14 , wherein the computing system is further configured to determine a plurality of classification results for a corresponding plurality of conditions, each of the plurality of classification results indicating one of the biomedical image having the at least one feature corresponding to the presence of a corresponding condition of the plurality of conditions or the biomedical image lacking the presence of the corresponding condition.

18. The system of claim 14 , wherein the computing system is further configured to provide, for display, the biomedical image and at least one visual indicator identifying the one or more portions within the biomedical image indicative of the classification result.

19. The system of claim 14 , wherein the computing system is further configured to establish the trained aggregation model using a sample biomedical image and a label indicating one of the presence or the absence of the condition within one or more second portions within the sample biomedical image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2023
From: FUCHS, THOMAS; CAMPANELLA, GABRIELE
To: MEMORIAL SLOAN KETTERING CANCER CENTER
Reel/Frame 063953/0681 →
Continuity (5)
Continuation 16599992 · Oct 11, 2019
Continuation 16362470 · Mar 22, 2019
Provisional Application 62670432 · May 11, 2018
Provisional Application 62647002 · Mar 23, 2018
Related Publication 20210049763A1 · Feb 18, 2021
Cited By (3)
US 12,288,603 US 12,347,569 US 12,725,691