Multi-scale tumor cell detection and classification
Methods and systems for training a machine learning model include generating pairs of training pixel patches from a dataset of training images, each pair including a first patch representing a part of a respective training image, and a second patch, centered at the same location as the first, representing a larger part of the training image, being resized to a same size of as the first patch. A detection model is trained using the first pixel patches, to detect and locate cells in the images. A classification model is trained using the first pixel patches, to classify cells according to whether the detected cells are cancerous, based on cell location information generated by the detection model. A segmentation model is trained using the second pixel patches, to locate and classify cancerous arrangements of cells in the images.
1. A method for detecting cancer, comprising:
generating, from a scammed tissue sample, a pair of input images, a first input image representing a part of the scanned tissue sample, and a second input image, centered at the same location as the first, representing a larger part of the sample and resized to a size of the first input image, using a hardware processor;
locating cells within the first input image using a detection model;
training a classification model with training data, wherein the training data includes an annotated set of regions of interest from a set of patients' scanned tissue sample slides, representative of a particular condition as encountered in clinical practice;
classifying detected cells in the first input image, using the classification model, to determine a respective cancer probability for each cell, based on cell location information generated by the detection model:
segmenting the second input image, using a segmentation model, to assign a cancerous-structure probability to arrangements of cells;
determining an updated cancerous probability of each detected cell in a first pixel patch, using a weighted sum of the cancer probability for each cell and the cancerous-structure probability;
determining a tumor cell ratio based on the updated cancerous probability of each detected cell in the first pixel patch;
indicating a type and extent of a cancer based on the tumor cell ratio; and
treating the cancer in accordance with the identified type and extent.
2. The method of claim 1 , wherein the tumor cell ratio compares a number of healthy cells to a number of cancerous cells.
3. The method of claim 1 , wherein the detection model outputs a density map of Gaussian peaks, each centered at a respective cell.
4. The method of claim 1 , wherein the classification model outputs a density map of Gaussian peaks, each centered at a respective cancerous cell.
5. The method of claim 1 , wherein the segmentation model outputs a regression map that identifies areas that are cancerous.
6. The method of claim 1 , wherein a cancer probability for each cell, determined by the classification model, is added to a cancerous structure probability, determined by the segmentation model, in a weighted sum to determine an updated cancerous probability for each cell.
7. The method of claim 1 , wherein the detection model, the classification model, and the segmentation model are implemented as fully-convolutional neural network models.
8. A system for detecting cancer, comprising:
a hardware processor; and
a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to:
generate, from a scanned tissue sample, a pair of input images, a first input image representing a part of the scanned tissue sample, and a second input image, centered at the same location as the first, representing a larger part of the sample and resized to a size of the first input image;
locate cells within the first input image using a detection model;
train a classification model with training data, wherein the training data includes an annotated set of regions of interest from a set of patients' scanned tissue sample slides, represent ative of a particular condition as encountered in clinical practice;
classify detected cells in the first input image, using the classification model, to determine a respective cancer probability for each cell, based on cell location information generated by the detection model;
segment the second input image, using a segmentation model, to assign a cancerous-structure probability to arrangements of cells;
determine an updated cancerous probability of each detected cell in a first pixel patch, using a weighted sum of the cancer probability for each cell and the cancerous-structure probability;
determine a tumor cell ratio based on the updated cancerous probability of each detected cell in the first pixel patch;
indicate a type and extent of a cancer based on the tumor cell ratio; and
trigger treatment of the cancer in accordance with the identified type and extent.
9. The system of claim 8 , wherein the tumor cell ratio compares a number of healthy cells to a number of cancerous cells.
10. The system of claim 8 , wherein the detection model outputs a density map of Gaussian peaks, each centered at a respective cell.
11. The system of claim 8 , wherein the classification model outputs a density map of Gaussian peaks, each centered at a respective cancerous cell.
12. The system of claim 8 , wherein the segmentation model outputs a regression map that identifies areas that are cancerous.
13. The system of claim 8 , wherein a cancer probability for each cell, determined by the classification model, is added to a cancerous structure probability, determined by the segmentation model, in a weighted sum to determine an updated cancerous probability for each cell.
14. The system of claim 8 , wherein the detection model, the classification model, and the segmentation model are implemented as fully-convolutional neural network models.