Systems and methods for processing images to determine image-based computational biomarkers from liquid specimens
A method of using a machine learning model to output a task-specific prediction may include receiving a digitized cytology image of a cytology sample and applying a machine learning model to isolate cells of the digitized cytology image. The machine learning model may include identifying a plurality of sub-portions of the digitized cytology image, identifying, for each sub-portion of the plurality of sub-portions, either background or cell, and determining cell sub-images of the digitized cytology image. Each cell sub-image may comprise a cell of the digitized cytology image, based on the identifying either background or cell. The method may further comprise determining a plurality of features based on the cell sub-images, each of the cell sub-images being associated with at least one of the plurality of features, determining an aggregated feature based on the plurality of features, and training a machine learning model to predict a target task based on the aggregated feature.
1. A computer-implemented method of using a machine learning model to output a task-specific prediction, the method comprising:
receiving a digitized cytology image of a cytology sample;
applying a machine learning model to isolate cells of the digitized cytology image, comprising:
identifying a plurality of sub-portions of the digitized cytology image; and
determining cell sub-images of the digitized cytology image, each cell sub-image comprising a cell of the digitized cytology image, based on isolating a set of individual images of cells within the digitized cytology image using a segmentation system and/or a detection system;
determining a plurality of features based on the cell sub-images, each of the cell sub-images being associated with at least one of the plurality of features;
determining an aggregated feature based on the plurality of features; and
training a machine learning model to determine a task-specific prediction based on the aggregated feature.
2. The computer-implemented method of claim 1 , further comprising identifying, for each sub-portion of the plurality of sub-portions, either background or cell.
3. The computer-implemented method of claim 2 , wherein using the segmentation system comprises:
identifying one or more individual pixels in the digitized cytology image as belonging to a cell or background;
classifying one or more cell regions, of a cell of interest, containing the one or more individual pixels into at least one granular structure; and
outputting the cell sub-images tightly bounded around the cell of interest within the digitized cytology image.
4. The computer-implemented method of claim 2 , wherein identifying either background or cell comprises using a detection system to identify bounding regions for each of the cell sub-images.
5. The computer-implemented method of claim 1 , further comprising using the aggregated feature to train a machine learning classifier to predict necessary quantifications for a target task on a per-cell level.
6. The computer-implemented method of claim 1 , wherein determining the aggregated feature comprises computing statistics of per-cell classifications, estimating feature means or cluster centers, and/or training deep learning aggregators that combine per-cell features through convolutional or recurrent mechanisms.
7. The computer-implemented method of claim 1 , wherein the task-specific prediction comprises a prediction that the cytology sample contains enough material for a user to categorize the cytology sample.
8. The computer-implemented method of claim 1 , wherein the task-specific prediction comprises a predicted classification of a specimen diagnostic category from the aggregated feature.
9. The computer-implemented method of claim 1 , wherein the task-specific prediction comprises a probability of detecting a specific mutation within a cytology specimen.
10. A system for using a machine learning model to output a task-specific prediction, the system comprising:
at least one memory storing instructions; and
at least one processor configured to execute the instructions to perform operations comprising:
receiving a digitized cytology image of a cytology sample;
applying a machine learning model to isolate cells of the digitized cytology image, comprising:
identifying a plurality of sub-portions of the digitized cytology image; and
determining cell sub-images of the digitized cytology image, each cell sub-image comprising a cell of the digitized cytology image, based on isolating a set of individual images of cells within the digitized cytology image using a segmentation system and/or a detection system;
determining a plurality of features based on the cell sub-images, each of the cell sub-images being associated with at least one of the plurality of features;
determining an aggregated feature based on the plurality of features; and
training a machine learning model to determine a task-specific prediction based on the aggregated feature.
11. The system of claim 10 , wherein the operations further comprise identifying, for each sub-portion of the plurality of sub-portions, either background or cell.
12. The system of claim 11 , wherein using the segmentation system comprises:
identifying one or more individual pixels in the digitized cytology image as belonging to a cell or background;
classifying one or more cell regions, of a cell of interest, containing the one or more individual pixels into at least one granular structure; and
outputting the cell sub-images tightly bounded around the cell of interest within the digitized cytology image.
13. The system of claim 11 , wherein identifying either background or cell comprises using a detection system to identify bounding regions for each of the cell sub-images.
14. The system of claim 10 , wherein the operations further comprise using the aggregated feature to train a machine learning classifier to predict necessary quantifications for a target task on a per-cell level.
15. The system of claim 10 , wherein determining the aggregated feature comprises computing statistics of per-cell classifications, estimating feature means or cluster centers, and/or training deep learning aggregators that combine per-cell features through convolutional or recurrent mechanisms.
16. The system of claim 10 , wherein the task-specific prediction comprises a prediction that cytology sample contains enough material for a user to categorize the cytology sample.
17. The system of claim 10 , wherein the task-specific prediction comprises a predicted classification of a specimen diagnostic category from the aggregated feature.
18. The system of claim 10 , wherein the machine learning model comprises a support vector machine, logistic regression, and/or a regression or classification-based neural network.
19. A non-transitory computer-readable medium storing instructions that, when executed by a processor, perform a method of using a machine learning model to output a task-specific prediction, the method comprising:
receiving a digitized cytology image of a cytology sample;
applying a machine learning model to isolate cells of the digitized cytology image, comprising:
identifying a plurality of sub-portions of the digitized cytology image; and
determining cell sub-images of the digitized cytology image, each cell sub-image comprising a cell of the digitized cytology image, based on isolating a set of individual images of cells within the digitized cytology image using a segmentation system and/or a detection system;
determining a plurality of features based on the cell sub-images, each of the cell sub-images being associated with at least one of the plurality of features;
determining an aggregated feature based on the plurality of features; and
training a machine learning model to determine a task-specific prediction based on the aggregated feature.
20. The non-transitory computer-readable medium of claim 19 , the method further comprising identifying, for each sub-portion of the plurality of sub-portions, either background or cell.