Method and apparatus for analyzing pathological slide images
A computing apparatus includes a memory storing at least one program and a processor configured to perform at least one operation by executing the at least one program, wherein the processor is further configured to analyze a pathological slide image to classify at least one of cells and tissues included in the pathological slide image into at least one type, segment the pathological slide image into subpatches on the basis of a result of the classification, and analyze the subpatches to output information regarding components of a cell included in each of the subpatches.
1 . A computing apparatus comprising:
a memory storing at least one program; and
a processor configured to perform at least one operation by executing the at least one program,
wherein the processor is further configured to:
analyze a pathological slide image to classify at least one of cells and tissues included in the pathological slide image into at least one type;
segment the pathological slide image into subpatches based on a result of the classification;
analyze the subpatches to output information regarding components of a cell included in each of the subpatches;
generate, by using a machine learning model, subcell information including a first index and a second index for at least one of the components determined based on information regarding the subpatches and the components;
calculate a third index corresponding to a cell membrane specificity of the at least one of cells and tissues included in the pathological slide image based on the second index, wherein the cell membrane specificity indicates how much of a targeted object is present in a cell membrane of the cell;
control a display apparatus to display information regarding the third index of the at least one of cells and tissues;
control the display apparatus to display at least one of a visualization result of the second index for at least one of components of each of cells included in the pathological slide image and a visualization result of the first index for any one of the components;
control the display apparatus to display a graph indicating a distribution of scores corresponding to a staining intensity of at least one of a membrane, cytoplasm, and a cell nucleus of each of the cells included in the pathological slide image; and
control the display apparatus to replace a heatmap generated based on a score corresponding to the second index with visualization results of components of each of the cells and display the visualization result, based on an enlargement magnification set according to a user input being a preset magnification or more,
wherein the visualization results of the components of each of the cells comprise at least one of a first visualization element in which an outline corresponding to a cell membrane of each of the cells is displayed in a certain color and a second visualization element in which a figure generated based on a class corresponding to the first index being displayed at a central point of each of the cells.
2 . The computing apparatus of claim 1 , wherein the first index comprises a class corresponding to a level of staining of the cell membrane of the cell, and the second index comprises at least one of a first score corresponding to a staining intensity of the cell membrane of the cell, a second score corresponding to a staining intensity of cytoplasm of the cell, and a third score corresponding to a staining intensity of a cell nucleus of the cell.
3 . The computing apparatus of claim 2 , wherein the processor is further configured to:
convert a format of the subpatches;
calculate a histogram of each of color channels for the cell membrane, the cytoplasm, and the cell nucleus included in the converted subpatches; and
calculate, by using the machine learning model, at least one of the first score, the second score, and the third score based on a combination of the histograms of the color channels.
4 . The computing apparatus of claim 1 , wherein the first index further comprises at least one of a first class indicating that the cell membrane is negative or unstained, a second class indicating that the cell membrane is partially stained, or a third class indicating that the cell membrane is completely stained.
5 . The computing apparatus of claim 1 , wherein the processor is further configured to control the display apparatus to overlay and display the heatmap generated based on the score corresponding to the second index, on a screen on which at least a portion of the pathological slide image is output.
6 . The computing apparatus of claim 1 , wherein the processor is further configured to control the display apparatus to display an object adjusting a threshold value determining, based on a score corresponding to the second index, whether or not at least one of the components of the cell is stained, and to update and display the visualization result of the second index based on a user input adjusting the threshold value through the object, and to update the visualization result of the second index, the processor is further configured to control the display apparatus to update and display at least one of a tumor proportion score (TPS) and a combined positive score (CPS) corresponding to the pathological slide image, and statistics according to types of the cells included in the pathological slide image as the threshold value is adjusted.
7 . The computing apparatus of claim 1 , wherein an area corresponding to the one of the cells comprises an area within the outline corresponding to a cell membrane of the one of the cells, and
wherein the processor is further configured to, based on the indicator being located on the area within the outline, control the display apparatus to display a separate window comprising information regarding the one of the cells, and the information regarding the one of the cells comprises at least one of a staining intensity of a cell membrane of the one of the cells, a staining intensity of the cytoplasm of the one of the cells, a type of the one of the cells, and the level of staining of the cell membrane of the one of the cells.
8 . The computing apparatus of claim 1 , wherein the processor is further configured to control the display apparatus to display an analysis result of an anti-cancer target or a prediction result of a treatment response based on the third index.
9 . A method of analyzing a pathological slide image, the method comprising:
analyzing the pathological slide image to classify at least one of cells and tissues included in the pathological slide image into at least one type;
segmenting the pathological slide image into subpatches based on a result of the classification;
generating information regarding components of a cell included in each of the subpatches by analyzing the subpatches;
generating, by using a machine learning model, subcell information including a first index and a second index for at least one of the components determined based on information regarding the subpatches and the components,
calculating a third index corresponding to a cell membrane specificity of the at least one of cells included in the pathological slide image based on the second index, wherein the cell membrane specificity indicates how much of a targeted object is present in a cell membrane of the cell;
controlling a display apparatus to display information regarding the third index of the at least one of cells and tissues;
controlling the display apparatus to display at least one of a visualization result of the second index for at least one of components of each of cells included in the pathological slide image and a visualization result of the first index for any one of the components;
controlling the display apparatus to display a graph indicating a distribution of scores corresponding to a staining intensity of at least one of a membrane, cytoplasm, and a cell nucleus of each of the cells included in the pathological slide image; and
controlling the display apparatus to replace a heatmap generated based on a score corresponding to the second index with visualization results of components of each of the cells and display the visualization result, based on an enlargement magnification set according to a user input being a preset magnification or more,
wherein the visualization results of the components of each of the cells comprise at least one of a first visualization element in which an outline corresponding to a cell membrane of each of the cells is displayed in a certain color and a second visualization element in which a figure generated based on a class corresponding to the first index being displayed at a central point of each of the cells.
10 . The method of claim 9 , wherein the first index comprises a class corresponding to a level of staining of the cell membrane of the cell, and the second index comprises at least one of a first score corresponding to a staining intensity of the cell membrane of the cell, a second score corresponding to a staining intensity of cytoplasm of the cell, and a third score corresponding to a staining intensity of a cell nucleus of the cell.
11 . The method of claim 10 , wherein the outputting comprises:
converting a format of the subpatches;
calculating a histogram of each of color channels for a cell membrane, cytoplasm, and the cell nucleus included in the converted subpatches; and
calculating, by using the machine learning model, at least one of the first score, the second score, and the third score based on a combination of the histograms of the color channels.
12 . The method of claim 9 , wherein the first index further comprises at least one of a first class indicating that the cell membrane is negative or unstained, a second class indicating that the cell membrane is partially stained, or a third class indicating that the cell membrane is completely stained.
13 . The method of claim 9 , wherein the displaying comprises overlaying and displaying the heatmap generated based on the score corresponding to the second index, on a screen on which at least a portion of the pathological slide image is output.
14 . The method of claim 9 , wherein the displaying comprises:
displaying an object adjusting a threshold value determining whether or not at least one of the components of the cell is stained, based on a score corresponding to the second index; and
updating and displaying the visualization result of the second index based on a user input adjusting the threshold value through the object, and the displaying the visualization result of the second index comprises updating and displaying at least one of a tumor proportion score (TPS) and a combined positive score (CPS) corresponding to the pathological slide image, and statistics according to types of the cells included in the pathological slide image as the threshold value is adjusted.
15 . A non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute the method of claim 9 .