Method and system for refining label information

1. A method for refining label information, performed by at least one computing device, the method comprising:

acquiring a pathology slide image including a plurality of patches;

inferring a plurality of label information items for the plurality of patches included in the acquired pathology slide image using a machine learning model;

applying the inferred plurality of label information items to the pathology slide image;

calculating at least one of a confidence score or an entropy value for each of the plurality of patches;

selecting at least one first patch to be refined from among the plurality of patches based on a comparison of at least one of the calculated confidence score or the entropy value and a predetermined value; and

providing the pathology slide image applied with the inferred plurality of label information items to an annotator terminal,

wherein the plurality of label information items for the plurality of patches includes a plurality of classes associated with the plurality of patches, and

the calculating at least one of the confidence score or the entropy value for each of the plurality of patches includes assigning a weight to the entropy value for a target class among the plurality of classes associated with the plurality of patches.

2. The method according to claim 1 , further comprising receiving, from the annotator terminal, a response to at least one label information item among the plurality of inferred label information items, wherein the at least one label information item is associated with at least one patch among the plurality of patches.

3. The method according to claim 2 , wherein the receiving, from the annotator terminal, the response to at least one label information item among the inferred plurality of label information items includes, when receiving a confirmation on the at least one label information item, classifying the at least one patch and the at least one label information item into a training dataset of the machine learning model.

4. The method according to claim 2 , wherein the receiving, from the annotator terminal, the response to at least one label information item among the inferred plurality of label information items includes:

receiving a refined label information item for the at least one patch; and

classifying the at least one patch and the refined label information item into a training dataset of the machine learning model.

5. The method according to claim 4 , wherein the classifying the at least one patch and the refined label information item into the training dataset of the machine learning model includes assigning a weight to the refined label information item for the at least one patch, which is used for training the machine learning model.

6. The method according to claim 4 , wherein the providing the pathology slide image applied with the inferred plurality of label information items to the annotator terminal includes providing a compressed image of the pathology slide image to the annotator terminal, wherein the compressed image is associated with compressed label information items of the plurality of label information items, and

the receiving the refined label information item for the at least one patch includes:

receiving, from the annotator terminal, a request for the at least one label information item corresponding to a first compressed label information item selected from the compressed image;

providing the at least one label information item to the annotator terminal; and

receiving a refined label information item for the provided at least one label information item.

7. The method according to claim 1 , wherein the applying the inferred plurality of label information items to the pathology slide image includes outputting a visual representation of the determined at least one first patch to be refined.

8. The method according to claim 7 , wherein the visual representation indicates an area where the label information item for the selected at least one first patch is uncertain.

9. The method according to claim 1 , wherein the determining at least one patch to be refined from among the plurality of patches includes determining, from among the plurality of patches, at least one second patch to be refined, which is associated with the target class having the weighted entropy value, and

the applying the inferred plurality of label information items to the pathology slide image includes outputting a visual representation of the determined at least one second patch to be refined.

10. An information processing system comprising:

a memory storing one or more instructions; and

a processor configured to, by executing of the stored one or more instructions:

acquire a pathology slide image including a plurality of patches;

infer a plurality of label information items for the plurality of patches included in the acquired pathology slide image using a machine learning model;

apply the inferred plurality of label information items to the pathology slide image;

calculate at least one of a confidence score or an entropy value for each of the plurality of patches;

select at least one first patch to be refined from among the plurality of patches based on a comparison of at least one of the calculated confidence score or the entropy value and a predetermined value; and

provide the pathology slide image applied with the inferred plurality of label information items to an annotator terminal

wherein the plurality of label information items for the plurality of patches includes a plurality of classes associated with the plurality of patches, and

the processor is further configured to assign a weight to the entropy value for a target class among the plurality of classes associated with the plurality of patches.

11. The information processing system according to claim 10 , wherein the processor is further configured to receive, from the annotator terminal, a response to at least one label information item among the inferred plurality of label information items, and

the at least one label information item is associated with at least one patch among the plurality of patches.

12. The information processing system according to claim 11 , wherein the processor is further configured to, when receiving a confirmation on the at least one label information item, classify the at least one patch and the at least one label information item into a training dataset of the machine learning model.

13. The information processing system according to claim 11 , wherein the processor is further configured to:

receive a refined label information item for the at least one patch; and

classify the at least one patch and the refined label information item into a training dataset of the machine learning model.

14. The information processing system according to claim 13 , wherein the processor is further configured to assign a weight to the refined label information item for the at least one patch, which is used for training the machine learning model.

15. The information processing system according to claim 13 , wherein the processor is further configured to:

provide a compressed image of the pathology slide image to the annotator terminal;

receive, from the annotator terminal, a request for the at least one label information item corresponding to a first compressed label information item selected from the image;

provide the at least one label information item to the annotator terminal; and

receive a refined label information item for the provided at least one label information item.

16. The information processing system according to claim 10 , wherein the processor is further configured to output a visual representation of the determined at least one first patch to be refined.

17. The information processing system according to claim 16 , wherein the visual representation indicates an area where the label information item for the selected at least one first patch is uncertain.

18. The information processing system according to claim 10 , wherein the processor is further configured to:

determine, from among the plurality of patches, at least one second patch to be refined, which is associated with the target class having the weighted entropy value; and

output a visual representation of the determined at least one second patch to be refined.