Rule out accuracy for detecting findings of interest in images

1. A system for training an artificial intelligence engine, the system comprising:

an electronic processor; and

memory storing instructions that, when executed by the electronic processor, cause the system to:

obtain a set of reports corresponding to a set of medical images;

for each report in the set of reports, determine whether the report includes a label for a finding of interest and, in response to determining that a label for the finding of interest was not mentioned in the report, identify the report as an ambiguous report;

generate a predicted label for the finding of interest for each report identified as an ambiguous report in the set of reports; and

train the artificial intelligence engine using a training set including each ambiguous report with the predicted label and each non-ambiguous report in the set of reports, thereby generating a classification model for the label for the finding of interest.

2. The system according to claim 1 , wherein the memory further includes instructions that, when executes by the electronic processor, cause the system to:

apply natural language processing to the set of reports.

3. The system according to claim 2 , wherein applying natural language processing includes applying context recognition natural language processing to differentiate between negative labels and positive labels for the finding of interest.

4. The system according to claim 1 , wherein generating the predicted label for each report identified as an ambiguous report using a normal distribution to assign each report identified as an ambiguous report a positive label or a negative label.

5. The system according to claim 4 , wherein the normal distribution is based on clinical data.

6. The system according to claim 4 , wherein at least 80% of the predicted labels are negative labels.

7. The system according to claim 4 , wherein at least 90% of the predicted labels are negative labels.

8. The system according to claim 1 , wherein the set of medical images are chest x-ray images.

9. The system according to claim 1 , wherein the memory further includes instructions that, when executed by the electronic processor, cause the system to:

apply the classification model in a subsequent analysis by the artificial intelligence engine.

10. A method for training an artificial intelligence engine hosted on a server, the method comprising:

obtaining a set of reports corresponding to a set of medical images;

applying natural language processing to the set of reports;

for each report in the set of reports, determining whether the report includes a label for a finding of interest and, in response to determining that a label for the finding of interest was not mentioned in the report, identifying the report as an ambiguous report;

generating, using an electronic processor, a predicted label for the finding of interest for each report identified as an ambiguous report in the set of reports; and

training the artificial intelligence engine using a training set including each ambiguous report with the predicted label and each non-ambiguous report in the set of reports, thereby generating a classification model for the finding of interest.

11. The method according to claim 10 , wherein applying natural language processing includes applying context recognition natural language processing to differentiate between negative labels and positive labels for the finding of interest.

12. The method according to claim 11 , wherein generating the predicted label for each report identified as an ambiguous report includes using a normal distribution to assign each report identified as an ambiguous report a positive label or a negative label.

13. The method according to claim 12 , wherein the normal distribution is based on clinical data.

14. The method according to claim 13 , wherein 80%-90% of the predicted labels are negative labels.

15. The method according to claim 14 , wherein the set of medical images are chest x-ray images.

16. The method according to claim 15 , further comprising applying the classification model in a subsequent analysis by the artificial intelligence engine.

17. Non-transitory computer-readable medium storing instructions that, when executed by an electronic processor, perform a set of functions, the set of functions comprising:

obtaining a set of reports corresponding to a set of medical images;

applying natural language processing to the set of reports;

for each report in the set of reports, determining whether the report includes a label for a finding of interest and, in response to determining that a label for the finding was not mentioned in the report, identifying the report as an ambiguous report;

generating a predicted label for the finding of interest for each report identified as an ambiguous report in the set of reports; and

training the artificial intelligence engine using a training set including each ambiguous report with the predicted label and each non-ambiguous report in the set of reports, thereby generating a classification model for the finding of interest.

18. The non-transitory computer-readable medium according to claim 17 , wherein

applying natural language processing includes applying context recognition natural language processing to differentiate between negative labels and positive labels for the finding of interest; and

wherein generating the predicted label for each report identified as an ambiguous report includes using a normal distribution to assign each report identified as an ambiguous report a positive label or a negative label.

19. The non-transitory computer-readable medium according to claim 18 , wherein

the normal distribution is based on clinical data; and

wherein 80%-90% of the predicted labels are negative labels.

20. The non-transitory computer-readable medium according to claim 19 , wherein the set of functions further comprises:

applying the classification model in a subsequent analysis by the artificial intelligence engine,

wherein the set of medical images are chest x-ray images.