Automatic clinical report generation
Methods and systems for clinical report generation. One system includes an electronic processor configured to receive a query image and determine a similarity metric for a plurality of medical images, where the similarity metric represents a similarity between the query image and each of the plurality of medical images. The electronic processor is also configured to determine a predetermined number of medical images from the plurality of medical images based on the similarity metric for each of the plurality of medical images. The electronic processor is also configured to rank a plurality of reports, where each of the plurality of reports correspond to one of the predetermined number of medical images. The electronic processor is also configured to generate a clinical report including information extracted from at least one of the plurality of reports based on the ranking of the plurality of reports.
1. A system for clinical report generation, the system comprising:
an electronic processor configured to
develop a plurality of hashing models with machine learning using training data, the training data including image level data, each of the plurality of hashing models associated with a different bit length,
receive a query image,
determine a similarity metric for each of a plurality of medical images, wherein the similarity metric represents a similarity between the query image and each of the plurality of medical images, wherein the similarity metric for each of the plurality of medical images is determined using the plurality of hashing models,
determine a predetermined number of medical images from the plurality of medical images based on the similarity metric for each of the plurality of medical images,
rank a plurality of reports based on a label included in each of the plurality of reports, wherein each of the plurality of reports correspond to one of the predetermined number of medical images,
generate a clinical report, including information extracted from at least one of the plurality of reports, based on the ranking of the plurality of reports, and
provide the generated clinical report for user interaction,
wherein ranking the plurality of reports based on the label included in each of the plurality of reports includes ranking a first report of the plurality of reports higher than a second report of the plurality of reports in response to the label included in the first report being associated with a percentage of the plurality of medical images satisfying a predetermined threshold.
2. The system of claim 1 , wherein the training information includes the plurality of medical images and an associated set of image features for each of the plurality of medical images.
3. The system of claim 1 , wherein the similarity metric is a hamming distance.
4. The system of claim 1 , wherein the electronic processor is configured to determine the similarity metric by
extracting a first set of image features from the query image,
extracting a second set of image features from each of the plurality of medical images, and
determining the similarity metric for each of the plurality of medical images based on a comparison of the first set of image features and the second set of image features.
5. The system of claim 4 , wherein the electronic processor is configured to extract the first set of image features and the second set of image features using a deep convolution network for object recognition.
6. The system of claim 1 , wherein the electronic processor is further configured to predict a label for the query image based on labels associated with each of the predetermined number of medical images, wherein the clinical report includes the predicted label for the query image.
7. The system of claim 1 , wherein the predetermined number of medical images includes a set of medical images having the most similarity with the query image.
8. A method for clinical report generation, the method comprising:
developing, with an electronic processor, a plurality of hashing models with machine learning using training data, the training data including image level data, each of the plurality of hashing models associated with a different bit length,
receiving, with the electronic processor, a query image,
determining, with the electronic processor, a similarity metric for each of a plurality of medical images, wherein the similarity metric represents a similarity between the query image and each of the plurality of medical images, wherein the similarity metric for each of the plurality of medical images is determined using the plurality of hashing models,
determining, with the electronic processor, a predetermined number of medical images from the plurality of medical images based on the similarity metric for each of the plurality of medical images,
ranking, with the electronic processor, a plurality of reports based on a label included in each of the plurality of reports, wherein each of the plurality of reports correspond to one of the predetermined number of medical images,
generating, with the electronic processor, a clinical report, including information extracted from at least one of the plurality of reports, based on the ranking of the plurality of reports, and
providing the generated clinical report for user interaction,
wherein ranking the plurality of reports based on the label included in each of the plurality of reports includes ranking a first report of the plurality of reports higher than a second report of the plurality of reports in response to the label included in the first report being associated with a percentage of the plurality of medical images satisfying a predetermined threshold.
9. The method of claim 8 , further comprising:
predicting a label for the query image based on labels associated with each of the predetermined number of medical images, wherein the clinical report includes the predicted label for the query image.
10. The method of claim 8 , further comprising:
normalizing the plurality of reports by extracting information from each of the plurality of reports and generating a new version for each of the plurality of reports such that the new version for each of the plurality of reports follow the same template.
11. The method of claim 8 , wherein generating the clinical report includes
extracting a first portion of information from a first report included in the plurality of reports, and
extracting a second portion of information from a second report included in the plurality of reports,
wherein the clinical report includes the first portion of information and the second portion of information.
12. A non-transitory computer readable medium including instructions that, when executed by an electronic processor, causes the electronic processor to execute a set of functions, the set of functions comprising:
developing a plurality of hashing models with machine learning using training data, the training data including image level data, each of the plurality of hashing models associated with a different bit length,
receiving a query image,
determining a similarity metric for each of a plurality of medical images, wherein the similarity metric represents a similarity between the query image and each of the plurality of medical images, wherein the similarity metric for each of the plurality of medical images is
determined using the plurality of hashing models, determining a predetermined number of medical images from the plurality of medical images based on the similarity metric for each of the plurality of medical images;
ranking a plurality of reports based on a label included in each of the plurality of reports, wherein each of the plurality of reports correspond to one of the predetermined number of medical images,
generating a clinical report, including information extracted from at least one of the plurality of reports, based on the ranking of the plurality of reports, and
providing the generated clinical report for user interaction,
wherein ranking the plurality of reports based on the label included in each of the plurality of reports includes ranking a first report of the plurality of reports higher than a second report of the plurality of reports in response to the label included in the first report being associated with a percentage of the plurality of medical images satisfying a predetermined threshold.
13. The computer readable medium of claim 12 , wherein the training information includes the plurality of medical images and an associated set of image features for each of the plurality of medical images.
14. The computer readable medium of claim 12 , wherein the set of functions further comprises:
predicting a label for the query image based on labels associated with each of the predetermined number of medical images, wherein the clinical report includes the predicted label for the query image.
15. The computer readable medium of claim 12 , wherein the set of functions further comprises:
normalizing the plurality of reports by extracting information from each of the plurality of reports and generating a new version for each of the plurality of reports such that the new version for each of the plurality of reports follow the same template.