Multi-variable heatmaps for computer-aided diagnostic models

1 . A computer-implemented method comprising:

acquiring an image generated during a diagnostic session;

applying a diagnostic model associated with multiple different types of medical conditions to the image to produce multiple diagnoses,

wherein upon being applied to the image, the diagnostic model employs a multi-headed neural network to produce multiple outputs indicating a separate diagnosis for each type of medical condition of the multiple different types of medical conditions;

determining contributions of pixels in the image to the multiple diagnoses produced by the multi-headed neural network by establishing contribution on a per-pixel and per-diagnosis basis;

identifying, based on an analysis of the contributions,

a first set of pixels determined to have contributed to the production of a first diagnosis for a first type of medical condition, and

a second set of pixels different from the first set of pixels and determined to have contributed to the production of a second diagnosis for a second type of medical condition; and

causing display, on an electronic device, of a single unified visualization of the image that includes modifications to both the first and second sets of pixels of the image to indicate a magnitude of the contributions of the first and second sets of pixels to the first and second diagnoses, respectively,

wherein the modifications visually distinguish the first and second sets of pixels from each other and from other pixels in the image, and

wherein the magnitude is positive or negative.

2 . The computer-implemented method of claim 1 ,

wherein said determining comprises applying an attribution model to each diagnosis of the multiple diagnoses produced by the multi-headed neural network,

wherein the attribution model produces a matrix of attribution values for each diagnosis of the multiple diagnoses, and

wherein each attribution value is representative of the importance of the corresponding pixel to the corresponding diagnosis.

3 . The computer-implemented method of claim 1 , wherein the visualization causes the first and second sets of pixels to have different image characteristics.

4 . The computer-implemented method of claim 1 , wherein the electronic device is associated with a medical professional responsible for managing the diagnostic session.

5 . The computer-implemented method of claim 1 , wherein the first and second sets of pixels share at least one pixel in common.

6 . The computer-implemented method of claim 1 , wherein the image comprises a retinal image, and wherein the multiple different medical conditions comprise multiple ocular conditions.

7 . A non-transitory computer-readable medium with instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:

applying a diagnostic model associated with multiple different types of conditions to an image generated during a diagnostic session to produce multiple outputs,

wherein each output of the multiple outputs is indicative of a corresponding one of the multiple different types of conditions, and

wherein upon being applied to the image, the diagnostic model employs a multi-headed neural network to produce a separate output for each type of condition of the multiple different types of conditions;

applying an attribution model to each output of the multiple outputs produced by the multi-headed neural network, the attribution model being programmed to:

identify a reference image to serve as a baseline, and

estimate the contribution of a given pixel in the image to a given output based on a comparison to a corresponding pixel of the reference image,

wherein the attribution model produces a separate matrix of attribution values for each output of the multiple outputs;

determining, for each output of the multiple outputs, which pixels in the image were determined to contribute to the output by examining a corresponding matrix of attribution values;

transforming the corresponding matrix of attribution values to a range of values in a color space; and

causing display of a single visualization of the image that includes modifications to the pixels in the image determined to contribute to each output of the multiple outputs according to the transformation, wherein the modifications visually distinguish the pixels in the image determined to contribute to each output from each other and from other pixels in the image.

8 . The non-transitory computer-readable medium of claim 7 , wherein each output of the multiple outputs is indicative of a proposed diagnosis for the corresponding type of condition.

9 . The non-transitory computer-readable medium of claim 7 , wherein the operations further comprise:

identifying

a first set of pixels determined to have contributed to the production of a first output for a first type of condition, and

a second set of pixels different from the first set of pixels and determined to have contributed to the production of a second output for a second type of condition,

wherein causing display of a visualization of the image comprises causing display of the visualization that visually distinguishes the first and second sets of pixels from other pixels in the image on an electronic device.

10 . The non-transitory computer-readable medium of claim 9 , wherein the visualization is a multi-variable heatmap.

11 . The non-transitory computer-readable medium of claim 10 , wherein the operations further comprise:

creating multiple single-variable heatmaps by producing, for each output of the multiple outputs, a separate single-variable heatmap that visually distinguishes pixels in the image determined to have contributed to a corresponding type of condition of the multiple different types of conditions by the multi-headed neural network; and

compiling the multiple single-variable heatmaps into the multi-variable heatmap.

12 . The non-transitory computer-readable medium of claim 9 , wherein the operations further comprise:

receiving input indicative of a selection of the first output; and

causing display of a different visualization that visually distinguishes only the first set of pixels from the other pixels in the image on the electronic device.

13 . The non-transitory computer-readable medium of claim 12 , wherein the different visualization is a single-variable heatmap.

14 . The non-transitory computer-readable medium of claim 7 , wherein the operations further comprise:

storing the matrices of attribution values in a profile associated with an individual captured in the image.

15 . An electronic device comprising:

a memory that includes instructions for producing multi-variable heatmaps to assist individuals in rendering diagnoses,

wherein the instructions, when executed by a processor, cause the processor to:

acquire an image generated by an imaging device during a diagnostic session;

apply a multi-headed neural network to the image to produce a diagnosis for each medical condition of multiple different medical conditions, thereby producing multiple diagnoses;

determine a contribution of each pixel in the image to each diagnosis of the multiple diagnoses produced by the multi-headed neural network;

identify

a first set of pixels determined to have contributed to the production of a first diagnosis, and

a second set of pixels different from the first set of pixels and determined to have contributed to the production of a second diagnosis; and

cause a multi-variable heatmap that includes modifications to colors of the first and second sets of pixels to be shown on a display,

wherein the modifications are based on original values of the first and second sets of pixels,

wherein the modifications visually distinguish the first and second sets of pixels from each other and from other pixels in the image, and

wherein the modifications to the first and second sets of pixels of the image to indicate a magnitude of the contributions of the first and second sets of pixels to the first and second diagnoses.

16 . The electronic device of claim 15 , wherein the multi-variable heatmap overlays the image when shown on the display.

17 . The electronic device of claim 15 , wherein the instructions further cause the processor to:

receive input indicative of a selection of the first set of pixels or the first diagnosis; and

cause display of a single-variable heatmap that visually distinguishes only the first set of pixels from other pixels of the image.

18 . The electronic device of claim 15 , wherein the instructions further cause the processor to:

receive input indicative of a selection of the first set of pixels or the first diagnosis; and

cause the first set of pixels to be rendered at a higher intensity than the second set of pixels in the multi-variable heatmap.

19 . The electronic device of claim 15 , wherein the processor is communicatively coupled to the display across a network.

20 . The electronic device of claim 15 , wherein the processor is communicatively coupled to the imaging device across a network.