Systems and methods for processing electronic images to visualize combinations of semantic pathology features

1 . A computer-implemented method for processing electronic medical images, comprising:

receiving one or more digital images of a pathology specimen;

detecting, by applying a first machine learning system, a presence of one or more spatial locations of one or more attributes in the received digital image, the one or more attributes representing one or more tissue types in the one or more digital images, the one or more attributes including a first attribute and a second attribute, the one or more spatial locations including a plurality of first spatial locations of the first attribute and a plurality of second spatial locations of the second attribute;

detecting, by the first machine learning system, a spatial relationship of the one or more spatial locations, the detecting the spatial relationship of the one or more spatial locations including determining a set of relative distances between the plurality of first spatial locations and plurality of second spatial locations;

determining one or more spatial locations of co-occurrence pairs of the first attribute and the second attribute based on the set of relative distances between the plurality of first spatial locations and the plurality of second spatial locations, the co-occurrence indicating a relationship between the first attribute and the second attribute;

determining an extent of disease for the one or more digital images based on a number of instances of the one or more spatial locations of co-occurrence pairs of the first attribute and the second attribute; and

outputting, to a display, a visual depiction of the plurality of first spatial locations and the plurality of second spatial locations, the spatial relationship, extent of disease, and the one or more spatial locations of co-occurrence pairs.

2 . The method of claim 1 , wherein determining the one or more spatial locations of co-occurrence pairs includes selecting specific spatial locations of the plurality of first spatial locations and the plurality of second spatial locations that determined the co-occurrence.

3 . The method of claim 1 , wherein outputting the visual depiction includes outputting a tissue map that indicates the plurality of first spatial locations and the plurality of second spatial locations of co-occurrence pairs.

4 . The method of claim 1 , wherein outputting the visual depiction includes outputting a first selectable icon corresponding to the first attribute and a second selectable icon corresponding to the second attribute.

5 . The method of claim 1 , wherein the method further comprises ranking at least one of:

the plurality of first spatial locations based on a proximity to the plurality of second spatial locations,

the plurality of second spatial locations based on a proximity to the plurality of first spatial locations,

the spatial locations of co-occurrence pairs, wherein the ranking is based on a predetermined policy that defines a priority of the spatial locations of co-occurrence pairs.

6 . The method of claim 1 , further comprising detecting that the first attribute has a higher priority than the second attribute.

7 . A system for identifying attributes of electronic images and displaying the attributes, the system comprising:

at least one memory storing instructions; and

at least one processor configured to execute the instructions to perform operations comprising:

receiving one or more digital images of a pathology specimen;

detecting, by applying a first machine learning system, a presence of one or more spatial locations of one or more attributes in the received digital image, the one or more attributes representing one or more tissue types in the one or more digital images, the one or more attributes including a first attribute and a second attribute, the one or more spatial locations including a plurality of first spatial locations of the first attribute and a plurality of second spatial locations of the second attribute;

detecting, by the first machine learning system, a spatial relationship of the one or more spatial locations, the detecting the spatial relationship of the one or more spatial locations including determining a set of relative distances between the plurality of first spatial locations and plurality of second spatial locations;

determining one or more spatial locations of co-occurrence pairs of the first attribute and the second attribute based on the set of relative distances between the plurality of first spatial locations and the plurality of second spatial locations, the co-occurrence pairs indicating a relationship between the first attribute and the second attribute;

determining an extent of disease for the one or more digital images based on a number of instances of the one or more spatial locations of co-occurrence pairs of the first attribute and the second attribute; and

outputting, to a display, a visual depiction of the plurality of first spatial locations and the plurality of second spatial locations, the spatial relationship, extent of disease, and the one or more spatial locations of co-occurrence pairs.

8 . The system of claim 7 , wherein determining the one or more spatial locations of co-occurrence pairs includes selecting specific spatial locations of the plurality of first spatial locations and the plurality of second spatial locations that determined the co-occurrence.

9 . The system of claim 7 , wherein outputting the visual depiction includes outputting a tissue map that indicates the plurality of first spatial locations and the plurality of second spatial locations of co-occurrence pairs.

10 . The system of claim 7 , wherein outputting the visual depiction includes outputting a first selectable icon corresponding to the first attribute and a second selectable icon corresponding to the second attribute.

11 . A non-transitory computer-readable medium storing instructions that, when executed by a processor, perform a method for identifying attributes of electronic images and displaying the attributes, the method comprising:

receiving one or more digital images of a pathology specimen;

detecting, by applying a first machine learning system, a presence of one or more spatial locations of one or more attributes in the received digital image, the one or more attributes representing one or more tissue types in the one or more digital images, the one or more attributes including a first attribute and a second attribute, the one or more spatial locations including a plurality of first spatial locations of the first attribute and a plurality of second spatial locations of the second attribute;

detecting, by the first machine learning system, a spatial relationship of the one or more spatial locations, the detecting the spatial relationship of the one or more spatial locations including determining a set of relative distances between the plurality of first spatial locations and plurality of second spatial locations;

determining one or more spatial locations of co-occurrence pairs of the first attribute and the second attribute based on the set of relative distances between the plurality of first spatial locations and the plurality of second spatial locations, the co-occurrence indicating a relationship between the first attribute and the second attribute;

determining an extent of disease for the one or more digital images based on a number of instances of the one or more spatial locations of co-occurrence pairs of the first attribute and the second attribute; and

outputting, to a display, a visual depiction of the plurality of first spatial locations and the plurality of second spatial locations, the spatial relationship, extent of disease, and the one or more spatial locations of co-occurrence pairs.

12 . The method of claim 1 , wherein the one or more attributes include tissue identifying muscularis propria, calcification, or ductal carcinoma in situ.

13 . The method of claim 1 , wherein the one or more attributes includes a third attribute, the one or more spatial locations including a plurality of third spatial locations of the third attribute.

14 . The method of claim 13 , further comprising:

determining, by the first machine learning system, a second spatial relationship of the one or more attributes, including determining a second set of relative distances between the plurality of third spatial locations, the plurality of second locations, and the plurality of third locations; and

determining, by the first machine learning system, a second one or more locations of co-occurrence pairs between the third attribute and the first attribute based on the set second of relative distances.

15 . The method of claim 14 , further including:

ranking a priority of the one or more spatial locations of co-occurrence pairs as compared to the second one or more locations of co-occurrence.

16 . The method of claim 1 , further comprising:

ranking a priority of the one or more spatial locations of co-occurrence pairs;

wherein, the display prioritizes a highest ranked spatial location of co-occurrence pairs by highlighting, boxing, or enlarging the highest ranked spatial location of co-occurrence pair.

17 . The system of claim 7 , further comprising:

ranking a priority of the one or more spatial locations of co-occurrence pairs;

wherein, the display prioritizes a highest ranked spatial location of co-occurrence pair by highlighting, boxing, or enlarging the highest ranked spatial location of co-occurrence pair.

18 . The non-transitory computer-readable medium of claim 11 , further comprising:

ranking a priority of the one or more spatial locations of co-occurrence pairs;

wherein, the display prioritizes a highest ranked spatial location of co-occurrence pairs by highlighting, boxing, or enlarging the highest ranked spatial location of co-occurrence pairs.