SYSTEMS AND METHODS FOR AUGMENTED REALITY BASED WHOLE SLIDE IMAGE VISUALIZATION

1 . A computer-implemented method for generating a navigable three-dimensional image of a tissue sample using augmented reality or virtual reality (AR/VR), the method comprising:

receiving, at one or more processors, a plurality of medical images, the plurality of medical images being associated with at least one whole slide image (WSI) of the tissue sample;

determining, by one or more processors, a layer level of each medical image of the plurality of medical images;

determining, by one or more processors, for each medical image of the plurality of medical images, one or more regions of tissue and one or more regions of non-tissue; and

stacking, by one or more processors, each medical image with one or more of the plurality of medical images, based on the layer level, to generate a three-dimensional rendering for display in an AR/VR environment.

2 . The method of claim 1 , further comprising:

using one or more cameras on an AR/VR device to track a wearer's eye gestures.

3 . The method of claim 2 , wherein the wearer's eye gestures comprise blinking, double-blinking, winking, straining, squinting, crossing of eyes, and movement of the eyes.

4 . The method of claim 3 , further comprising:

modifying one or more display settings of the three-dimensional rendering using the wearer's eye gestures, the one or more display settings comprising zooming into a feature on the three-dimensional rendering, cycling through the layer level of the three-dimensional rendering, or locking onto the feature on the three-dimensional rendering.

5 . The method of claim 3 , further comprising:

modifying one or more display settings of a laboratory information system (LIS) using the wearer's eye gestures, the one or more display settings including reading a physical medical record file a user is looking at, querying an LIS database, and retrieving relevant information such as patient health history, case history, and whole slide images.

6 . The method of claim 1 , further comprising:

receiving a user selection of a layer of the three-dimensional rendering; and

generating a revised three-dimensional rendering, based on the user selection of the layer.

7 . The method of claim 1 , wherein a plurality of users have simultaneous access to the three-dimensional rendering, and each of the plurality of users are associated with a different access and/or modification permission.

8 . The method of claim 7 , wherein the different access and/or modification permission includes providing annotations of ROIs and/or providing suggestions of a user selection of a layer of the three-dimensional rendering.

9 . The method of claim 1 , further comprising:

automatically determining one or more floaters and/or non-tissue items from each medical image of the plurality of medical images; and

removing, from each medical image of the plurality of medical images, the floaters and/or non-tissue items.

10 . The method of claim 1 , wherein the AR/VR environment can be contoured to a room in which a user is viewing the three-dimensional rendering, wherein the contours are determined automatically, and wherein the three-dimensional rendering is sized based on the determined contours of the room.

11 . A system for generating a navigable three-dimensional image of a tissue sample an augmented reality or virtual reality (AR/VR) environment, the system comprising:

a memory storing instructions and a processor operatively connected to the memory and configured to execute the instructions to perform operations comprising:

receiving, at one or more processors, a plurality of medical images, the plurality of medical images being associated with at least one whole slide image (WSI) of the tissue sample;

determining, by one or more processors, a layer level of each medical image of the plurality of medical images;

determining, by one or more processors, for each medical image of the plurality of medical images, one or more regions of tissue and one or more regions of non-tissue; and

stacking, by one or more processors, each medical image with one or more of the plurality of medical images, based on the layer level, to generate a three-dimensional rendering for display in an AR/VR environment.

12 . The system of claim 11 , further comprising:

using on or more cameras on an AR/VR device to track a wearer's eye gestures, wherein the wearer's eye gestures comprise blinking, double-blinking, winking, straining, squinting, crossing of eyes, and movement of the eyes.

13 . The system of claim 11 , wherein a plurality of users have simultaneous access to the three-dimensional rendering, and each of the plurality of users are associated with a different access and/or modification permission.

14 . The system of claim 13 , wherein the different access and/or modification permission includes providing annotations of ROIs and/or providing suggestions of a user selection of a layer of the three-dimensional rendering.

15 . A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, perform operations including:

receiving, at the one or more processors, a plurality of medical images, the plurality of medical images being associated with at least one whole slide image (WSI) of a tissue sample;

determining, by one or more processors, a layer level of each medical image of the plurality of medical images;

determining, by one or more processors, for each medical image of the plurality of medical images, one or more regions of tissue and one or more regions of non-tissue; and

stacking, by one or more processors, each medical image with one or more of the plurality of medical images, based on the layer level, to generate a three-dimensional rendering for display in an AR/VR environment.

16 . The computer-readable medium of claim 15 , further comprising:

using one or more cameras on the AR/VR device to track a wearer's eye gestures, wherein the wearer's eye gestures comprise blinking, winking, straining, squinting, crossing of eyes and sight movement.

17 . The computer-readable medium of claim 16 , further comprising:

controlling one or more display settings of the three-dimensional rendering using the wearer's eye gestures, the one or more display settings including zooming into a feature on the three-dimensional rendering, cycling through the layer level of the three-dimensional rendering, or locking onto the feature on the three-dimensional rendering.

18 . The computer-readable medium of claim 16 , further comprising:

controlling one or more display settings of a laboratory information system (LIS) using the wearer's eye gestures, the one or more display settings including reading a physical medical record file a user is looking at, querying an LIS database, and retrieving relevant information such as patient health history, case history, and whole slide images.

19 . The computer-readable medium of claim 15 , wherein a plurality of users have simultaneous access to the three-dimensional rendering, and each of the plurality of users are associated with a different access and/or modification permission.

20 . The computer-readable medium of claim 19 , wherein the different access and/or modification permission includes providing annotations of ROIs and/or providing suggestions of a user selection of a layer of the three-dimensional rendering.