Correction of flow projection artifacts in octa volumes using neural networks
A system and/or method uses a trained U-Net neural network to remove flow artifacts from optical coherence tomography (OCT) angiography (OCTA) data. The trained U-Net receives as input both OCT structural volume data and OCTA volume data, but expands the OCTA volume data to include depth information. The U-Net applies dynamic pooling along the depth direction and weighs more heavily the portion of the data that follows (e.g., along the contours of) select retinal layers. In this manner the U-Net applies contextually different computations at different axial locations based at least in part on the depth index information and/or (e.g., proximity to) the select retinal layers. The U-net outputs OCT volume data of reduced flow artifacts as compared with the input OCTA data.
1. A method to reduce artifacts in optical coherence tomography (OCT)-based images of an eye, said method comprising:
collecting OCT image data of the eye from an OCT system, the OCT image data including depth index information; and
submitting the OCT image data to a trained neural network, the neural network applying contextually different computations at different axial locations based at least in part on the depth index information, and producing an output OCT-based image of reduced artifacts as compared to the collected OCT image data.
2. The method of claim 1 , wherein the different computations are contextually dependent upon predefined, local retinal landmarks.
3. The method of claim 2 , wherein the retinal landmarks are predefined retinal layers.
4. The method of claim 1 , wherein the artifacts are one or more of projection artifacts, decorrelation tails, shadow artifacts, and opacities.
5. The method of claim 1 , wherein the neural network includes:
an input layer for receiving the OCT image data;
a dynamic pooling layer following the input layer for condensing image information outside a variable depth range defined by the positions of pre-selected retinal landmarks within the received OCT image data;
a plurality of data processing layers following the dynamic pooling layer, the data processing layers performing contextually different computations at different axial locations based at least in part on the depth index information;
an output layer that compares an output of the plurality of data processing layers with the target-output OCTA image and adjusts internal weights of the data processing layers by a back-propagation process.
6. The method of claim 5 , wherein the neural network applies a loss function having different weights based on the local proximity of the pre-selected retinal landmarks to a current axial position of the OCT image data being processed.
7. The method of claim 6 , wherein the loss function is an L1 function.
8. The method of claim 5 , wherein the pre-selected retinal landmarks are specific retinal layers.
9. The method of claim 8 , wherein the neural network applies a loss function having different weights based on specific retinal layers.
10. The method of claim 9 , wherein the loss function has a first weight for a region between the Inner Limiting Membrane (ILM) and the Retinal Pigment Epithelium (RPE), and a second weight elsewhere.
11. The method of claim 10 , wherein the first weight is at least an order of magnitude greater than the second weight.
12. The method of claim 1 , wherein the neural network has a convolutional structure.
13. The method of claim 1 , wherein the neural network includes a U-Net structure, the U-net structure including:
a plurality of encoding modules in a contracting path; and
a plurality of decoding modules in an expanding path, each decoding module corresponding to a separate encoding module in the contracting path;
wherein each encoding module applies a convolution to its input and columnwise max pooling to the convolution result to define a reduced image, the reduced image being then upsampled to the dimensions of its input and concatenated to its input followed by another convolution.
14. The method of claim 13 , wherein the U-Net structure further includes a bottleneck module between the contracting path and the expanding path, and the bottleneck module applies column-wise pooling.
15. The method of claim 1 , further comprising:
calculating motion contrast information in the collected OCT image data using an OCT angiography (OCTA) processing technique;
creating a structural image of the eye from the collected OCT image data, the structural image depicting tissue structure information;
creating a flow image of the eye from the motion contrast information, the flow image depicting vasculature flow information and containing artifacts;
assigning depth index information to the flow image along its axial direction; and
submitting the structural image, flow image, and assigned depth index information to the trained neural network, the produced output OCT-based image being a vascular image of reduced artifacts as compared to the flow image.
16. The method of claim 15 , wherein the artifacts are one or more of projection artifacts, decorrelation tails, shadow artifacts, and opacities.
17. The method of claim 15 , wherein producing the output OCT-based image includes determining a difference between an ideal corrected flow data and the created flow image, and adding the difference to the created flow image.
18. The method of claim 1 , wherein training the neural network includes:
collecting a plurality of OCT acquisitions to define training-input OCT images;
defining a plurality of OCTA images from the OCT acquisitions to define corresponding training-input OCTA images;
submitting each OCTA image to an artifact removal algorithm to define a corresponding target-output OCTA image of reduced artifacts;
defining a plurality of training-input sets, each training-input set include a training-input OCT image, a corresponding OCTA image, and depth information for the axial positions of pixels within the OCTA image.
19. The method of claim 1 , wherein the neural network includes:
an input layer for receiving the structural image, flow image, and assigned depth index information;
a dynamic pooling layer following an input layer for condensing information outside a variable depth range defined by the positions of pre-selected retinal landmarks;
a plurality of data processing layers following the dynamic pooling layer, the data processing layers performing contextually different computations at different axial locations based at least in part on the depth index information;
an output layer that compares an output of the plurality of data processing layers with the target-output OCTA image and adjusts internal weights of the data processing layers by a back-propagation process.
20. The method of claim 19 , wherein the retinal landmarks are predefined retinal layers.