Methods and systems for ocular imaging, diagnosis and prognosis
Data representative of the eye obtained from multiple modalities are combined into a virtual model of the eye. The multiple modalities indicate anatomical, physiological, and/or functional features of the eye. The data from different modalities is registered in order to combine the data into the virtual model. Eye data can be analyzed in the form of the virtual model. Neural networks can be used to obtain insights about medical conditions of the eye, such as for diagnosis or prognosis of conditions. It can also be predicted how the eye will respond to certain treatments.
1. A computer-implemented method for diagnosing disease in an eye, the method comprising:
receiving first eye image data representative of the eye, the first eye image data being obtained from a first imaging modality;
receiving second eye image data representative of the eye, the second eye image data being obtained from a second imaging modality, wherein the second imaging modality is different from the first imaging modality;
analyzing the first and second eye image data using an ensemble neural network to generate a diagnostic parameter for the eye,
wherein the ensemble neural network comprises a first neural network to analyze the first eye image data and a second neural network to analyze the second eye image data, wherein the ensemble neural network comprises a fully connected layer receiving outputs from each of the first and second neural networks, the diagnostic parameter for the eye being output from the fully connected layer, and
wherein the first and second imaging modalities indicate features of the eye that are selected from different members of the group consisting of: anatomical; physiological; and functional features.
2. A computer-implemented method as claimed in claim 1 , wherein the first and second imaging modalities are selected from the group consisting of: magnetic resonance imaging (MRI); fundus photography; optical coherence tomography (OCT); optical coherence tomography angiography (OCTA); X-ray; computer tomography (CT); biometry; ultrasound; keratometry; corneal topography imaging; retinal perfusion mapping and laser speckle flowmetry.
3. A computer-implemented method as claimed in claim 1 , wherein the method comprises receiving the first eye image data and the second eye image data as a combined data set.
4. A computer-implemented method as claimed in claim 3 , wherein the method comprises receiving the first eye image data and the second eye image data in the form of data representative of a model of the eye.
5. A computer-implemented method as claimed in claim 1 , wherein the method comprises weighting the outputs from each of the first and second neural networks in the fully connected layer.
6. A computer-implemented method as claimed in claim 1 , wherein the method comprises:
generating first and second feature maps using each of the first and second neural networks respectively;
generating first and second one dimensional arrays from each of the first and second feature maps respectively; and
combining the first and second one dimensional arrays in the fully connected layer with a weighting.
7. A computer-implemented method as claimed in claim 1 , wherein the method further comprises outputting the diagnostic parameter for the eye.
8. A computer-implemented method for diagnosing disease in an eye, the method comprising:
receiving first eye image data representative of the eye, the first eye image data being obtained from a first imaging modality;
receiving second eye image data representative of the eye, the second eye image data being obtained from a second imaging modality, wherein the second imaging modality is different from the first imaging modality;
analyzing the first and second eye image data using an ensemble neural network to generate a diagnostic parameter for the eye,
wherein the ensemble neural network comprises a first neural network to analyze the first eye image data and a second neural network to analyze the second eye image data, wherein the ensemble neural network comprises a fully connected layer receiving outputs from each of the first and second neural networks, the diagnostic parameter for the eye being output from the fully connected layer,
wherein the method further comprises:
generating first and second feature maps using each of the first and second neural networks respectively;
generating first and second one dimensional arrays from each of the first and second feature maps respectively; and
combining the first and second one dimensional arrays in the fully connected layer with a weighting.