Methods and systems for ocular imaging, diagnosis and prognosis
Embodiments of the invention involve combining data representative of the eye obtained from multiple modalities into a virtual model of the eye. The multiple modalities indicate anatomical, physiological, and/or functional features of the eye. The data from different modalities is registered in order to combine the data into the virtual model. Further embodiments involve analysing eye data, for example in the form of the virtual model, using neural networks to obtain insights about medical conditions of the eye, for example the diagnosis or prognosis of conditions, and/or predicting how the eye will respond to certain treatments.
1. A computer-implemented method for prognosing disease in an eye, the method comprising:
receiving first present eye image data representative of the eye at a present time, the first present eye image data being obtained from a first imaging modality;
receiving second present eye image data representative of the eye at the present time, the second present eye image data being obtained from a second imaging modality, wherein the second imaging modality is different from the first imaging modality;
analyzing the first and second present eye image data using a neural network to generate a prediction for future eye image data representative of the eye at a future time; and
generating a prognostic parameter for the eye from the future eye image data,
wherein the neural network is trained, using first past eye image data representative of a plurality of eyes at first and second past times in the first imaging modality and using second past eye image data representative of the plurality of eyes at first and second past times in the second imaging modality, to generate one or more eye image data change functions, wherein the eye image data change function is applied by the neural network to the first and second present eye image data to generate the prediction for the future eye image data.
2. The computer-implemented method as claimed in claim 1 , wherein the neural network is further trained using first and second past eye image data representative of a plurality of eyes at a third past time in the first and second imaging modalities respectively to refine the eye image data change function.
3. The computer-implemented method as claimed in claim 1 , wherein the first and second imaging modalities indicate features of the eye that are selected from the group consisting of: anatomical; physiological; and functional features.
4. The computer-implemented method as claimed in claim 1 , wherein the first and second imaging modalities are selected from the group consisting of: magnetic resonance imaging (MRI); fundus photography; optical coherence tomography (OCT); optical coherence tomography angiography (OCTA); X-ray; computer tomography (CT); biometry; ultrasound; keratometry; corneal topography imaging; retinal perfusion mapping and laser speckle flowmetry.
5. The computer-implemented method as claimed in claim 3 , wherein the first and second imaging modalities indicate features of the eye that are selected from different members of the group consisting of: anatomical; physiological; and functional features.
6. The computer-implemented method as claimed in claim 1 , wherein the eye image data change function comprises one or more matrices of change.
7. The computer-implemented method as claimed in claim 1 , wherein the neural network comprises a long short-term memory (LTSM) network.
8. The computer-implemented method as claimed in claim 7 , wherein the neural network is an ensemble neural network comprising a first neural network to analyze past and/or present eye image data from the first imaging modality and a second neural network to analyze past and/or present eye image data from the second imaging modality, wherein the ensemble neural network comprises a fully connected layer receiving outputs from each of the first and second neural networks, the output of the fully connected layer being input to the long short-term memory (LTSM) network.
9. The computer-implemented method as claimed claim 1 , wherein the method comprises receiving the first past and/or present eye image data and the second past and/or present eye image data as a combined data set.
10. The computer-implemented method as claimed in claim 9 , wherein the method comprises receiving the first past and/or present eye image data and the second past and/or present eye image data in the form of data representative of a model of the eye.
11. The computer-implemented method as claimed in claim 10 , wherein the data representative of the model of the eye is generated using a method comprising:
processing the first eye image data to generate first processed eye image data, wherein the first processed eye image data has identified therein one or more features of the eye;
processing the second eye image data to generate second processed eye image data, wherein the second processed eye image data has identified therein one or more features of the eye;
registering the first processed eye image data and the second processed eye image data; and
combining the registered first and second processed eye image data to generate data representative of a model of the eye.
12. The computer-implemented method as claimed in claim 1 , wherein the neural network is trained using past eye treatment data representative of prior treatments undergone by the plurality of eyes, and the method comprises generating a treatment prediction parameter comprising the prognostic parameter for the eye on the assumption a treatment is used on the eye.
13. The computer-implemented method as claimed in claim 1 , wherein the method further comprises outputting the prognostic parameter for the eye.