Systems and methods for processing of fundus images
Methods and systems for detecting glycosylated haemoglobin (HbA1c) levels from at least one fundus image are disclosed. At least one fundus image associated with an individual is processed using a first set of one or more convolutional neural networks to determine a glycosylated haemoglobin (HbA1c) level for the at least one fundus image. Methods and systems of determining a risk level of progression of diabetic retinopathy of an individual are also disclosed. At least one fundus image associated with the individual is processed using a second set of one or more convolutional neural networks to determine a retinopathy grade for the at least one fundus image. A risk level of progression of diabetic retinopathy of the individual is determined based on at least the HbA1c level and the retinopathy grade.
1 . A method of detecting glycosylated haemoglobin (HbA1c) levels from at least one fundus image, the method performed by one or more processors, the method comprising the following three steps in sequence:
(a) processing one or more fundus images associated with an individual using a first Quality Assurance (QA) set of one or more convolutional neural networks (CNNs) to determine whether the one or more fundus images are of sufficient quality for further processing;
(b) processing the one or more fundus images determined to be of sufficient quality for further processing using a second eye-identification set of one or more CNNs (eye-ID CNN), to identify the one or more fundus images belonging to a single eye; and
(c) processing the one or more fundus images belonging to the single eye using a third glycosylated haemoglobin (HbA1c) set of one or more CNNs to determine an HbA1c level for the individual,
wherein the first QA set of one or more CNNs, the second eye-identification set of one or more CNNs and the third glycosylated haemoglobin (HbA1c) set of one or more CNNs are computer-driven CNNs that have been previously trained over multiple iterations by a plurality of sample images previously labelled and quantified by an expert clinician, and previously validated using a plurality of test images from a plurality of test subjects,
wherein the at least one fundus image is a two-dimensional visible-light photograph, and
wherein an accuracy in determining the HbA1c level from the at least one fundus image is improved by using the combination of the three sets of the one or more CNNs compared to a determination made without using the three sets of the one or more CNNs.
2 . The method of claim 1 , further comprising the steps of:
processing the at least one fundus image associated with the individual using a fourth set of one or more convolutional neural networks to determine a retinopathy grade for the at least one fundus image; and
determining, based on at least the HbA1c level and the retinopathy grade, a risk level of progression of diabetic retinopathy of the individual.
3 . The method of claim 2 , wherein the fourth set of one or more convolutional neural networks is configured to also determine a maculopathy grade for the at least one fundus image.
4 . The method of claim 3 , wherein the fourth set of one or more convolutional neural networks is trained on a plurality of training fundus images of individuals having a HbA1c of 40 mmol/mol or greater.
5 . The method of claim 4 , wherein each of the training fundus images comprise at least one image label comprising one or more of: a clinically triaged retinopathy grade, and a clinically triaged maculopathy grade.
6 . The method of claim 2 , wherein determination of the risk level of progression of diabetic retinopathy is performed based on a plurality of factors comprising two or more of: baseline grade, age, Hba1c level, duration of diabetes, ethnicity, and insulin use.
7 . The method of claim 2 , further comprising the step of providing a recommendation for management of the individual's condition based on the determined risk level of progression of diabetic retinopathy.
8 . The method of claim 1 , wherein the first QA set of one or more convolutional neural networks is configured to classify the at least one fundus image as one of a plurality of categories, wherein at least a first one of the categories indicates the at least one fundus image is unsuitable for further processing using the first QA set of one or more convolutional neural networks, and a second one of the categories indicates the at least one fundus image is suitable for further processing using the first set of one or more convolutional neural networks.
9 . The method of claim 8 , wherein the plurality of categories comprises a third category indicating the at least one fundus image should be reviewed by a clinician, but is unsuitable for further processing using the first set of one or more convolutional neural networks.
10 . The method of claim 8 , wherein classifying the at least one image as unsuitable comprises one or more of: determining that the at least one fundus image is not directed to a relevant region of an eye of the individual, and determining that at least one property of the at least one fundus image is unsuitable.
11 . The method of claim 1 , further comprising the step of performing image adjustment on the at least one fundus image prior to processing using the first set of one or more convolutional neural networks.
12 . The method of claim 11 , wherein the image adjustment is normalisation of the at least one fundus image.
13 . The method of claim 1 , wherein the at least one fundus image comprises a plurality of fundus images, and the method further comprises processing the plurality of fundus images using a fifth set of one or more convolutional neural networks to classify each of the fundus images according to orientation.
14 . The method of claim 13 , wherein the fifth set of one or more convolutional neural networks is configured to group the fundus images according to the classification of left-eye or right-eye.
15 . The method of claim 13 , wherein the fifth set of one or more convolutional neural networks is configured to group the fundus images according to at least one identifier.
16 . The method of claim 1 , wherein the first QA set of one or more convolutional neural networks is trained on a plurality of training fundus images of individuals having stable HbA1c levels over a predetermined period of time.
17 . A system for detecting glycosylated haemoglobin (HbA1c) from at least one fundus image, the system comprising:
a memory storing program instructions;
a processor configured to execute program instructions stored in the memory and configured to:
process one or more fundus images associated with an individual using a first Quality Assurance (QA) set of one or more convolutional neural networks (CNNs) to determine whether the one or more fundus images are of sufficient quality for further processing;
process the one or more fundus images determined to be of sufficient quality for further processing using a second eye-identification set of one or more CNNs (eye-ID CNN), to identify the one or more fundus images belonging to a single eye; and
process the one or more fundus images belonging to the single eye using a third glycosylated haemoglobin (HbA1c) set of one or more convolutional neural networks to determine an HbA1c level for the individual,
wherein the at least one fundus image is a two-dimensional visible-light photograph.
18 . The system as claimed in claim 17 , wherein the processor is further configured to:
process the at least one fundus image using a fourth set of one or more convolutional neural networks to determine a retinopathy grade for the at least one fundus image; and
determine, based on at least the HbA1c level and the retinopathy grade, a risk level of progression of diabetic retinopathy of the individual.