Semi-supervised fundus image quality assessment method using IR tracking
System/Method/Device for labelling images in an automated manner to satisfy a performance of a different algorithm and then applying active learning to learn a deep learning model which would enable ‘real-time’ operation of quality assessment and with high accuracy.
1 . An ophthalmic imaging device, comprising:
an imaging unit for capturing a sequence of images of a patient's eye;
an image assessment unit configured to, for one or more sample images selected from among the sequence of images, determine a similarity measure between each sample image and one or more other images in the sequence of images; and
a learning model trained to assign a classification to an input image;
wherein the learning model is retrained using a selection of the assessed sample images based on their respective similarity measure,
wherein the imaging unit is an optical coherence tomography (OCT) unit,
wherein the sequence of images is a sequence of B-scans comprising a cube-scan, and
wherein the assessed sample images that are used to retrain the learning model are a fraction of the B-scans that are selected for being the most dissimilar to the rest of the B-scans in the cube-scan, as determined by their respective similarity measures.
2 . The ophthalmic imaging device of claim 1 , wherein the learning model is updated by retraining a stored version of the learning model, and the retrained version of the learning model replaces the existing learning model.
3 . The ophthalmic imaging device of claim 2 , wherein the stored version of the learning model is stored and retrained within the ophthalmic imaging device.
4 . The ophthalmic imaging device of claim 2 , wherein:
the ophthalmic imaging device includes a communication system for transmitting assessed images to a remote service site; and
the stored version of the learning model is stored and retrained at the remote service site and the retrained version of the learning model is transmitted from the remote service site to the ophthalmic imaging device to replace the existing learning model at the ophthalmic imaging device.
5 . The ophthalmic imaging device of claim 1 , wherein the learning model designates a confidence measure to an assigned classification of the learning model, and assessed images whose classifications have designated confidence measures above a predefined threshold are used to retrain the learning model.
6 . The ophthalmic imaging device of claim 1 , wherein:
the learning model determines a confidence measure for each image assigned classification;
assessed images whose assigned classifications have a confidence measure below a predefined threshold are flagged for manual inspection and selective reclassification based on the visual inspection; and
the learning model is retrained using the reclassified images.
7 . The ophthalmic imaging device of claim 1 , wherein the learning model is trained to assign a first classification indicating that an input image is suitable for further processing and a second classification indicating that the input image is not suitable for further processing.
8 . The ophthalmic imaging device of claim 1 , wherein:
a select image within the sequence of images is designated a reference image;
the similarity measure is based on the similarity between each sample image and the reference image; and
selection of the reference image is based on the output of the learning model.
9 . The ophthalmic imaging device of claim 7 , wherein images assigned the first classification are submitted to an image processing module configured to identify a tissue type within the input image and input images assigned the second classification are not submitted to the image processing module.
10 . The ophthalmic imaging device of claim 9 , wherein the image processing module is a secondary machine learning module within the ophthalmic imaging device.
11 . The ophthalmic imaging device of claim 1 , wherein for each B-scan being assessed, the similarity measure is based on the square differences (SqD) or the cross correlation (CC) of the B-scan being assessed and the other B-scans in the cube-scan.
12 . The ophthalmic imaging device of claim 1 , wherein the imaging unit is an imaging system for an anterior segment imaging of the eye, an imaging system for posterior segment imaging of the eye, an optical coherence tomography (OCT) system, or an OCT angiography (OCTA) system.
13 . The ophthalmic imaging device of claim 1 , wherein the learning model is based on a Visual Geometry Group (VGG), Residual Neural Network (ResNet), EfficientNet-B0, convolutional neural network, U-net, or deep learning neural network.
14 . The ophthalmic imaging device of claim 1 , wherein:
the sequence of images is divided into a plurality of image groups;
a select image within each image group is designated a reference image; and
the similarity measure is based on the similarity between the sample image and the reference image of the image group to which the sample image belongs.
15 . The ophthalmic imaging device of claim 14 , wherein the images within each image group are ordered based on the original sequence of the images within the sequence of images.
16 . The ophthalmic imaging device of claim 1 , wherein the learning model is configured to receive a plurality of input images as an input group, and assign the classification to at least one image within the input group.
17 . The ophthalmic imaging device of claim 1 , wherein the assigned classification is an indicator of motion tracking, image quality assessment (IQA), fluid flow, tissue structure, and decease disease type.
18 . An ophthalmic imaging device, comprising:
an imaging unit for capturing a sequence of images of a patient's eye;
an image assessment unit configured to, for one or more sample images selected from among the sequence of images, determine a similarity measure between each sample image and one or more other images in the sequence of images;
a learning model trained to assign a classification to an input image,
wherein the learning model is retrained using a selection of the assessed sample images based on their respective similarity measure, and
wherein the learning model is trained to assign a first classification indicating that an input image is suitable for further processing and a second classification indicating that an input image is not suitable for further processing;
wherein the image assessment unit is a motion tracking system,
wherein the similarity measure is based on motion tracking parameters determined by the motion tracking system,
wherein the assessed sample images are assigned the first classification label based on the assessed sample images respective similarity measure, and
wherein the learning model is retrained using the assessed sample images that received the first classification from the image assessment unit.
19 . The ophthalmic imaging device of claim 18 , wherein the further processing is motion tracking, and the output of the learning model is sent to the image assessment unit for processing.