Ophthalmic image processing method, ophthalmic image processing device, and ophthalmic image processing program
An ophthalmic image of an evaluation target is acquired, a plurality of subsection images is extracted from the ophthalmic image, a state of a subject's eye is predicted for each of the subsection images based on a learned model in which learning has been performed in advance regarding extracting a plurality of subsection images from an ophthalmic image for learning, and predicting a state of a subject's eye for the each of subsection image by machine learning using correct answer data related to a state of each subsection image, and the subsection image is extracted from the ophthalmic image so as to have an image size corresponding to a state of a subject's eye of an evaluation target.
1. An ophthalmic image processing method for evaluating a state of a subject's eye from an ophthalmic image in which a subject's eye is shot using machine learning, the ophthalmic image processing method comprising:
a learning step of, in order to learn a state of a subject's eye set in advance as a prediction target, obtaining a learned model by performing learning with respect to a neural network in advance regarding extracting a plurality of subsection images from an ophthalmic image for learning, and predicting a state of a subject's eye for each subsection image by machine learning using correct answer data related to the state of the subject's eye of each subsection image;
an image acquisition step of acquiring an ophthalmic image for a test;
an extraction step of extracting a plurality of subsection images from an ophthalmic image for a test; and
a prediction step of predicting a state of a subject's eye of each subsection image using the learned model, wherein
the extraction of the plurality of subsection images is performed from the ophthalmic image after a predetermined image size is set for each state of a subject's eye that is the prediction target such that a detection correct answer rate becomes equal to or greater than a predetermined value in verification in advance regarding a relationship between a size of a subsection image and a detection correction rate of a state of a subject's eye.
2. The ophthalmic image processing method according to claim 1 , further comprising a result reflection region setting step of setting a result reflection region that is a region having a predetermined size equal to or less than a size of the subsection image, the result reflection region for reflecting a prediction result of the subsection image, wherein
in the extraction step, a region that includes the result reflection region and has a predetermined size equal to or greater than the result reflection region is extracted as the subsection image so as to correspond to each of the result reflection regions, and
in the prediction step, a prediction result obtained for each of the subsection images is reflected to the result reflection region that corresponds.
3. The ophthalmic image processing method according to claim 2 , further comprising an edge image generation step of extracting an edge from the ophthalmic image to generate an edge image, wherein
in the prediction step, when a luminance value in the edge image corresponding to a position of the result reflection region is equal to or greater than a predetermined threshold, the subsection image corresponding to the result reflection region is set as a target of prediction processing.
4. The ophthalmic image processing method according to claim 1 , wherein in the prediction step, a mean luminance value is calculated for each of the subsection images, and the subsection image having a mean luminance value equal to or greater than a predetermined threshold is set as a target of prediction processing.
5. The ophthalmic image processing method according claim 1 , further comprising a prediction result display step of causing a display means to display an image indicating a prediction result superimposed and displayed on the ophthalmic image.
6. The ophthalmic image processing method according to claim 1 , wherein
in the image acquisition step, a plurality of ophthalmic images arranged in time series is acquired,
in the prediction step, prediction processing is executed for each of a plurality of ophthalmic images, and
the ophthalmic image processing method further includes a change determination step of determining whether or not a temporal change and/or a spatial change in a state of a subject's eye satisfies a predetermined condition for a prediction result obtained for each of a plurality of ophthalmic images.
7. The ophthalmic image processing method according to claim 1 , wherein a state of the subject's eye is presence or absence of an interference fringe of a tear film.
8. The ophthalmic image processing method according to claim 1 , wherein a state of the subject's eye is presence or absence of a break.
9. The ophthalmic image processing method according to claim 1 , wherein
in the learning step, the learned model is obtained by performing learning with respect to the neural network in advance regarding extracting the plurality of subsection images from the ophthalmic image for learning, and, predicting by machine learning, for each of the subsection images, at least which of a healthy state, a state of tear decreasing dry eye, and a state of evaporative dry eye the state of dry eye of the subject's eye corresponds to, using the correct answer data related to the state of dry eye of the subject's eye of each of the subsection images, and
the plurality of subsection images are extracted from the ophthalmic image after an image size is set such that a detection correct answer rate becomes equal to or greater than a predetermined value in verification in advance regarding a relationship between a size of a subsection image and a detection correction rate of the state of dry eye of the subject's eye.
10. The ophthalmic image processing method according to claim 9 , further comprising a result reflection region setting step of setting a result reflection region that is a region having a predetermined size equal to or less than a size of the subsection image, the result reflection region for reflecting a prediction result of the subsection image, wherein
in the extraction step, a region that includes the result reflection region and has a predetermined size equal to or greater than the result reflection region is extracted as the subsection image so as to correspond to each of the result reflection regions, and
in the prediction step, a prediction result obtained for each of the subsection images is reflected to the result reflection region that corresponds.
11. The ophthalmic image processing method according to claim 10 , further comprising an edge image generation step of extracting an edge from the ophthalmic image to generate an edge image, wherein
in the prediction step, when a luminance value in the edge image corresponding to a position of the result reflection region is equal to or greater than a predetermined threshold, the subsection image corresponding to the result reflection region is set as a target of prediction processing.
12. The ophthalmic image processing method according to claim 9 , wherein in the prediction step, a mean luminance value is calculated for each of the subsection images, and the subsection image having a mean luminance value equal to or greater than a predetermined threshold is set as a target of prediction processing.
13. The ophthalmic image processing method according to claim 9 , further comprising a prediction result display step of causing a display means to display an image indicating a prediction result superimposed on the ophthalmic image.
14. The ophthalmic image processing method according to claim 9 , wherein
in the image acquisition step, a plurality of ophthalmic images arranged in time series are acquired,
in the prediction step, prediction processing is executed for each of a plurality of ophthalmic images, and
the ophthalmic image processing method further includes a change determination step of determining whether or not a temporal change and/or a spatial change in a state of dry eye of a subject's eye satisfies a predetermined condition for a prediction result obtained for each of a plurality of ophthalmic images.
15. An ophthalmic image processing device for evaluating a state of a subject's eye from an ophthalmic image in which a subject's eye is shot, the ophthalmic image processing device comprising:
an image acquisition unit that acquires the ophthalmic image of an evaluation target;
an extraction unit that extracts a plurality of subsection images from the ophthalmic image; and
a prediction unit that in order to learn a state of a subject's eye set in advance as a prediction target, predicts a state of a subject's eye for each of the subsection images based on a learned model in which learning has been performed with respect to a neural network in advance regarding extracting a plurality of subsection images from an ophthalmic image for learning, and predicting a state of a subject's eye for the each of subsection image by machine learning using correct answer data related to the state of the subject's eye of each subsection image, wherein
the subsection image is extracted from the ophthalmic image after an image size is set for each state of a subject's eye that is the prediction target such that a detection correct answer rate becomes equal to or greater than a predetermined value in verification in advance regarding a relationship between a size of a subsection image and a detection correction rate of a state of a subject's eye.
16. The ophthalmic image processing device according to claim 15 , further comprising a result reflection region setting unit that sets a result reflection region that is a region having a predetermined size equal to or less than a size of the subsection image, the result reflection region for reflecting a prediction result of the subsection image, wherein
the extraction unit extracts a region that includes the result reflection region and has a predetermined size equal to or greater than the result reflection region as the subsection image so as to correspond to each of the result reflection regions, and
the prediction unit reflects a prediction result obtained for each of the subsection images to the result reflection region that corresponds.
17. The ophthalmic image processing device according to claim 15 , wherein
the prediction unit predicts the state of dry eye of the subject's eye for each of the subsection images based on the learned model in which the learning has been performed with respect to a neural network in advance regarding extracting the plurality of subsection images from the ophthalmic image for learning, and predicting by machine learning, for each of the subsection images, at least which of a healthy state, a state of tear decreasing dry eye, and a state of evaporative dry eye the state of dry eye of the subject's eye corresponds to, using correct answer data related to the state of dry eye of the subject's eye of each of the subsection images, and
each of the subsection images is extracted from the ophthalmic image after an image size is set such that a detection correct answer rate becomes equal to or greater than a predetermined value in verification in advance regarding a relationship between a size of a subsection image and a detection correction rate of the state of dry eye of the subject's eye.
18. A non-transitory computer-readable medium storing an ophthalmic image processing program which, when executed, causes a computer to perform processing of evaluating a state of a subject's eye from an ophthalmic image in which a subject's eye is shot, the ophthalmic image processing program, when executed, causing the computer to implement:
an image acquisition function of acquiring the ophthalmic image of an evaluation target;
an extraction function of extracting a plurality of subsection images from the ophthalmic image; and
a prediction function of, in order to learn a state of a subject's eye set in advance as a prediction target, predicting a state of a subject's eye for each of the subsection images based on a learned model in which learning has been performed with respect to a neural network in advance regarding extracting a plurality of subsection images from an ophthalmic image for learning, and predicting a state of a subject's eye for the each of subsection images by machine learning using correct answer data related to the state of the subject's eye of each subsection image, wherein
the subsection image is extracted from the ophthalmic image after an image size is set for each state of a subject's eye that is the prediction target such that a detection correct answer rate becomes equal to or greater than a predetermined value in verification in advance regarding a relationship between a size of a subsection image and a detection correction rate of a state of a subject's eye.
19. The non-transitory computer-readable medium according to claim 18 , the ophthalmic image processing program, when executed, further causing the computer to implement a result reflection region setting function of setting a result reflection region that is a region having a predetermined size equal to or less than a size of the subsection image, the result reflection region for reflecting a prediction result of the subsection image, wherein
the extraction function extracts a region that includes the result reflection region and has a predetermined size equal to or greater than the result reflection region as the subsection image so as to correspond to each of the result reflection regions, and
the prediction function reflects a prediction result obtained for each of the subsection images to the result reflection region that corresponds.
20. The non-transitory computer-readable medium according to claim 18 , wherein
the prediction function predicts the state of dry eye of the subject's eye for each of the subsection images based on the learned model in which the learning has been performed with respect to the neural network in advance regarding extracting the plurality of subsection images from the ophthalmic image for learning, and, predicting by machine learning, for each of the subsection images, at least which of a healthy state, a state of tear decreasing dry eye, and a state of evaporative dry eye the state of dry eye of the subject's eye corresponds to, using the correct answer data related to the state of dry eye of the subject's eye of each of the subsection images, and
each of the subsection images is extracted from the ophthalmic image after an image size is set such that a detection correct answer rate becomes equal to or greater than a predetermined value in verification in advance regarding a relationship between a size of a subsection image and a detection correction rate of the state of dry eye of the subject's eye.