AI-based video analysis of cataract surgery for dynamic anomaly recognition and correction
A computer-implemented method for recognizing deviations from plan parameters during an ophthalmological operation is described, the method including: providing video sequences of cataract operations, the video sequences having been recorded by means of an image recording apparatus, training a machine learning system using the video sequences provided and also, in each case, a planned refractive power of an intraocular lens to be inserted during a cataract operation and a target refraction value following the cataract operation as training input data and associated prediction results in the form of an actual refraction value following the cataract operation to form a machine learning model for predicting the actual refraction value following the cataract operation, and persistently storing parameter values of the trained machine learning system.
1 . A computer-implemented method for recognizing deviations from plan parameter values during a cataract operation, the method comprising:
providing video sequences of cataract operations, the video sequences having been recorded using one or more image recording apparatuses;
training a machine learning system using as training input data:
the video sequences, and
for each cataract operation:
a planned refractive power of an intraocular lens to be inserted during the cataract operation;
a target refraction value following the cataract operation; and
an associated prediction result comprising an actual refraction value following the cataract operation,
wherein the machine learning system is trained to form a machine learning model for dynamically predicting, during a cataract operation, the actual refraction value following the cataract operation, and predicting a deviation value between a target refraction value following the cataract operation and a predicted actual refraction value following the cataract operation; and
persistently storing parameter values of the trained machine learning system.
2 . The method of claim 1 , wherein additional input data for the machine learning system includes at least one of ophthalmological measurement data before the cataract operation on an eye to be operated on or a shape of the intraocular lens to be inserted.
3 . The method of claim 1 , wherein the machine learning system comprises a recurrent neural network.
4 . The method of claim 1 , wherein the machine learning system comprises a 3-D convolutional neural network.
5 . The method of claim 1 , wherein a type of cataract operation is used as additional training input data during the training to form a learning model.
6 . The method of claim 1 , further comprising:
recording a video sequence using an image recording apparatus during a current cataract operation; and
dynamically predicting the actual refraction value following the current cataract operation using the trained machine learning system, the recorded video sequence of the current cataract operation being continuously and dynamically supplied to the trained machine learning system as input data, and a current target refraction value and a current planned refractive power of an intraocular lens to be inserted being used as further input data for the trained machine learning system.
7 . The method of claim 6 , wherein additional input data are used for the trained machine learning system for the dynamic prediction of the actual refraction value following the current cataract operation, the additional input data including at least one of ophthalmological measurement data of an eye to be operated on before the cataract operation or a shape of the intraocular lens to be inserted.
8 . The method of claim 6 , further comprising:
at least one of determining a refraction deviation value from the planned refractive power of the intraocular lens to be inserted and the predicted actual refraction value following the current cataract operation, or dynamically determining a new refractive power of the intraocular lens to be inserted during a cataract operation; and
visualizing at least one of:
the planned refractive power of the intraocular lens to be inserted,
the target refraction value,
the refraction deviation value,
the new refractive power of the intraocular lens to be inserted, or
a shape of the intraocular lens to be inserted.
9 . The method of claim 6 , wherein a type of cataract operation is used as additional input value during the dynamic prediction of the actual refraction value.
10 . The method of claim 9 , wherein the type of cataract operation is based on phacoemulsification, employs a Yamane technique, relates to an insertion of an anterior chamber intraocular lens, or relates to a fixation of the intraocular lens in the sulcus.
11 . The method of claim 1 , wherein the machine learning system is pre-trained.
12 . The method of claim 1 , wherein the intraocular lens to be inserted comprises a spherical, toric, or multifocal intraocular lens to be inserted.
13 . The method of claim 1 , wherein the trained machine learning system comprises an explaining machine learning system.
14 . An operation assistance system for recognizing deviations from plan parameter values during a cataract operation, the operation assistance system comprising:
memory that stores program code; and
one or more processors that are connected to the memory and that, when they execute the program code, prompt the operation assistance system to control;
a video sequence storage apparatus for providing video sequences of cataract operations, the video sequences having been recorded using one or more image recording apparatuses;
a training control system for training a machine learning system using, as training input data, the video sequences and, for each cataract operation:
a planned refractive power of an intraocular lens to be inserted during the cataract operation; a target refraction value following the cataract operation; and
an associated prediction result comprising an actual refraction value following the cataract operation,
wherein the machine learning system is trained to form a machine learning model for dynamically predicting, during a cataract operation, the actual refraction value following the cataract operation, and predicting a deviation value between a target refraction value following the cataract operation and a predicted actual refraction value following the cataract operation; and
parameter value memory for persistently storing parameter values of the trained machine learning system.
15 . A computer-readable storage medium storing a computer program product for recognizing deviations from plan parameters during a cataract operation, wherein the computer program product, when executed by one or more computers or control units, cause the one or more computers or control units to perform operations comprising:
providing video sequences of cataract operations, the video sequences having been recorded using one or more image recording apparatuses,
training a machine learning system using as training input data:
the video sequences, and
for each cataract operation:
a planned refractive power of an intraocular lens to be inserted during the cataract operation;
a target refraction value following the cataract operation; and
an associated prediction result comprising an actual refraction value following the cataract operation,
wherein the machine learning system is trained to form a machine learning model for dynamically predicting, during a cataract operation, the actual refraction value following the cataract operation, and predicting a deviation value between a target refraction value following the cataract operation and a predicted actual refraction value following the cataract operation; and
persistently storing parameter values of the trained machine learning system.
16 . The computer-readable storage medium of claim 15 , wherein additional input data for the machine learning system includes at least one of ophthalmological measurement data before the cataract operation on an eye to be operated on or a shape of the intraocular lens to be inserted.
17 . The computer-readable storage medium of claim 15 , wherein the machine learning system comprises a recurrent neural network.
18 . The computer-readable storage medium of claim 15 , wherein the machine learning system comprises a 3-D convolutional neural network.
19 . The computer-readable storage medium of claim 15 , wherein a type of cataract operation is used as additional training input data during the training to form a learning model.
20 . The computer-readable storage medium of claim 15 , wherein the operations further comprise:
recording a video sequence using an image recording apparatus during a current cataract operation; and
dynamically predicting the actual refraction value following the current cataract operation using the trained machine learning system, the recorded video sequence of the current cataract operation being continuously and dynamically supplied to the trained machine learning system as input data, and a current target refraction value and a current planned refractive power of an intraocular lens to be inserted being used as further input data for the trained machine learning system.