Method and apparatus for training operation determination model for medical instrument control device
This system for training an operation determination model for a medical instrument control device generates training data by means of a reinforcement learning model, and, using the training data, can train an operation determination model configured to output information associated with operational commands for a driving unit that transports medical instruments.
1 . A method, performed by a processor, of training an operation determination model of a medical instrument control device, the method comprising:
when a medical instrument inserted into a vascular model reaches a branching region in the vascular model, identifying a procedure environment in the branching region;
selecting a reinforcement learning model corresponding to the identified procedure environment from a plurality of reinforcement learning models and training the selected reinforcement learning model by reinforcement learning based on a vascular patch image extracted for the branching region;
after training of the selected reinforcement learning model is finished, calculating a training output based on the selected reinforcement learning model from the vascular patch image for the branching region and generating training data in which the training output pairs with the extracted vascular patch image as a training input; and
training the operation determination model by supervised learning based on the generated training data.
2 . The method of claim 1 , wherein the identifying of the procedure environment comprises
when the medical instrument reaches the branching region, identifying a branching shape of the branching region based on the vascular patch image extracted for the branching region.
3 . The method of claim 2 , wherein the identifying of the branching shape comprises
identifying based on an angle difference between a direction of a main branch and a direction of branch closest to a branch point in the vascular patch image.
4 . The method of claim 1 , wherein the identifying of the procedure environment comprises
when the medical instrument reaches the branching region, identifying an orientation characteristic of a tip of the medical instrument in the branching region.
5 . The method of claim 4 , wherein the identifying of the orientation characteristic comprises:
when the medical instrument rotates in a predetermined rotation angle based on a longitudinal direction axis of a medical wire connected to a body of the medical instrument, observing an orientation direction of the tip of the medical instrument; and
calculating a ratio of observed directions during rotation of the medical instrument and determining the orientation characteristic based on the calculated ratio.
6 . The method of claim 1 , wherein the identifying of the procedure environment comprises
when a plurality of vascular patch images for the branching region is extracted, mapping a procedure environment, which is identified for one vascular patch image from among the plurality of vascular patch images, to the other vascular patch images.
7 . The method of claim 6 , wherein the identifying of the procedure environment comprises
until the medical instrument enters the branching region and reaches an outside of the branching region, extracting a plurality of vascular patch images related to the branching region based on a location of the medical instrument that changes each time the medical instrument drives,
wherein the training of the selected reinforcement learning model by reinforcement learning comprises
training the selected reinforcement learning model corresponding to the identified procedure environment, based on the plurality of vascular patch images related to the branching region.
8 . The method of claim 1 , further comprising:
preprocessing and simplifying the vascular patch image.
9 . The method of claim 8 , wherein the simplifying comprises
rotating the vascular patch image such that a proceeding direction of the medical instrument captured in the vascular patch image is oriented to one direction of the vascular patch image and a central axis of a branch where the medical instrument is located is aligned with an axis of the vascular patch image.
10 . The method of claim 1 , wherein the training of the selected reinforcement learning model by reinforcement learning comprises
when a reinforcement learning model corresponding to the identified procedure environment for the branching region is not found, excluding at least a portion of vascular patch images related to the branching region from training.
11 . The method of claim 10 , wherein the excluding comprises
excluding, from training based on reinforcement learning, vascular patch images related to a branching region having a branching shape with an angle difference that is out of a predetermined angular range designated to a plurality of reinforcement learning models.
12 . The method of claim 1 , wherein the training of the selected reinforcement learning model by reinforcement learning comprises
iteratively training the plurality of reinforcement learning models by using vascular patch images collected from a plurality of branching regions of one or more vascular models.
13 . The method of claim 1 , wherein the identifying of the procedure environment comprises
mapping the identified procedure environment to the vascular patch image,
wherein the generating of the training data comprises:
for each of the plurality of vascular patch images collected during training of the plurality of reinforcement learning models, loading a reinforcement learning model corresponding to the procedure environment that is mapped to the vascular patch image; and
generating the training output by applying the loaded reinforcement learning model to the vascular patch image.
14 . The method of claim 1 , wherein the training of the operation determination model comprises
updating a parameter of the operation determination model until a loss between the training output and an output calculated based on the operation determination model from the vascular patch image is less than a threshold loss.
15 . The method of claim 1 , further comprising:
while the medical instrument is inserted, calculating an expectation value for each operation command as an output by using the operation determination model from an input patch image that is extracted based on a location of the medical instrument inserted into a blood vessel without procedure environment information.
16 . The method of claim 15 , further comprising:
selecting an operation command having a greatest expectation value among expectation values calculated for each operation command; and
performing any one of proceeding, rotating, and retracting of the medical instrument by driving a driving unit connected to the medical instrument based on the selected operation command.
17 . A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a method comprising the steps of:
when a medical instrument inserted into a vascular model reaches a branching region in the vascular model, identifying a procedure environment in the branching region;
selecting a reinforcement learning model corresponding to the identified procedure environment from a plurality of reinforcement learning models and training the selected reinforcement learning model by reinforcement learning based on a vascular patch image extracted for the branching region;
after training of the selected reinforcement learning model is finished, calculating a training output based on the selected reinforcement learning model from the vascular patch image for the branching region and generating training data in which the training output pairs with the extracted vascular patch image as a training input; and
training the operation determination model by supervised learning based on the generated training data.
18 . A system for training an operation determination model of a medical instrument control device, the system comprising:
a memory configured to store a plurality of reinforcement learning models and operation determination models; and
a processor configured to, when a medical instrument inserted into a vascular model reaches a branching region in the vascular model, identify a procedure environment in the branching region, select a reinforcement learning model corresponding to the identified procedure environment among the plurality of reinforcement learning models, train the selected reinforcement learning model by reinforcement learning based on a vascular patch image extracted for the branching region, after training of the selected reinforcement learning model is finished, calculate a training output based on the selected reinforcement learning model from the vascular patch image for the branching region, generate training data in which the training output pairs with the extracted vascular patch image as a training input, and train the operation determination model by supervised learning based on the generated training data.