Method of determining hysteresis of surgical robot, method of compensating for the same, and endoscopic surgical apparatus

1. A method of determining hysteresis of a surgical robot according to a drive input, the method comprising:

generating a hysteresis evaluation model for the surgical robot based on similarities between (i) an actual image of the surgical robot and (ii) images rendered by driving a virtual surgical robot on a simulation that models the surgical robot based on a kinetic model;

capturing a real-time image of the surgical robot driven according to a drive input; and

measuring a drive angle of the surgical robot by comparing the real-time image to a previously rendered image data set of the surgical robot, and calculating a difference between the drive input and the measured drive angle.

2. The method of claim 1 , wherein the generating of the hysteresis evaluation model comprises:

extracting a first feature vector by inputting rendered image data, which are a set of the rendered images, into a first neural network model;

extracting a second feature vector by inputting actual image data of the surgical robot driven according to a drive input into a second neural network model; and

training the first neural network model and the second neural network model by supervised learning, based on a difference in relative magnitudes of respective drive inputs of the rendered images and the actual image and on a difference in relative magnitudes between the first feature vector and the second feature vector.

3. The method of claim 2 , wherein the first feature vector and the second feature vector have the same number of dimensions, and

the training comprises training the first neural network model and the second neural network model such that a norm size between the first feature vector and the second feature vector decreases as the respective drive inputs of the rendered images and the actual image are more similar to each other.

4. The method of claim 3 , wherein the surgical robot comprises a proximal joint part and a distal joint part that have different rotation axes are articulated and driven,

the drive inputs comprise a drive angle displacement of the proximal joint part and a drive angle displacement of the distal joint part, and

the generating of the hysteresis evaluation model comprises generating at least two hysteresis evaluation models, for each interval of the drive angle displacement of any one of the proximal joint part and the distal joint part.

5. The method of claim 4 , wherein the generating of the hysteresis evaluation model comprises generating at least two hysteresis evaluation models, for each interval of the drive angle displacement of the proximal joint part.

6. The method of claim 2 , wherein the generating of the hysteresis evaluation model further comprises preprocessing the actual image by separating and removing a background image except for the surgical robot from the actual image.

7. The method of claim 2 , wherein the calculating of the difference comprises:

extracting a real-time feature vector by inputting the real-time image into the second neural network model;

retrieving a rendered image having the first feature vector most similar to the real-time feature vector; and

determining the measured drive angle based on a drive input of the retrieved rendered image.

8. The method of claim 7 , wherein the retrieving of the rendered image comprises indexing using a binary tree structure having the number of dimensions the same as the number of dimensions of the first feature vector.

9. The method of claim 8 , wherein the calculating of the difference further comprises preprocessing the real-time image by separating and removing a background image except for the surgical robot from the real-time image.

10. The method of claim 9 , wherein the preprocessing further comprises adjusting a scale of the real-time image based on a distance from a camera capturing the real-time image of the surgical robot to a proximal joint part of the surgical robot.

11. A method of compensating for hysteresis of a surgical robot according to a drive input, the method comprising:

generating a hysteresis evaluation model for the surgical robot based on similarities between (i) an actual image of the surgical robot and (ii) images rendered by driving a virtual surgical robot on a simulation that models the surgical robot based on a kinetic model;

capturing a real-time image of the surgical robot driven according to a drive input;

measuring a drive angle of the surgical robot by comparing the real-time image to a previously rendered image data set of the surgical robot, and determining hysteresis of the surgical robot by calculating a difference between the drive input and the measured drive angle; and

compensating for the drive input based on the determined hysteresis.

12. The method of claim 11 , wherein the compensating comprises compensating for the drive input with feedback control based on the difference between the drive input and the measured drive angle.

13. An endoscopic surgical apparatus, comprising:

a tube;

a surgical robot received in the tube, the surgical robot having a drive shaft with one or more degrees of freedom;

an endoscopy camera configured to capture a actual image of the surgical robot at an end of the tube;

a driver configured to drive the surgical robot;

a controller configured to control the drive of the surgical robot by controlling the driver; and

a drive input part configured to receive a drive input from a user,

wherein the controller comprises a hysteresis evaluation model, wherein the hysteresis evaluation model is generated based on similarities between the actual image of the surgical robot and images rendered by driving a virtual surgical robot on a simulation that models the surgical robot based on a kinetic model,

wherein the controller comprises at least one processor configured to compensate for hysteresis of the surgical robot based on the drive input and a real-time image captured by the endoscopy camera.

14. The endoscopic surgical apparatus of claim 13 , wherein the hysteresis evaluation model for the surgical robot comprises:

a first neural network model configured to receive the rendered images and output a first feature vector of a plurality of dimensions;

a second neural network model configured to receive the actual image of the surgical robot and output a second feature vector having the same number of dimensions as the first feature vector; and

a supervised learning model configured to train the first neural network model and the second neural network model by supervised learning, such that a difference in relative magnitudes of respective drive inputs of the rendered images and the actual image is proportional to a difference in relative magnitudes between the first feature vector and the second feature vector.

15. The endoscopic surgical apparatus of claim 14 , wherein the hysteresis evaluation model for the surgical robot further comprises:

an index part configured to store first feature vectors extracted respectively for the rendered images in a binary tree structure having the number of dimensions the same as the number of dimensions of the first feature vector.

16. The endoscopic surgical apparatus of claim 14 , wherein the surgical robot comprises a proximal joint part and a distal joint part that have different rotation axes are articulated and driven,

the drive inputs comprise a drive angle displacement of the proximal joint part and a drive angle displacement of the distal joint part, and

the hysteresis evaluation model is generated as at least two different models, for each interval of a drive angle displacement of any one of the proximal joint part and the distal joint part.

17. The endoscopic surgical apparatus of claim 14 , wherein the controller is configured to extract a real-time feature vector by inputting a real-time image of the surgical robot into the second neural network model, and determine a drive error according to the drive input based on a rendered image having the first feature vector most similar to the real-time feature vector.

18. The endoscopic surgical apparatus of claim 17 , wherein the controller further comprises a drive compensator configured to compensate for the drive of the driver based on a size of the drive error evaluated according to the drive input.

19. The endoscopic surgical apparatus of claim 18 , wherein the drive compensator is configured to compensate for the drive input with feedback control based on the size of the drive error.