Robot-assisted automatic introduction method and device for trocar

1 . A robot-assisted automatic introduction method for a trocar, comprising:

acquiring a dataset of the trocar, and training through a preset U-Net to acquire a first model;

detecting, by the first model, a position of the trocar in a target image, and outputting position information of the trocar that meets a preset condition;

parameterizing, according to the position information of the trocar that meets the preset condition, a rotation angle of the trocar, and acquiring a rotation matrix of the trocar; and

acquiring, according to the rotation matrix of the trocar, an orientation of the trocar, and controlling, according to the orientation of the trocar, an instrument at an end of a robot to introduce the trocar;

wherein the step of outputting the position information of the trocar that meets the preset condition comprises:

outputting the position information of the trocar that meets the following condition:

pred

⁡

(

x

,

y

)

≥

0.8

*

max

⁡

(

pred

)

;

wherein (x,y) denotes a pixel position of the trocar; pred(x,y) denotes a confidence of the pixel position (x,y) being classified as the trocar; and max(pred) denotes an overall maximum value of an image output by the U-Net.

2 . The robot-assisted automatic introduction method for the trocar according to claim 1 , wherein the step of acquiring the rotation matrix of the trocar comprises:

acquiring a rotation matrix R z of the trocar in a z-direction and a rotation matrix R y of the trocar in a y-direction, and calculating a six-dimensional rotation matrix R of the trocar;

R

=

[

R

⁢

1

R

⁢

2

R

⁢

3

]

,

{

R

⁢

1

=

ϕ

⁡

(

R

Z

)

R

⁢

3

=

ϕ

⁡

(

R

⁢

1

×

R

Y

)

R

⁢

2

=

R

⁢

3

×

R

⁢

1

;

wherein R1, R2, and R3 denote columns of the six-dimensional rotation matrix R.

3 . The robot-assisted automatic introduction method for the trocar according to claim 2 , wherein the step of acquiring, according to the rotation matrix of the trocar, the orientation of the trocar comprises:

acquiring a true value

R

Z

pred

 of the trocar in the dataset, and calculating the orientation of the trocar according to the rotation matrix

R

Z

gt

 of the trocar:

Δθ

=

across

(

R

Z

gt

·

R

Z

pred

)

;

wherein the orientation of the trocar is expressed by an angle Δθ between the rotation matrix

R

Z

gt

 of the trocar and the true value

R

Z

pred

 of the trocar.

4 . The robot-assisted automatic introduction method for the trocar according to claim 2 , wherein the step of acquiring, according to the rotation matrix of the trocar, the orientation of the trocar comprises:

acquiring a true value

R

z

pred

 of the trocar in the dataset, and calculating the orientation of the trocar according to the rotation matrix

R

z

gt

 of the trocar:

Δθ

=

across

(

R

Z

gt

·

R

Z

pred

)

;

wherein the orientation of the trocar is expressed by an angle Δθ between the rotation matrix

R

z

gt

 of the trocar and the true value

R

z

pred

 of the trocar.

5 . A robot-assisted automatic introduction method for a trocar, comprising:

acquiring a dataset of the trocar, and training through a preset U-Net to acquire a first model;

detecting, by the first model, a position of the trocar in a target image, and outputting position information of the trocar that meets a preset condition;

parameterizing, according to the position information of the trocar that meets the preset condition, a rotation angle of the trocar, and acquiring a rotation matrix of the trocar; and

acquiring, according to the rotation matrix of the trocar, an orientation of the trocar, and controlling, according to the orientation of the trocar, an instrument at an end of a robot to introduce the trocar;

wherein before parameterizing the rotation angle of the trocar, the robot-assisted automatic introduction method further comprises: a step of processing the position information of the trocar that meets the preset condition, comprising:

calculating a median value of the position information in every seven consecutive image frames, calculating a Euclidean distance between the median value and the position information in each of the seven consecutive image frames, and averaging position information with the Euclidean distance less than or equal to a quarter of a standard deviation to acquire final position information of the trocar.

6 . The robot-assisted automatic introduction method for the trocar according to claim 5 , wherein the step of acquiring the rotation matrix of the trocar comprises:

acquiring a rotation matrix R z of the trocar in a z-direction and a rotation matrix R y of the trocar in a v-direction, and calculating a six-dimensional rotation matrix R of the trocar;

R

=

[

R

⁢

1

R

⁢

2

R

⁢

3

]

,

{

R

⁢

1

=

ϕ

⁡

(

R

Z

)

R

⁢

3

=

ϕ

⁡

(

R

⁢

1

×

R

Y

)

R

⁢

2

=

R

⁢

3

×

R

⁢

1

;

wherein R1, R2, and R3 denote columns of the six-dimensional rotation matrix R.

7 . The robot-assisted automatic introduction method for the trocar according to claim 6 , wherein the step of acquiring, according to the rotation matrix of the trocar, the orientation of the trocar comprises:

acquiring a true value

R

z

pred

 of the trocar in the dataset, and calculating the orientation of the trocar according to the rotation matrix

R

z

gt

 of the trocar:

Δθ

=

across

(

R

Z

gt

·

R

Z

pred

)

;

wherein the orientation of the trocar is expressed by an angle Δθ between the rotation matrix

R

z

gt

 of the trocar and the true value

R

z

pred

 of the trocar.

8 . A robot-assisted automatic introduction device for a trocar, comprising a training module, a detection module, a rotation matrix module, and an introduction module, wherein

the training module is configured to acquire a dataset of the trocar, and train through a preset U-Net to acquire a first model;

the detection module is configured to detect, by the first model, a position of the trocar in a target image, and output position information of the trocar that meets a preset condition;

the rotation matrix module is configured to parameterize, according to the position information of the trocar that meets the preset condition, a rotation angle of the trocar, and acquire a rotation matrix of the trocar; and

the introduction module is configured to acquire, according to the rotation matrix of the trocar, an orientation of the trocar, and control, according to the orientation of the trocar, an instrument at an end of a robot to introduce the trocar;

wherein the detection module is configured to output the position information of the trocar that meets the preset condition, wherein

the detection module is configured to output the position information of the trocar that meets the following condition:

pred

⁡

(

x

,

y

)

⩾

0.8

*

max

⁡

(

pred

)

;

wherein (x,y) denotes a pixel position of the trocar; pred(x,y) denotes a confidence of the pixel position (x,y) being classified as the trocar; and max(pred) denotes an overall maximum value of an image output by the U-Net.

9 . The robot-assisted automatic introduction device for the trocar according to claim 8 , further comprising a screening module configured to process the position information of the trocar that meets the preset condition before the rotation matrix module parameterizes the rotation angle of the trocar, wherein

the screening module is configured to calculate a median value of the position information in every seven consecutive image frames, calculate a Euclidean distance between the median value and the position information in each of the seven consecutive image frames, and average position information with the Euclidean distance less than or equal to a quarter of a standard deviation to acquire final position information of the trocar.

10 . The robot-assisted automatic introduction device for the trocar according to claim 9 , wherein the rotation matrix module is configured to acquire the rotation matrix of the trocar, wherein

the rotation matrix module is configured to acquire a rotation matrix R z of the trocar in a z-direction and a rotation matrix R y of the trocar in a y-direction, and calculate a six-dimensional rotation matrix R of the trocar;

R

=

[

R

⁢

1

R

⁢

2

R

⁢

3

]

,

{

R

⁢

1

=

ϕ

⁡

(

R

Z

)

R

⁢

3

=

ϕ

⁡

(

R

⁢

1

×

R

Y

)

R

⁢

2

=

R

⁢

3

×

R

⁢

1

;

wherein R1, R2, and R3 denote columns of the six-dimensional rotation matrix R.

11 . The robot-assisted automatic introduction device for the trocar according to claim 10 , wherein the introduction module is configured to acquire, according to the rotation matrix of the trocar, the orientation of the trocar, wherein

the introduction module is configured to acquire a true value

R

z

pred

 of the trocar in the dataset, and calculate the orientation of the trocar according to the rotation matrix

R

z

gt

 of the trocar:

Δθ

=

across

(

R

Z

gt

·

R

Z

pred

)

;

wherein the orientation of the trocar is expressed by an angle Δθ between the rotation matrix

R

z

gt

 of the trocar and the true value

R

z

pred

 of the trocar.

12 . The robot-assisted automatic introduction device for the trocar according to claim 8 , wherein the rotation matrix module is configured to acquire the rotation matrix of the trocar, wherein

the rotation matrix module is configured to acquire a rotation matrix R z of the trocar in a z-direction and a rotation matrix R y of the trocar in a y-direction, and calculate a six-dimensional rotation matrix R of the trocar;

R

=

[

R

⁢

1

R

⁢

2

R

⁢

3

]

,

{

R

⁢

1

=

ϕ

⁡

(

R

Z

)

R

⁢

3

=

ϕ

⁡

(

R

⁢

1

×

R

Y

)

R

⁢

2

=

R

⁢

3

×

R

⁢

1

;

wherein R1, R2, and R3 denote columns of the six-dimensional rotation matrix R.

13 . The robot-assisted automatic introduction device for the trocar according to claim 12 , wherein the introduction module is configured to acquire, according to the rotation matrix of the trocar, the orientation of the trocar, wherein

the introduction module is configured to acquire a true value

R

Z

pred

 of the trocar in the dataset, and calculate the orientation of the trocar according to the rotation matrix

R

z

gt

 of the trocar:

Δθ

=

across

(

R

Z

gt

·

R

Z

pred

)

;

wherein the orientation of the trocar is expressed by an angle Δθ between the rotation matrix

R

z

gt

 of the trocar and the true value

R

z

pred

 of the trocar.

14 . The robot-assisted automatic introduction device for the trocar according to claim 8 , wherein the rotation matrix module is configured to acquire the rotation matrix of the trocar, wherein

the rotation matrix module is configured to acquire a rotation matrix R z of the trocar in a z-direction and a rotation matrix R y of the trocar in a y-direction, and calculate a six-dimensional rotation matrix R of the trocar;

R

=

[

R

⁢

1

R

⁢

2

R

⁢

3

]

,

{

R

⁢

1

=

ϕ

⁡

(

R

Z

)

R

⁢

3

=

ϕ

⁡

(

R

⁢

1

×

R

Y

)

R

⁢

2

=

R

⁢

3

×

R

⁢

1

;

wherein R1, R2, and R3 denote columns of the six-dimensional rotation matrix R.

15 . The robot-assisted automatic introduction device for the trocar according to claim 14 , wherein the introduction module is configured to acquire, according to the rotation matrix of the trocar, the orientation of the trocar, wherein

the introduction module is configured to acquire a true value

R

z

pred

 of the trocar in the dataset, and calculate the orientation of the trocar according to the rotation matrix

R

z

gt

 of the trocar:

Δθ

=

across

(

R

Z

gt

·

R

Z

pred

)

;

wherein the orientation of the trocar is expressed by an angle Δθ between the rotation matrix

R

z

gt

 of the trocar and the true value

R

z

pred

 of the trocar.