Computer implemented method, a computing device and a system for assisting bending of a reinforcing rod for attachment to a plurality of chirurgical implants

1 . A computer implemented method of assisting bending of a reinforcing rod ( 10 ), the method comprising the steps carried out by a computing device ( 100 ):

a. receiving spatial positions (P 1-n ) of a plurality of chirurgical implants ( 20 1-n ) captured by a camera-based positioning device ( 50 );

b. calculating a rod shape ( 10 c ) corresponding to the spatial positions (P 1-n ) of the plurality of chirurgical implants ( 20 1-n ), allowing the plurality of chirurgical implants ( 20 1-n ) to be attached to the reinforcing rod ( 10 );

c. based on the calculated rod shape ( 10 c ), calculating a sequence of bending parameter set(s);

d. generating tool operation guidance for bending tool(s) ( 70 ) based on the bending parameter set(s), the tool operation guidance indicating a sequence of prescribed operation steps (S 1-n ) of the bending tool(s) ( 70 ), wherein the sequence of prescribed operation steps (S 1-n ) are determined such, that when carried out using the bending tool(s) ( 70 ), causes the bending tool(s) ( 70 ) to shape the reinforcing rod ( 10 ) from an initial shape ( 10 i ) to a shaped form ( 10 s ) corresponding to the calculated rod shape ( 10 c ); and

e. generating augmented reality data based on the tool operation guidance, the augmented reality data comprising overlay(s) representing one or more of the prescribed operation steps (S 1-n ) of the tool operation guidance (OG 1-n ) for superimposing on a user's view of the bending tool(s) ( 70 ), the overlay(s) comprising visualization(s) of the bending tool(s) ( 70 ) in actuated position(s) which actuated position(s) cause the reinforcing rod ( 10 ) to be bent according to the bending parameter set(s).

2 . The method according to claim 1 , further comprising controlling the camera-based positioning device ( 50 ) to display the overlay(s) representing the tool operation guidance (OG 1-n ) on a display device ( 52 ) comprising a see-through display device ( 52 ) of the camera-based positioning device ( 50 ), the overlay(s) (OG 1-n ) being superimposed on a user's view of the bending tool(s) ( 70 ) through the display device ( 52 ) and/or on the display device ( 52 ).

3 . The method according to claim 1 , further comprising:

a. controlling the camera-based positioning device ( 50 ) to track a progress of a user's operation of the bending tool(s) ( 70 ) in accordance with the sequence of prescribed operation steps (S 1-n ) of the bending tool(s) ( 70 ); and

b. controlling the display device ( 52 ) to display the overlay (OG 1-n ) representing a particular operation step (Sx) of the sequence of prescribed operation steps (S 1-n ) of the tool operation guidance (OG 1-n ) according to the tracking.

4 . The method according to claim 1 , further comprising generating augmented reality data based on the calculated rod shape ( 10 c ), the augmented reality data comprising an overlay representing the calculated rod shape (O 10 c ).

5 . The method according to claim 4 , further comprising controlling the camera-based positioning device ( 50 ) such as to display the overlay representing the calculated rod shape (O 10 c ), on a display device ( 52 ) comprising a see-through display device ( 52 ) of the camera-based positioning device ( 50 ), the overlay representing the calculated rod shape (O 10 c ) being superimposed on a user's view of the reinforcing rod ( 10 ) through the display device ( 52 ) and/or on the display device ( 52 ).

6 . The method according to claim 1 , wherein the tool operation guidance is generated based on at least one of:

i. parameter(s) of the bending tool(s) ( 70 );

ii. parameter(s) of the reinforcing rod ( 10 ); and

iii. the positions of fixation points ( 22 ) of the plurality of chirurgical implants ( 20 1-n ).

7 . The method according to claim 1 , further comprising:

a. controlling the camera-based positioning device ( 50 ) to capture image(s) of the plurality of chirurgical implants ( 20 1-n ); and

b. determining the positions of the plurality of chirurgical implants ( 20 1-n ) based on the captured images.

8 . The method according to claim 7 , wherein the computing device controls the camera-based positioning device ( 50 ) to capture stereo images of the plurality of chirurgical implants ( 20 1-n ) using image capture sensors ( 54 L, 54 R) of the camera-based positioning device ( 50 ), and wherein the computing device determines the spatial positions (P 1-n ) of the plurality of chirurgical implants ( 20 1-n ) by processing the stereo images.

9 . The method according to claim 8 , further comprising:

a. controlling the camera-based positioning device ( 50 ) to capture a stream of stereo images of the plurality of chirurgical implants ( 20 1-n ); and

b. iteratively refining the determined spatial positions (P 1-n ) of the plurality of chirurgical implants ( 20 1-n ) by processing the stream of stereo images.

10 . The method according to claim 8 , wherein the computing device determines the spatial positions (P 1-n ) of the plurality of chirurgical implants ( 20 1-n ) by processing the stereo images using a stereo neural network, the neural network having been trained with a dataset of stereo images of chirurgical implants and corresponding annotations indicative of spatial positions of chirurgical implants.

11 . The method according to claim 1 , wherein a bending parameter set(s) comprises at least one of:

i. a rod distance (dARP 1-n );

ii. an axial reorientation angle (α 1-n );

iii. a rod bending angle (θ 1-n ); and

iv. a bending radius (R 1-n ).

12 . The method according to claim 1 , further comprising:

a. controlling the camera-based positioning device ( 50 ) to capture image(s) of the bending tool(s) ( 70 ); and

i. identifying the bending tools(s) ( 70 ) using the captured images and retrieve parameter(s) of the bending tool(s) ( 70 ) based on the identification; or

ii. determining parameter(s) of the bending tool(s) ( 70 ) using the captured images.

13 . A computing device ( 100 ) comprising a processing unit ( 120 ) and a memory unit ( 130 ) comprising instructions, which, when executed by the processing unit ( 120 ) cause the computing device ( 100 ) to carry out the method according to claim 1 .

14 . A system ( 1 ) for assisting bending of a reinforcing rod ( 10 ), the system ( 1 ) comprising:

a. a computing device ( 100 ) according to claim 12 ; and

b. a camera-based positioning device ( 50 ) communicatively connected to the computing device ( 100 ), the camera-based positioning device ( 50 ) comprising two or more image capture sensors ( 54 L, 54 R) and a display device ( 52 ) for the display of overlay(s) superimposed on a user's view.

15 . A computer program product, comprising instructions, which, when carried out by a processing unit ( 120 ) of a computing device ( 100 ), cause the computing device ( 100 ) to carry out the method according to claim 1 .

16 . The method of claim 1 wherein the plurality of chirurgical implants ( 20 1-n ) comprise pedicle screws.