IP Library › Granted Patent US 12,096,993

Granted Patent B2

US 12,096,993 · App. 17/434,734 · Granted Sep 24, 2024

Feedback continuous positioning control of end-effectors

Inventors: Grzegorz Andrzej Toporek (Cambridge, MA); Marcin Arkadiusz Balicki (Cambridge, MA); Aleksandra Popovic (Boston, MA)

Assignee: KONINKLIJKE PHILIPS N.V.

A61B34/20 A61B8/12 A61B34/30 A61B90/37 B25J9/1607 B25J9/1697 G06N3/044 G06N3/045 G06N3/047 G06N3/08 G06N3/084 G06T7/246 G06T7/70 G16H20/40 G16H30/20 A61B2034/107 A61B2034/2061 A61B2034/2065 A61B2034/301 A61B2090/378 G06T2207/20084 G06T2207/30244

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12,096,993
App. No.: 17/434,734
Granted: Sep 24, 2024
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

A positioning controller ( 50 ) including an imaging predictive model ( 80 ) and inverse control predictive model ( 70 ). In operation, the controller ( 50 ) applies the imaging predictive model ( 80 ) to imaging data generated by an imaging device ( 40 ) to render a predicted navigated pose of the imaging device ( 40 ), and applies the control predictive model ( 70 ) to error positioning data derived from a differential aspect between a target pose of the imaging device ( 40 ) and the predicted navigated pose of the imaging device ( 40 ) to render a predicted corrective positioning motion of the imaging device ( 40 ) (or a portion of the interventional device associated with this imaging device) to the target pose. From the predictions, the controller ( 50 ) further generates continuous positioning commands controlling a corrective positioning by the interventional device ( 30 ) of the imaging device ( 40 ) (or said portion of interventional device) to the target pose based on the predicted corrective positioning motion of the interventional device ( 30 ).

Claims (67)

1. A positioning controller for an interventional device, the positioning controller comprising:

a memory storing:

an imaging predictive model configured with kinematics of the interventional device to receive imaging data and to output data related to a prediction of a navigated pose of the interventional device, and

a control predictive model configured with kinematics of the interventional device to receive error positioning data and to output data related to a prediction of a corrective positioning motion of the interventional device,

wherein the interventional device includes an imaging device and at least one joint; and

at least one processor in communication with the memory, the at least one processor configured to:

apply the imaging predictive model to imaging data generated by the imaging device of the interventional device to predict a navigated pose of a portion of the interventional device associated with the imaging device,

generate error positioning data derived based on a differential aspect between a target pose of the portion of the interventional device associated with the imaging device and the predicted navigated pose of the portion of the interventional device associated with the imaging device, wherein the error positioning data comprises at least one velocity for the portion of the interventional device associated with the imaging device,

apply the control predictive model to the generated error positioning data to predict a corrective positioning motion of the interventional device, wherein the predicted corrective positioning motion comprises one or more joint velocities for the at least one joint, the one or more joint velocities derived, by the control predictive model, based on the at least one velocity for the portion of the interventional device associated with the imaging device, and

generate positioning commands that control a corrective positioning of the portion of the interventional device associated with the imaging device to the target pose based on the predicted corrective positioning motion.

2. The positioning controller of claim 1 , wherein at least one of:

the imaging predictive model is trained on a correlation of a relative imaging by the imaging device and forward kinematics of the interventional device, and

the control predictive model is an inverse predictive model is trained on inverse kinematics of the interventional device.

3. The positioning controller of claim 2 ,

wherein the inverse predictive model is further trained based on at least one auxiliary navigation parameter of the interventional device that is auxiliary to the inverse kinematics of the interventional device predictive of the corrective positioning motion of the interventional device; and

wherein the at least one processor is further configured to apply the inverse predictive model to both the error positioning data and the at least one auxiliary navigation parameter of the interventional device to predict the corrective positioning motion of the interventional device.

4. The positioning controller of claim 1 , wherein the portion of the interventional device associated with the imaging device comprises an end-effector.

5. The positioning controller of claim 1 , wherein the at least one processor is further configured to generate the positioning commands continuously.

6. The positioning controller of claim 1 , wherein the imaging predictive model includes:

a neural network having an input layer configured to input the imaging data generated by the imaging device, and an output layer configured to output at least one of a translation, a rotation, and a pivoting of the interventional device relative to a position of the interventional device in a reference image,

wherein the neural network is configured to infer the predicted navigated pose of the portion of the interventional device associated with the imaging device based on at least one of the translation, the rotation, and the pivoting of the interventional device.

7. The positioning controller of claim 1 , wherein the imaging predictive model includes:

a neural network having an input layer configured to input the imaging data generated by the imaging device, and an output layer configured to output at least one of a linear velocity and an angular velocity of the interventional device relative to a position of the interventional device in a reference image,

wherein the neural network is configured to infer the predicted navigated pose of the portion of the interventional device associated with the imaging device based on the at least one of the linear velocity and the angular velocity of the interventional device.

8. The positioning controller of claim 1 , wherein the imaging predictive model is configured to:

track a motion of features in the imaging data generated by the imaging device; and

output at least one of a linear velocity and an angular velocity of the interventional device derived from a Jacobian transformation of the tracked motion of features in the imaging data,

9. The positioning controller of claim 8 , wherein the imaging predictive model is further configured to:

detect the features from the imaging data generated by the imaging device; and

track the motion of the detected features.

10. The positioning controller of claim 1 , wherein the control predictive model includes:

a neural network having an input layer configured to input the error positioning data derived from a differential pose between the target pose of the portion of the interventional device associated with the imaging device and the predicted navigated pose of the portion of the interventional device associated with the imaging device, and an output layer configured to output at least one of a translation, a rotation, and a pivoting of the interventional device derived from a regression of the differential pose between the target pose of the portion of the interventional device associated with the imaging device and the predicted navigated pose of the portion of the interventional device associated with the imaging device,

wherein the neural network is configured to infer the predicted corrective positioning motion of the interventional device based on the at least one of the translation, the rotation, and the pivoting of the interventional device.

11. The positioning controller of claim 1 , wherein the control predictive model includes:

a neural network having an input layer configured with the error positioning data derived from a differential motion between at least one of a linear velocity and an angular velocity of the portion of the interventional device associated with the imaging device and at least one of a predicted linear velocity and a predicted angular velocity of the portion of the interventional device associated with the imaging device, and an output layer configured to output the one or more joint velocities of the interventional device from a regression of the differential motion between the at least one of the linear velocity and the angular velocity of the portion of the interventional device associated with the imaging device and the at least one of the predicted linear velocity and the predicted angular velocity of the portion of the interventional device associated with the imaging device.

12. The positioning controller of claim 1 ,

wherein the control predictive model is configured to further receive at least one auxiliary navigation parameter of the interventional device, and further process the at least one auxiliary navigation parameter to output the predicted corrective positioning motion of the interventional device; and

wherein the at least one processor is further configured to apply the control predictive model to both the error positioning data and the at least one auxiliary navigation parameter of the interventional device to predict the corrective positioning motion of the interventional device.

13. A non-transitory computer-readable storage medium having stored a computer program comprising instructions, which, when executed by a processor, cause the processor to:

apply an imaging predictive model to imaging data generated by an imaging device of an interventional device to predict a navigated pose of a portion of the interventional device associated with the imaging device, wherein the imaging predictive model is configured with kinematics of the interventional device to receive the imaging data and to output the predicted navigated pose of the interventional device;

generate an error positioning data derived from a differential aspect between a target pose of a portion of the interventional device associated with the imaging device and the predicted navigated pose of the portion of the interventional device associated with the imaging device, wherein the error positioning data comprises at least one velocity for the portion of the interventional device associated with the imaging device,

apply a control predictive model to the error positioning data to predict a corrective positioning motion of the interventional device, wherein the control predictive model is configured with kinematics of the interventional device to receive the error positioning data and to output the predicted corrective positioning motion of the interventional device,

wherein the predicted corrective positioning motion comprises one or more joint velocities for at least one joint of the interventional device, the one or more joint velocities derived, by the control predictive model, based on the at least one velocity for the portion of the interventional device associated with the imaging device; and

generate positioning commands that control a corrective positioning by the interventional device of the portion of the interventional device associated with the imaging device to the target pose based on the predicted corrective positioning motion of the interventional device.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the imaging predictive model includes:

15. The non-transitory computer-readable storage medium of claim 13 , wherein the imaging predictive model includes:

16. The non-transitory computer-readable storage medium of claim 13 , wherein the control predictive model includes:

a neural network having an input layer configured with the error positioning data derived from a differential motion between at least one of a linear velocity and an angular velocity of the portion of the interventional device associated with the imaging device and at least one of a predicted linear velocity and a predicted angular velocity of the portion of the interventional device associated with the imaging device, and an output layer configured to output the one or more joint velocities of the interventional device from a regression of the differential motion between the at least one of the linear velocity and the angular velocity and the at least one of the predicted linear velocity and the predicted angular velocity.

17. A positioning method for an interventional device including an imaging device, the method comprising

applying an imaging predictive model to imaging data generated by the imaging device to predict a navigated pose of a portion of the interventional device associated with the imaging device, wherein the imaging predictive model is configured with kinematics of the interventional device to receive the imaging data and to output the predicted navigated pose of the interventional device;

generating error positioning data derived from a differential aspect between a target pose of the portion of the interventional device associated with the imaging device and the predicted navigated pose of the portion of the interventional device associated with the imaging device, wherein the error positioning data comprises at least one velocity for the portion of the interventional device associated with the imaging device;

applying a control predictive model to the error positioning data to predict a corrective positioning motion of the interventional device, wherein the control predictive model configured with kinematics of the interventional device to receive the error positioning data and to output the predicted corrective positioning motion of the interventional device,

generating positioning commands controlling a corrective positioning of the portion of the interventional device associated with the imaging device to the target pose based on the predicted corrective positioning motion of the interventional device.

18. The positioning method of claim 17 , wherein the imaging predictive model includes:

19. The positioning method of claim 17 , wherein the imaging predictive model includes:

a neural network having an input layer configured to input the imaging data generated by the imaging device, and an output layer configured to output at least one of a linear velocity and an angular velocity of interventional device relative to a position of the interventional device in a reference image,

20. The positioning method of claim 17 , wherein the control predictive model includes:

a neural network having an input layer configured with the error positioning data derived from a differential motion between at least one of a linear velocity and an angular velocity of the portion of the interventional device associated with the imaging device and at least one of a predicted linear velocity and a predicted angular velocity of the portion of the interventional device associated with the imaging device, and an output layer configured to output the one or more joint velocities of the imaging device from a regression of the differential motion between the at least one of the linear velocity and the angular velocity and the at least one of the predicted linear velocity and the predicted angular velocity.

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 28, 2021

From: TOPOREK, GRZEGORZ ANDRZEJ; BALICKI, MARCIN ARKADIUSZ; POPOVIC, ALEKSANDRA

To: KONINKLIJKE PHILIPS N.V.

Reel/Frame 057319/0688 →

Continuity (4)

Provisional Application 62825914 · Mar 29, 2019

Provisional Application 62825905 · Mar 29, 2019

Provisional Application 62811705 · Feb 28, 2019

Related Publication 20220125530A1 · Apr 28, 2022

Cited By (4)

US 12,400,355 US 12,433,690 US 12,527,628 US 12,636,048