Training data collection for machine learning models
A training data collection method for an interventional device ( 130 ) including a portion of an interventional device ( 140 ) and sensors ( 332 ) adapted to provide position and/or orientation and/or shape information with at least a part of the sensors being affixed to said portion of device ( 140 ). The method involves controlling one or more motion variables of the interventional device ( 130 ) in accordance with a pre-defined data point pattern, and determining, from shape data derived from said information, an estimating of a pose of said device portion ( 140 ) and an estimating of a positioning motion of the interventional device ( 130 ). The method further involves a storage of a temporal data sequence for the interventional device ( 130 ) derived from the estimated pose of the end-effector ( 140 ) for each data point, the estimated positioning motion of the interventional device ( 130 ) for each data point, and the motion variable(s) of the interventional device ( 130 ) for each data point.
1. A training data collection system for an interventional device, the training data collection system comprising:
a robot controller configured to control at least one motion variable of the interventional device, wherein the interventional device includes a device portion and one or more sensors configured to provide at least one of position information, orientation information, or shape information, and wherein the one or more sensors are affixed to the interventional device, wherein the at least one motion variable includes a set of joint variables associated with motion of the interventional device;
a data acquisition controller configured to command the robot controller to control the at least one motion variable of the interventional device in accordance with a pre-defined data point pattern;
a positioning determination processor configured to:
receive an image of a patient anatomy,
process the received image via an image predictive model to output image inference data,
determine positioning information based on the image inference data and at least one of the position information, the orientation information, or the shape information received from the one or more sensors,
at each data point of the pre-defined data point pattern, estimate a pose of the device portion based on the positioning information and estimate a positioning motion of the interventional device based on the positioning information, wherein estimation of the positioning motion comprises generation of a temporal motion sequence of the set of joint variables at each former data point to reach the pose of the device portion, and
output the estimated pose of the device portion at each data point and the estimated positioning motion of the interventional device at each data point; and
a data storage controller configured to:
throughout the robot controller controlling the at least one motion variable of the interventional device in accordance with the pre-defined data point pattern, store a temporal data sequence configured to train a model to infer kinematics for the interventional device, wherein the temporal data sequence derived from the estimated pose of the device portion at each data point and the temporal motion sequence of the set of joint variables to reach the estimated pose of the device portion at each data point.
2. The training data collection system of claim 1 , wherein the one or more sensors are in a guidewire.
3. The training data collection system of claim 1 , wherein the pre-defined data point pattern includes one of a spiral data point pattern, a radial data point pattern, or a square data point pattern.
4. The training data collection system of claim 1 , wherein the positioning determination processor is configured to estimate the pose of the device portion based on matching a curvature shape of the one or more sensors affixed to the device portion to a template.
5. The training data collection system of claim 1 , wherein the data storage controller is configured to continuously command the robot controller to continuously control the at least one motion variable of the interventional device in accordance with the pre-defined data point pattern.
6. The training data collection system of claim 1 , wherein, for each data point, the temporal data sequence includes:
the estimated pose of the device portion,
a sequence of consecutive shapes of the one or more sensors, and
a sequence of consecutive joint variables of the set of joint variables.
7. The training data collection system of claim 1 , wherein the positioning determination processor is configured to receive shape data or to derive shape data from the at least one of the position information, the orientation information, or the shape information received from the one or more sensors to determine the estimated pose of the device portion at each data point and the estimated positioning motion of the interventional device at each data point.
8. The training data collection system of claim 1 , wherein the one or more sensors comprise a marking visible in an imaging system to be used to visualize a region of interest comprising the interventional device, one or more electromagnetic tracking sensors, and at least one of one or more transducer sensors or one or more optical shape sensing elements provided by an optical fiber.
9. The training data collection system of claim 1 ,
wherein the one or more sensors comprise one or more optical shape sensors and the one or more sensors affixed to the device portion comprise a segment of the one or more optical shape sensors,
wherein the position determination processor is configured to:
control a shape sensing of the one or more optical shape sensors, and estimate the pose of the device portion from the segment of the one or more
optical shape sensors affixed to the device portion.
10. The training data collection system of claim 1 , wherein the one or more sensors are affixed to the device portion in a fixed shape.
11. The training data collection system of claim 1 , wherein the device portion comprises an end-effector of the interventional device.
12. The training data collection system of claim 1 , further comprising the interventional device.
13. A non-transitory machine-readable storage medium having stored a computer program comprising instructions, which, when executed by a processor, cause the processor to:
control at least one motion variable of an interventional device in accordance with a pre-defined data point pattern, wherein the interventional device includes a device portion and one or more sensors configured to provide at least one of position information, orientation information, or shape information, and wherein the one or more sensors are affixed to the interventional device, wherein the at least one motion variable includes a set of joint variables associated with movement of the interventional device;
receive an image of a patient anatomy;
process the received image via an image predictive model to output image inference data;
determine positioning information based on the image inference data and at least one of the position information, the orientation information, or the shape information received from the one or more sensors;
at each data point of the pre-defined data point pattern, estimate a pose of the device portion based on the positioning information and estimate a positioning motion of the interventional device based on the positioning information, wherein estimation of the positioning motion comprises generation of a temporal motion sequence of the set of joint variables at each former data point to reach the pose of the device portion; and
throughout the control of the at least one motion variable of the interventional device in accordance with the pre-defined data point pattern, store a temporal data sequence configured to train a model to infer kinematics for the interventional device, wherein the temporal data sequence derived from the estimated pose of the device portion at each data point and the temporal motion sequence of the set of joint variables to reach the estimated pose of the device portion at each data point.
14. The non-transitory machine-readable storage medium of claim 13 , wherein the pre-defined data point pattern includes one of a spiral data point pattern, a radial data point pattern or a square data point pattern.
15. The non-transitory machine-readable storage medium of claim 13 , wherein, for each data point, the temporal data sequence includes:
the estimated pose of the device portion,
a sequence of consecutive shapes of the one or more sensors, and
a sequence of consecutive joint variables of the set of joint variables.
16. The non-transitory machine-readable storage medium of claim 13 , wherein the one or more sensors comprise one or more optical shape sensors and the one or more sensors affixed to the device portion comprise a segment of the one or more optical shape sensors, wherein determining position information comprises controlling a shape sensing of the one or more optical shape sensors, and estimate the pose of the device portion from the segment of the one or more optical shape sensors affixed to the device portion.
17. A training data collection method for an interventional device, the training data collection method comprising:
controlling at least one motion variable of the interventional device in accordance with a pre-defined data point pattern, wherein the interventional device includes a device portion and one or more sensors configured to provide at least one of position information, orientation information, or shape information, and wherein the one or more sensors are affixed to-said the interventional device, wherein the at least one motion variable includes a set of joint variables associated with movement of the interventional device;
receive an image of a patient anatomy;
process the received image via an image predictive model to output image inference data;
determining positioning information based on the image inference data and at least one of the position information, the orientation information, or the shape information received from the one or more sensors;
at each data point of the pre-defined data point pattern, estimating a pose of the device portion based on the positioning information and estimating a positioning motion of the interventional device based on the positioning information, wherein estimating the positioning motion comprises generating of a temporal motion sequence of the set of joint variables at each former data point to reach the pose of the device portion; and
throughout the controlling the at least one motion variable of the interventional device in accordance with the pre-defined data point pattern, storing a temporal data sequence configured to train a model to infer kinematics for the interventional device, wherein the temporal data sequence derived from the estimated pose of the device portion at each data point and the temporal motion sequence of the set of joint variables to reach the estimated pose of the device portion at each data point.
18. The training data collection method of claim 17 , wherein the pre-defined data point pattern includes one of a spiral data point pattern, a radial data point pattern, or a square data point pattern.
19. The training data collection method of claim 17 , wherein, for each data point, the temporal data sequence includes:
the estimated pose of the device portion,
a sequence of consecutive shapes of the one or more sensors, and
a sequence of consecutive joint variables of the set of joint variables.
20. The training data collection method of claim 17 , wherein the one or more sensors comprise one or more optical shape sensors and the one or more sensors affixed to the device portion comprise a segment of the one or more optical shape sensors, wherein determining position information data comprises controlling a shape sensing of the one or more optical shape sensors, and estimate the pose of the device portion from the segment of the one or more optical shape sensors affixed to the device portion.