System and method for heart strain determination
Heart strain determination includes receiving a series of 2D-slice images as input. A pose estimation module estimates a slicing-pose of the inputted series of 2D-slice images in the heart. A 3D deformation estimation module estimates a 3D deformation field from the series of 2D-slice images and the estimated slicing-pose. A strain measurement module computes a heart strain measure from the 3D deformation field and a predefined definition for strain computation.
1. A system for heart strain determination, the system comprising:
a computer configured to receive a series of 2D-slice images that have been recorded at different times of the cardiac cycle as input, to execute several modules, to pass intermediate result-data from an output of one module to an input of another module, and to output a result, the several modules including:
a pose estimation module configured to estimate a slicing-pose, a position and orientation in space, of the inputted series of 2D-slice images in the heart,
a 3D deformation estimation module configured to estimate a 3D deformation field from the series of 2D-slice images and the estimated slicing-pose, wherein the 3D deformation estimation module is a machine learning model that has been trained with results of the pose estimation module, and
a strain measurement module configured to compute a heart strain measure from the 3D deformation field and a predefined definition for strain computation.
2. The system according to claim 1 , wherein the pose estimation module is a convolutional neural network trained to estimate the pose of the series of 2D-slice images in the heart and/or the 3D deformation estimation module is a physics-informed neural network trained to estimate the 3D-deformation field from the series of 2D-slice images and the pose of these 2D-slice images.
3. The system according to claim 2 , wherein the pose estimation module and the 3D deformation estimation module have been trained with a same data generation device.
4. The system according to claim 1 , wherein the 3D-deformation estimation module is configured for additional determination of a statistical model for the deformations of the 3D deformation field and/or for estimation of cardiac contours.
5. The system according to claim 1 wherein the 3D-deformation estimation module is configured for estimation of a 3D deformation field on the 2D slice along the cardiac contours or along a 3D neighborhood of the contours.
6. The system according to claim 1 , wherein the strain measurement module is a geometric computation module configured to receive as an input the 3D deformation field and to use the predefined definition for strain computation, wherein the strain measurement module is configured to calculate a linear strain as a relative change in length between two states.
7. The system according to claim 1 wherein the strain measurement module is a symmetric 3-tensor.
8. The system according to claim 1 , wherein the pose estimation module or an additional detection module was trained for detecting cardiac contours within the 2D-slice images.
9. The system according to claim 1 , wherein the pose estimation module and/or the 3D deformation estimation module have been trained with data generated by a data generation device comprising:
a data-conditioning module configured to condition and output conditioned 3D+time image sequences of a heart, 3D cardiac chamber masks and cardiac chamber meshes to a slicing module, the data-conditioning module comprising a data interface and a generation module,
a) wherein the conditioned 3D+time image sequences are based on images of a real heart, the data interface is configured for receiving scan data of a heart in form of 3D+time image sequences, and the generation module is configured to generate 3D cardiac chamber masks and cardiac chamber meshes based on the 3D scan data and 3D deformation fields determined from the 3D scan data, and/or
b) wherein the conditioned 3D+time image sequences are based on synthetic conditioned 3D+time image sequences, the data interface is configured for receiving 3D deformation fields form a predefined library and predefined 3D cardiac chamber masks, and the generation module is configured for rendering the synthetic 3D+time image sequences from the 3D cardiac chamber masks and the 3D deformation fields,
a slicing module configured to generate 2D-slice images from the 3D+time image sequences received from the data-conditioning module along a predefined plane, and
a sampling module configured to sample features of interest onto slices.
10. The system according to claim 9 , wherein both the pose estimation module and the 3D deformation estimation module based on neural networks trained with the 2D-slice images generated by the slicing module as training data and the 3D deformation fields, the cardiac chamber meshes, and the conditioned 3D+time image sequences as ground truth.
11. The system according to claim 9 , wherein the conditioned 3D+time image sequences are based on the synthetic conditioned 3D+time image sequences, wherein the generation module is configured to render the synthetic 3D+time image sequences from the 3D cardiac chamber masks, the 3D deformation fields, and cardiac chamber meshes using a GauGAN or TecoGAN algorithm.
12. A method for heart strain determination, the method comprising:
providing a series of 2D-slice images of a cross section of a heart that are recorded at different times,
determining an evaluation of a slicing-pose of the 2D-slices by inputting the 2D-slices in a pose estimation module,
determining a 3D-deformation field by inputting the series of 2D-slice images and the evaluated slicing-pose in a 3D-deformation estimation module, and
determining a 3D-strain by inputting the determined 3D-deformation field in a measurement module.
13. The method according to claim 12 , further comprising detecting cardiac contours within the 2D-slice images.
14. The method according to claim 13 wherein the cardiac contours are detected before determining the 3D-deformation field, wherein the 3D-deformation field is determined for the detected cardiac contours.
15. A method for manufacturing a system for heart strain determination, the method comprising:
providing multiple datasets of different 3D+time image sequences, each of the 3D+time image sequences showing a number of cycles of a beating heart,
providing 3D cardiac chamber masks and cardiac chamber meshes for respective ones of the 3D+time image sequences,
slicing the 3D+time image sequences along a predefined plane to form a 2D-slice image and adding features to the 2D-slice image based on the 3D cardiac chamber mask and cardiac chamber mesh combined with this 3D+time image sequence,
training a pose estimation module and/or a 3D deformation estimation module using the sliced 3D+time image sequences as training data and ground truths based on the 3D+time image sequences and the 3D cardiac chamber masks and the cardiac chamber meshes combined with the 3D+time image sequences.
16. The method of claim 15 , wherein a data generation module trains the pose estimation module and/or the 3D deformation estimation module.
17. The method of claim 16 , wherein a trained pose estimation module is used for training the 3D deformation estimation module.
18. The method of claim 15 , wherein the 3D+time image sequences based on images of a real heart are received and the corresponding 3D deformation fields, the corresponding 3D cardiac chamber masks, and the cardiac chamber meshes are derived from the 3D+time image sequences.
19. The method of claim 15 , wherein the 3D deformation fields are received form a predefined library and predefined 3D cardiac chamber masks are received, and the corresponding synthetic 3D+time image sequences are generated based on 3D deformation fields, the 3D cardiac chamber masks, and the cardiac chamber meshes.
20. The method of claim 19 , wherein the cardiac chamber meshes are received, and the synthetic 3D+time image sequences are generated using a GauGAN or TecoGAN algorithm.