Method and apparatus for generating a combined three-dimensional ultrasound image
A method and an apparatus for generating a combined three-dimensional ultrasound image of a body portion are described. Therein, two ultrasound scans are preformed, using a relative tracking system, specifically an inertial tracking, and two partial volume images are generated therefrom. The first and second volumes have an overlapping volume containing a predetermined anatomical structure. Then, respective anatomical structure position data indicative of the position of the predetermined anatomical structure in the respective partial volume image are identified by image segmentation. Then, an initial spatial relationship between the first and second partial volume images is established, based on the identified first and second anatomical structure position data. Then a fine registration algorithm using the initial spatial relationship as an initial input is carried out; and the partial volume images are combined in the common volume using the fine-registered spatial relationship.
1. A method for generating a combined three-dimensional ultrasound image of a body portion, the method comprising:
performing a first ultrasound scan of a first volume of the body portion using an ultrasound probe, whereby the ultrasound probe generates first ultrasound image data, and whereby a relative tracking system generates first relative displacement data by monitoring the relative displacement of the ultrasound probe;
generating a first partial volume image from the first ultrasound image data and from the first relative displacement data;
performing a second ultrasound scan of a second volume of the body portion using the ultrasound probe, whereby the ultrasound probe generates second ultrasound image data, and whereby the relative tracking system generates second relative displacement data by monitoring the relative displacement of the ultrasound probe,
generating a second partial volume image from the second ultrasound image data and from the second relative displacement data,
wherein the first and second volumes have an overlapping volume containing a predetermined anatomical structure of the body portion;
identifying first anatomical structure position data indicative of the position of the predetermined anatomical structure in the first partial volume image by image segmentation;
identifying second anatomical structure position data indicative of the position of the predetermined anatomical structure in the second partial volume image by image segmentation; and
establishing an initial spatial relationship between the first and second partial volume images in a common volume, based on the identified first and second anatomical structure position data;
adjusting the spatial relationship by a fine registration algorithm using the initial spatial relationship as an initial guess; and
combining the first and second partial volume images in the common volume using the adjusted spatial relationship,
wherein the relative tracking system is an inertial tracking system using at least one from a group comprising an accelerometer and a gyroscope, and
wherein segmentation of the first partial volume image and the second partial image volume comprises classifying voxels of the first partial volume image and the second partial image volume as one of multiple classes, either by probability maps or by a class label for each voxel, and outputs a three-dimensional classification output.
2. The method according to claim 1 , wherein establishing the initial spatial relationship is based on data from at least one from the group comprising the accelerometer and the gyroscope, obtained at least in a time period between the first and second ultrasound scans.
3. The method according to claim 1 , wherein establishing the initial spatial relationship is based on an integrated signal from at least one from the group comprising the accelerometer and the gyroscope.
4. The method according to claim 1 , wherein establishing a rotational relationship of the initial spatial relationship is based on data from the gyroscope obtained at least in a time period between the first and second ultrasound scans.
5. The method according to claim 1 , wherein establishing a rotational relationship of the initial spatial relationship is based on an integrated gyroscopic signal.
6. The method according to claim 1 , wherein establishing the initial spatial relationship, which is a translational relationship, is based on the first and second anatomical structure position data.
7. The method according to claim 1 , wherein establishing an initial rotational relationship between the first and second partial volume images is based on data obtained from the gyroscope at least in a time period between the first and second ultrasound scans.
8. The method according to claim 1 , wherein the initial spatial relationship is obtained without any external reference to determine the spatial position and without any input from an absolute tracking system.
9. The method according to claim 1 , wherein the relative tracking system remains activated between the first and second ultrasound scans.
10. The method according to claim 1 , wherein the first and second volumes are different from each other, so that the overlapping volume of the first and second volumes is less than 70 vol % of the first volume and less than 70 vol % of the second volume.
11. The method according to claim 1 , wherein the three-dimensional displacement of the ultrasound probe is monitored by the relative tracking system only, without an absolute position of the ultrasound probe being tracked.
12. The method according to claim 1 , wherein the first and second ultrasound scans are performed during a single ultrasound scan procedure.
13. The method according to claim 1 , wherein the body portion and the predetermined anatomical structure are one of the following:
(i) the body portion includes the thyroid gland, and the predetermined anatomical structure is a trachea portion;
(ii) the body portion includes a blood vessel and is a limb portion or a neck portion, and the predetermined anatomical structure is a bone portion or a muscle portion.
14. The method according to claim 1 , wherein the relative tracking system includes at least one of an Inertial Measurement Unit (IMU) sensor, and image analysis of the ultrasound images, which is by a machine learning module having been trained to determine the relative three-dimensional motion between ultrasound image frames.
15. The method according to claim 1 , wherein the relative tracking system is adapted for determining only relative motion between image frames of the first and second ultrasound image data.
16. The method according to claim 1 , wherein the first and second partial volume images are segmented using a convolutional neural network.
17. The method according to claim 1 , wherein the initial spatial relationship between the first and second partial volume images is established by a position aligning algorithm, the position aligning algorithm being configured for aligning
the position of the predetermined anatomical structure in the first partial volume image and
the position of the predetermined anatomical structure in the second partial volume image
with each other, or with a position of the predetermined anatomical structure in the common volume,
the position aligning algorithm is further configured for
minimizing a spatial misalignment functional as a function of the spatial relationship between the first and second partial volume images, wherein the spatial misalignment functional further depends on the first and second anatomical structure position data.
18. The method according to claim 1 , wherein the fine registration algorithm is a cross-correlation algorithm.
19. The method according to claim 1 , further comprising applying an ultrasound decompression algorithm to the fine-registered spatial relationship.
20. The method of claim 1 , wherein the three-dimensional classification output comprises a map of classification labels and/or probabilities as function of position in the first partial volume image and/or second partial volume image.
21. The method of claim 20 , wherein the image segmentation creates a labeled volume image for the first partial volume image and a labeled volume image for the second partial volume image.
22. The method of claim 1 , wherein the three-dimensional classification output indicates a position of the predetermined anatomical structure in the first partial volume image and/or the second partial volume image.
23. The method of claim 22 , wherein the position of the predetermined anatomical structure in the first partial volume image and/or the second partial volume image comprises three coordinates indicating a position of a representative point of the predetermined anatomical structure.
24. The method of claim 23 , wherein the representative point of the predetermined anatomical structure comprises a center of mass of the predetermined anatomical structure.
25. The method of claim 24 , wherein the three-dimensional classification output further includes an orientation and/or a size of the predetermined anatomical structure.
26. An apparatus for generating a combined three-dimensional ultrasound image of a body portion, the apparatus comprising a relative tracking system and an ultrasound image combining system, wherein
the relative tracking system is fixable to an ultrasound probe and is adapted for generating relative displacement data by monitoring the relative displacement of the ultrasound probe, and wherein
the ultrasound image combining system comprises
an input interface adapted for receiving first and second ultrasound image data generated by the ultrasound probe during a first and second ultrasound image acquisition, respectively, and first and second relative displacement data generated by the relative tracking module during the first and second ultrasound image acquisition, respectively;
an image reconstruction module adapted for reconstructing a first and second partial volume image from the received first and second ultrasound image data and from the received first and second relative displacement data, respectively;
an image segmentation module adapted for identifying first anatomical structure position data indicative of the position of the predetermined anatomical structure in the first partial volume image and for identifying second anatomical structure position data indicative of the position of the predetermined anatomical structure in the second partial volume image by image segmentation of the first and second partial volume image, respectively;
an initial registration module adapted for establishing an initial spatial relationship between the first and second partial volume images in a common volume, based on the identified predetermined anatomical structure in the first and second partial volume images;
a fine registration module adapted for adjusting the spatial relationship by a fine registration algorithm using the initial spatial relationship as an initial guess; and
a combining module adapted for combining the first and second partial volume images in the common volume using the adjusted spatial relationship;
wherein the relative tracking system is an inertial tracking using at least one from a group comprising an accelerometer and a gyroscope, and
wherein segmentation of the first partial volume image and the second partial image volume comprises classifying voxels of the first partial volume image and the second partial image volume as one of multiple classes, either by probability maps or by a class label for each voxel, and outputs a three-dimensional classification output.