System and method for real-time ultrasound guided prostate needle biopsy based on biomechanical model of the prostate from magnetic resonance imaging data

1. A method comprising, comprising:

segmenting a prostate in a 3D ultrasound image of a patient by mapping a binary mask of the reference patient-specific biomechanical model of the prostate to the 2D ultrasound image;

fusing a reference patient-specific biomechanical model of the prostate extracted from a 3D planning image of the patient to a boundary of the segmented prostate in the 3D ultrasound image, resulting in a fused 3D biomechanical prostate model;

in response to movement of an ultrasound probe to a new location, receiving a current 2D ultrasound image acquired by the ultrasound probe at the new location; and

deforming the fused 3D biomechanical prostate model based on a prostate contour in the current 2D ultrasound image to match a current deformation of the prostate due to the movement of the ultrasound probe to the new location.

2. The method of claim 1 , further comprising:

fusing the reference patient-specific biomechanical model of the prostate to a boundary of the deformed fused 3D biomechanical prostate model, resulting in a new fused 3D biomechanical prostate model.

3. The method of claim 2 , further comprising:

mapping locations of suspicious regions annotated in the new fused 3D biomechanical prostate model to the current 2D ultrasound image; and

displaying the current 2D ultrasound image with the mapped locations of suspicious regions.

4. The method of claim 3 , wherein displaying the current 2D ultrasound image with the mapped locations of suspicious regions comprises:

displaying the locations of suspicious regions as one of cross-hairs overlaid onto current 2D ultrasound image or contours overlaid onto current 2D ultrasound image showing boundaries of the suspicious regions using different line-styles or colors.

5. The method of claim 2 , further comprising:

displaying a segmented 2D boundary of the prostate in the current 2D ultrasound image as an overlaid contour on the current 2D ultrasound image using a first line style or color; and

displaying a cross section of the new fused 3D biomechanical prostate model in the imaging plane of the current 2D ultrasound image as an overlaid contour on the current 2D ultrasound image using a second line style or color.

6. The method of claim 1 , wherein the reference patient-specific biomechanical model of the prostate comprises a patient-specific anatomical model of the prostate extracted from the 3D planning image of the patient and biomechanical properties of the prostate.

7. The method of claim 6 , wherein the biomechanical properties of the prostate are patient-specific biomechanical properties extracted from elastography scans of the patient's prostate.

8. The method of claim 6 , wherein the current 2D ultrasound image is received in real time during a needle biopsy procedure, and the steps of fusing the reference patient-specific biomechanical model of the prostate to a boundary of the deformed fused 3D biomechanical prostate model, mapping locations of suspicious regions annotated in the new fused 3D biomechanical prostate model to the current 2D ultrasound image, and displaying the current 2D ultrasound image with the mapped locations of suspicious regions are performed in real time in response to receiving the current 2D ultrasound image.

9. The method of claim 1 , wherein the 3D planning image is a 3D magnetic resonance image.

10. The method of claim 1 , further comprising:

acquiring a plurality of tracked ultrasound images with a 3D sweep of the prostate by the ultrasound probe; and

generating the 3D ultrasound image from the plurality of tracked ultrasound images.

11. The method of claim 1 , wherein segmenting a prostate in a 3D ultrasound image of a patient comprises:

shrinking the mapped binary mask by a predetermined percentage;

automatically selecting foreground seeds from within the shrunken binary mask;

expanding the mapped binary mask by a predetermined percentage;

automatically selecting background seeds outside of the expanded binary mask; and

automatically segmenting the prostate in the 3D ultrasound image based on the automatically selected foreground seeds and the automatically selected background seeds.

12. The method of claim 1 , wherein fusing a reference patient-specific biomechanical model of the prostate extracted from a 3D planning image of the patient to a boundary of the segmented prostate in the 3D ultrasound image comprises:

registering the reference patient-specific biomechanical model to the boundary of the segmented prostate in the 3D ultrasound based on differences between a boundary of the reference patient-specific biomechanical model and the boundary of the segmented prostate in the 3D ultrasound and biomechanical properties of the reference patient-specific biomechanical model using rigid and deformable registrations.

13. The method of claim 1 , wherein deforming the fused 3D

biomechanical prostate model based on a prostate contour in the current 2D ultrasound image to match a current deformation of the prostate due to the movement of the ultrasound probe to the new location comprises:

deforming the fused 3D biomechanical prostate model based on the prostate contour extracted from the 2D ultrasound image, a 3D position and orientation of the ultrasound probe, and biomechanical properties of the fused 3D biomechanical prostate model.

14. The method of claim 1 , wherein deforming the fused 3D biomechanical prostate model based on a prostate contour in the current 2D ultrasound image to match a current deformation of the prostate due to the movement of the ultrasound probe to the new location comprises:

extracting a first prostate contour from the current 2D ultrasound image;

reconstructing a second prostate contour from the fused 3D biomechanical prostate model in the imaging plane of the current 2D ultrasound image using a tracked 3D position and orientation of the ultrasound probe;

generating a deformation map showing deformations that map points on the second prostate contour to corresponding points on the first prostate contour; and

deforming the fused 3D biomechanical prostate model based on the deformation map.

15. The method of claim 14 , wherein deforming the fused 3D biomechanical prostate model based on the deformation map comprises:

prescribing the deformation map onto the fused 3D biomechanical prostate model;

deforming a boundary of the fused 3D biomechanical prostate model in the imaging plane of the current 2D ultrasound image according to the deformation map; and

propagating the deformation of the boundary of the fused 3D biomechanical prostate model outside of the imaging plane of the current 2D ultrasound image using biomechanical properties of the fused 3D biomechanical prostate model.

16. An apparatus, comprising:

a processor; and

a memory storing computer program instructions, which when executed by the processor cause the processor to perform operations comprising:

segmenting a prostate in a 3D ultrasound image of a patient by mapping a binary mask of the reference patient-specific biomechanical model of the prostate to the 2D ultrasound image;

fusing a reference patient-specific biomechanical model of the prostate extracted from a 3D planning image of the patient to a boundary of the segmented prostate in the 3D ultrasound image, resulting in a fused 3D biomechanical prostate model;

receiving a current 2D ultrasound image acquired by the ultrasound probe that has moved to a new location; and

deforming the fused 3D biomechanical prostate model based on a prostate contour in the current 2D ultrasound image to match a current deformation of the prostate due to the movement of the ultrasound probe to the new location.

17. The apparatus of claim 16 , wherein the operations further comprise:

fusing the reference patient-specific biomechanical model of the prostate to a boundary of the deformed fused 3D biomechanical prostate model, resulting in a new fused 3D biomechanical prostate model.

18. The apparatus of claim 16 , wherein the reference patient-specific biomechanical model of the prostate comprises a patient-specific anatomical model of the prostate extracted from the 3D planning image of the patient and biomechanical properties of the prostate.

19. The apparatus of claim 16 , wherein the 3D planning image is a 3D magnetic resonance image.

20. The apparatus of claim 16 , wherein deforming the fused 3D biomechanical prostate model based on a prostate contour in the current 2D ultrasound image to match a current deformation of the prostate due to the movement of the ultrasound probe to the new location comprises:

deforming the fused 3D biomechanical prostate model based on the prostate contour extracted from the 2D ultrasound image, a 3D position and orientation of the ultrasound probe, and biomechanical properties of the fused 3D biomechanical prostate model.

21. The apparatus of claim 16 , wherein deforming the fused 3D biomechanical prostate model based on a prostate contour in the current 2D ultrasound image to match a current deformation of the prostate due to the movement of the ultrasound probe to the new location comprises:

extracting a first prostate contour from the current 2D ultrasound image;

reconstructing a second prostate contour from the fused 3D biomechanical prostate model in the imaging plane of the current 2D US image using a tracked 3D position and orientation of the ultrasound probe;

generating a deformation map showing deformations that map points on the second prostate contour to corresponding points on the first prostate contour; and

deforming the fused 3D biomechanical prostate model based on the deformation map.

22. A non-transitory computer readable medium storing computer program instructions which when executed by a processor cause the processor to perform operations comprising:

segmenting a prostate in a 3D ultrasound image of a patient by mapping locations of suspicious regions annotated in the new fused 3D biomechanical prostate model to the current 2D ultrasound image;

fusing a reference patient-specific biomechanical model of the prostate extracted from a 3D planning image of the patient to a boundary of the segmented prostate in the 3D ultrasound image, resulting in a fused 3D biomechanical prostate model;

in response to movement of an ultrasound probe to a new location, receiving a current 2D ultrasound image acquired by the ultrasound probe at the new location; and

deforming the fused 3D biomechanical prostate model based on a prostate contour in the current 2D ultrasound image to match a current deformation of the prostate due to the movement of the ultrasound probe to the new location.

23. The non-transitory computer readable medium of claim 22 , wherein the operations further comprise:

fusing the reference patient-specific biomechanical model of the prostate to a boundary of the deformed fused 3D biomechanical prostate model, resulting in a new fused 3D biomechanical prostate model.

24. The non-transitory computer readable medium of claim 23 , wherein the operations further comprise:

displaying the current 2D ultrasound image with the mapped locations of suspicious regions.

25. The non-transitory computer readable medium of claim 22 , wherein the reference patient-specific biomechanical model of the prostate comprises a patient-specific anatomical model of the prostate extracted from the 3D planning image of the patient and biomechanical properties of the prostate.

26. The non-transitory computer readable medium of claim 22 , wherein the 3D planning image is a 3D magnetic resonance image.

27. The non-transitory computer readable medium of claim 22 , wherein the operations further comprise:

acquiring a plurality of tracked ultrasound images with a 3D sweep of the prostate by the ultrasound probe; and

generating the 3D ultrasound image from the plurality of tracked ultrasound images.

28. The non-transitory computer readable medium of claim 22 , wherein fusing a reference patient-specific biomechanical model of the prostate extracted from a 3D planning image of the patient to a boundary of the segmented prostate in the 3D ultrasound image comprises:

registering the reference patient-specific biomechanical model to the boundary of the segmented prostate in the 3D ultrasound based on differences between a boundary of the reference patient-specific biomechanical model and the boundary of the segmented prostate in the 3D ultrasound and biomechanical properties of the reference patient-specific biomechanical model using rigid and deformable registrations.

29. The non-transitory computer readable medium of claim 22 , wherein deforming the fused 3D biomechanical prostate model based on a prostate contour in the current 2D ultrasound image to match a current deformation of the prostate due to the movement of the ultrasound probe to the new location comprises:

deforming the fused 3D biomechanical prostate model based on the prostate contour extracted from the 2D ultrasound image, a 3D position and orientation of the ultrasound probe, and biomechanical properties of the fused 3D biomechanical prostate model.

30. The non-transitory computer readable medium of claim 22 , wherein deforming the fused 3D biomechanical prostate model based on a prostate contour in the current 2D ultrasound image to match a current deformation of the prostate due to the movement of the ultrasound probe to the new location comprises:

extracting a first prostate contour from the current 2D ultrasound image;

reconstructing a second prostate contour from the fused 3D biomechanical prostate model in the imaging plane of the current 2D US image using a tracked 3D position and orientation of the ultrasound probe;

generating a deformation map showing deformations that map points on the second prostate contour to corresponding points on the first prostate contour; and

deforming the fused 3D biomechanical prostate model based on the deformation map.

31. The non-transitory computer readable medium of claim 30 , wherein deforming the fused 3D biomechanical prostate model based on the deformation map comprises:

prescribing the deformation map onto the fused 3D biomechanical prostate model;

deforming a boundary of the fused 3D biomechanical prostate model in the imaging plane of the current 2D ultrasound image according to the deformation map; and

propagating the deformation of the boundary of the fused 3D biomechanical prostate model outside of the imaging plane of the current 2D ultrasound image using biomechanical properties of the fused 3D biomechanical prostate model.