Image processing apparatus and image processing method
Based on the position and orientation information of an ultrasonic probe, an image generation unit acquires, from three-dimensional volume data, an image of a slice corresponding to a tomographic image of an object obtained by the ultrasonic probe. An image composition unit composes the image of the slice with the tomographic image to generate and output a composite image. In this composition, a region of the tomographic image is specified as an unclear image sensing region, and the image in the unclear image sensing region is replaced with the image in a region of the image of the slice corresponding to the unclear image sensing region, thereby composing the tomographic image with the image of the slice.
1. An image processing apparatus comprising:
at least one processor; and
a memory, to which the processor is operatively coupled, storing instructions that, when executed by the processor, cause the processor to:
acquire a tomographic image of an object obtained by an image sensing unit;
acquire three-dimensional volume data of the object;
detect an unclear image sensing region on the basis of luminance values within the tomographic image; and
generate an image by complementing the unclear image sensing region using an image of a slice which is in the three dimensional volume data and which corresponds to the unclear image sensing region,
wherein a part of a region of interest is detected from the tomographic image using a method based on edge detection and a region behind the detected part of the region of interest is defined as the unclear image sensing region.
2. The image processing apparatus according to claim 1 , wherein an image sensing clarity is a value obtained by dividing a sum of differences between a threshold and pixel values of pixels of a selected vertical line sequentially selected from the tomographic image by a product of the threshold and the number of pixels of the selected vertical line.
3. The image processing apparatus according to claim 1 , wherein the instructions further cause the image processing apparatus to:
obtain, for each pixel of the tomographic image, a first image sensing clarity that takes a higher value corresponding to a lower pixel value,
obtain, as a second image sensing clarity, a value obtained by dividing a difference between an average pixel value X in a region immediately under the part of the region of interest of the tomographic image and an average pixel value Y in regions adjacent to left and right sides of the region immediately under by the average pixel value Y in the regions adjacent to the left and right sides if Y>=X or a value obtained by subtracting the average pixel value Y in the regions adjacent to the left and right sides from a maximum pixel value that the pixel value can take if Y<X, and
compose the tomographic image with the image of the slice by composing, for each pixel, the tomographic image with the image of the slice in accordance with a ratio calculated based on the corresponding first image sensing clarity and the second image sensing clarity.
4. The image processing apparatus according to claim 1 , wherein the instructions further cause the image processing apparatus to:
obtain, for each pixel of the tomographic image, an image sensing clarity that takes a higher value corresponding to a lower pixel value, and
compose the tomographic image with the image of the slice by composing, for each pixel, the tomographic image with the image of the slice in accordance with a ratio based on a corresponding image sensing clarity and a corresponding reliability, and
wherein the reliability is a value predetermined for each pixel of the image of the slice and predetermined for each voxel of the three-dimensional volume data used to generate the pixels.
5. The image processing apparatus according to claim 4 , wherein the instructions further cause the image processing apparatus to display an image obtained by composing the tomographic image with the image of the slice, and
wherein each pixel of the image obtained by composing the tomographic image is displayed with the image of the slice, in a display form corresponding to a value based on the corresponding image sensing clarity and the corresponding reliability.
6. The image processing apparatus according to claim 1 , wherein the instructions further cause the image processing apparatus to display the generated image, and
wherein, when displaying the generated image, the unclear image sensing region and remaining regions are displayed discriminately.
7. The image processing apparatus according to claim 1 , wherein the instructions further cause the image processing apparatus to display the generated image, and
wherein, when displaying the generated image, unclear image sensing regions of different types are displayed discriminately.
8. The apparatus according to claim 1 , wherein the instructions further cause the image processing apparatus to detect, as the unclear image sensing region, a lacked image region from the tomographic image.
9. The apparatus according to claim 1 , wherein the unclear image sensing region is detected on the basis of a relationship between a pixel value of each pixel within the tomographic image and a threshold.
10. The apparatus according to claim 1 , wherein the unclear image sensing region is detected on the basis of a relationship between a pixel value of each pixel within the tomographic image and pixel values of surrounding pixels thereof.
11. The apparatus according to claim 1 , wherein when pixel values of all pixels within a line is equal to or less than a threshold, the line is defined as the unclear image sensing region.
12. An image processing method comprising:
acquiring a tomographic image of an object obtained by an image sensing unit; acquiring three-dimensional volume data of the object;
detecting an unclear image sensing region on the basis of luminance values within the tomographic image; and
generating an image by complementing the unclear image sensing region using an image of a slice which is in the three dimensional volume data and which corresponds to the unclear image sensing region,
wherein a part of a region of interest is detected from the tomographic image using a method based on edge detection and a region behind the detected part of the region of interest is defined as the unclear image sensing region.
13. An image processing apparatus comprising:
at least one processor;
a memory, to which the processor is operatively coupled, storing instructions that, when executed by the processor, cause the processor to:
acquire a tomographic image of an object obtained by an image sensing unit;
acquire three-dimensional volume data of the object;
calculate image sensing clarity for each pixel of the tomographic image on the basis of a pixel value of each pixel of the tomographic image;
detect an unclear image sensing region on the basis of luminance values within the tomographic image; and
compose the tomographic image with an image of a slice which is in the three-dimensional volume data and which corresponds to the unclear image sensing region by compositing, for each pixel, the tomographic image with the image of the slice in accordance with a ratio represented by a corresponding image sensing clarity,
wherein a part of a region of interest is detected from the tomographic image using a method based on edge detection and a region behind the detected part of the region of interest is defined as the unclear image sensing region.
14. The image processing apparatus according to claim 13 , wherein the image sensing clarity is a value obtained by dividing a sum of differences between a threshold and pixel values of pixels of a selected vertical line sequentially selected from the tomographic image by a product of the threshold and the number of pixels of the selected vertical line.
15. The image processing apparatus according to claim 13 , wherein the instructions further cause the image processing apparatus to:
obtain, for each pixel of the tomographic image, a first image sensing clarity that takes a higher value corresponding to a lower pixel value,
obtain, as a second image sensing clarity, a value obtained by dividing a difference between an average pixel value X in a region immediately under the part of the region of interest of the tomographic image and an average pixel value Y in regions adjacent to left and right sides of the region immediately under by the average pixel value Y in the regions adjacent to the left and right sides if Y>=X or a value obtained by subtracting the average pixel value Y in the regions adjacent to the left and right sides from a maximum pixel value that the pixel value can take if Y<X, and
compose the tomographic image with the image of the slice by composing, for each pixel, the tomographic image with the image of the slice in accordance with a ratio calculated based on the corresponding first image sensing clarity and the second image sensing clarity.
16. The image processing apparatus according to claim 13 , wherein the instructions further cause the image processing apparatus to:
obtain, for each pixel of the tomographic image, an image sensing clarity that takes a higher value corresponding to a lower pixel value, and
compose the tomographic image with the image of the slice by composing, for each pixel, the tomographic image with the image of the slice in accordance with a ratio based on a corresponding image sensing clarity and a corresponding reliability, and
wherein the reliability is a value predetermined for each pixel of the image of the slice and predetermined for each voxel of the three-dimensional volume data used to generate the pixels.
17. The apparatus according to claim 16 , wherein the instructions further cause the image processing apparatus to display an image obtained by composing the tomographic image with the image of the slice, and
wherein each pixel of the image obtained by composing the tomographic image with the image of the slice is displayed in a display form corresponding to a value based on the corresponding image sensing clarity and the corresponding reliability.