3D reconstruction of cross-sectional cutting planes in IVUS images
An image processing device for causing a display to display, as a three-dimensional image, three-dimensional data representing a biological tissue includes: a control unit configured to form, in the three-dimensional data, a cutting region exposing a lumen of the biological tissue in the three-dimensional image, and cause the display to display, together with the three-dimensional image, a two-dimensional image representing a cross section of the biological tissue and a region corresponding to the cutting region in the cross section.
1 . An image processing device for causing a display to display, as a three-dimensional image, three-dimensional data representing a biological tissue, the image processing device comprising:
a processor configured to:
acquire tomographic data of the biological tissue obtained by a sensor configured to acquire the tomographic data while moving through a lumen of the biological tissue;
generate the three-dimensional data based on the tomographic data;
cause the display to display a two-dimensional image representing a cross section indicated by the tomographic data;
set, as one point, a centroid position of the cross section represented by the two-dimensional image;
display, on the two-dimensional image, a first straight line and a second straight line that extend from the one point;
accept a first operation by a user of setting a direction of the first straight line from the one point and a second operation by the user of setting an angle formed by the first and second straight lines, thereby setting a two-dimensional cutting region partitioned by the first and second straight lines;
display, on the two-dimensional image, the two-dimensional cutting region by coloring, together with the biological tissue included in the two-dimensional cutting region, the two-dimensional cutting region with a color different from a color of a remaining region, such that the biological tissue within the two-dimensional cutting region remains entirely visible on the two-dimensional image;
set, as cutting planes, two planes that intersect along any line passing through the one point and that respectively include the first and second straight lines;
form, in the three-dimensional data, a three-dimensional cutting region sandwiched between the cutting planes and exposing the lumen of the biological tissue in the three-dimensional image;
set, based on a position of the two-dimensional cutting region, a viewpoint such that the three-dimensional image is displayed on a screen as seen from the viewpoint;
display, according to the position of the two-dimensional cutting region, a virtual camera representing the viewpoint on the two-dimensional image; and
cause the display to display on the screen, as the three-dimensional image as seen from the viewpoint, the three-dimensional data in which the three-dimensional cutting region is formed, together with the two-dimensional image on which the two-dimensional cutting region and the virtual camera representing the viewpoint are displayed.
2 . The image processing device according to claim 1 , wherein
the processor is configured to cause the display to display, as the three-dimensional image, a three-dimensional image in which at least a voxel representing an inner surface of the biological tissue or a voxel that is adjacent to the voxel representing the inner surface and that represents the lumen among a first voxel group corresponding to the cross section represented by the two-dimensional image is colored in a manner of being distinguished from a second voxel group corresponding to another cross section of the biological tissue, together with the two-dimensional image.
3 . The image processing device according to claim 2 , wherein
the processor is configured to cause the display to display, as the three-dimensional image, a three-dimensional image in which all of voxels representing the biological tissue among the first voxel group are colored in a manner of being distinguished from the second voxel group, together with the two-dimensional image.
4 . An image processing system comprising:
the image processing device according to claim 1 ;
a probe including the sensor.
5 . The image processing system according to claim 4 , further comprising the display.
6 . The image processing device according to claim 1 , wherein
the processor is configured to accept, as the first operation, an operation of setting a base angle that is a rotation angle about the one point for the first straight line, and, as the second operation, an operation of setting an opening angle that is the angle formed by the first and second straight lines, thereby setting the two-dimensional cutting region.
7 . The image processing device according to claim 1 , wherein
the processor is configured to cause the display to display, as the two-dimensional image, a latest cross-sectional image corresponding to a current position of the sensor.
8 . The image processing device according to claim 1 , wherein
the processor is configured to, during an operation by the user for changing the position of the two-dimensional cutting region on the two-dimensional image, display the virtual camera on the two-dimensional image at a position corresponding to the position of the two-dimensional cutting region before being changed.
9 . An image display method for displaying on a display, as a three-dimensional image, three-dimensional data representing a biological tissue, the image display method comprising:
acquiring, by a processor, tomographic data of the biological tissue obtained by a sensor configured to acquire the tomographic data while moving through a lumen of the biological tissue;
generating, by the processor, the three-dimensional data based on the tomographic data;
displaying, on the display, a two-dimensional image representing a cross section indicated by the tomographic data;
setting, by the processor, as one point, a centroid position of the cross section represented by the two-dimensional image;
displaying, by the processor, on the two-dimensional image, a first straight line and a second straight line that extend from the one point;
accepting, by the processor, a first operation by a user of setting a direction of the first straight line from the one point and a second operation by the user of setting an angle formed by the first and second straight lines, thereby setting a two-dimensional cutting region partitioned by the first and second straight lines;
displaying, by the processor, on the two-dimensional image, the two-dimensional cutting region by coloring, together with the biological tissue included in the two-dimensional cutting region, the two-dimensional cutting region with a color different from a color of a remaining region, such that the biological tissue within the two-dimensional cutting region remains entirely visible on the two-dimensional image;
setting, by the processor, as cutting planes, two planes that intersect along any line passing through the one point and that respectively include the first and second straight lines;
forming, by the processor, in the three-dimensional data, a three-dimensional cutting region sandwiched between the cutting planes and exposing the lumen of the biological tissue in the three-dimensional image;
setting, by the processor, based on a position of the two-dimensional cutting region, a viewpoint such that the three-dimensional image is displayed on a screen as seen from the viewpoint;
displaying, by the processor, according to the position of the two-dimensional cutting region, a virtual camera representing the viewpoint on the two-dimensional image; and
displaying on the screen, on the display, as the three-dimensional image as seen from the viewpoint, the three-dimensional data in which the three-dimensional cutting region is formed, together with the two-dimensional image on which the two-dimensional cutting region and the virtual camera representing the viewpoint are displayed.
10 . The image display method according to claim 9 , further comprising:
displaying, on the display, as the three-dimensional image, a three-dimensional image in which at least a voxel representing an inner surface of the biological tissue or a voxel that is adjacent to the voxel representing the inner surface and that represents the lumen among a first voxel group corresponding to the cross section represented by the two-dimensional image is colored in a manner of being distinguished from a second voxel group corresponding to another cross section of the biological tissue, together with the two-dimensional image.
11 . The image display method according to claim 10 , further comprising:
displaying, on the display, as the three-dimensional image, a three-dimensional image in which all of voxels representing the biological tissue among the first voxel group are colored in a manner of being distinguished from the second voxel group, together with the two-dimensional image.
12 . The image display method according to claim 9 , further comprising:
accepting, by the processor, as the first operation, an operation of setting a base angle that is a rotation angle about the one point for the first straight line, and, as the second operation, an operation of setting an opening angle that is the angle formed by the first and second straight lines, thereby setting the two-dimensional cutting region.
13 . The image display method according to claim 12 , further comprising:
displaying, on the display, as the two-dimensional image, a latest cross-sectional image corresponding to a current position of the sensor.
14 . The image display method according to claim 12 , further comprising:
during an operation by the user for changing the position of the two-dimensional cutting region on the two-dimensional image, displaying the virtual camera on the two-dimensional image at a position corresponding to the position of the two-dimensional cutting region before being changed.
15 . A non-transitory computer readable medium storing an image processing program which, when executed by a computer, causes a display to display, as a three-dimensional image, three-dimensional data representing a biological tissue, and performs, by a processor, processing comprising:
acquiring tomographic data of the biological tissue obtained by a sensor configured to acquire the tomographic data while moving through a lumen of the biological tissue;
generating the three-dimensional data based on the tomographic data;
causing the display to display a two-dimensional image representing a cross section indicated by the tomographic data;
setting, as one point, a centroid position of the cross section represented by the two-dimensional image;
displaying, on the two-dimensional image, a first straight line and a second straight line that extend from the one point;
accepting a first operation by a user of setting a direction of the first straight line from the one point and a second operation by the user of setting an angle formed by the first and second straight lines, thereby setting a two-dimensional cutting region partitioned by the first and second straight lines;
displaying, on the two-dimensional image, the two-dimensional cutting region by coloring, together with the biological tissue included in the two-dimensional cutting region, the two-dimensional cutting region with a color different from a color of a remaining region, such that the biological tissue within the two-dimensional cutting region remains entirely visible on the two-dimensional image;
setting as cutting planes, two planes that intersect along any line passing through the one point and that respectively include the first and second straight lines;
forming, in the three-dimensional data, a three-dimensional cutting region sandwiched between the cutting planes and exposing the lumen of the biological tissue in the three-dimensional image;
setting, based on a position of the two-dimensional cutting region, a viewpoint such that the three-dimensional image is displayed on a screen as seen from the viewpoint;
displaying, according to the position of the two-dimensional cutting region, a virtual camera representing the viewpoint on the two-dimensional image; and
causing the display to display on the screen, as the three-dimensional image as seen from the viewpoint, the three-dimensional data in which the three-dimensional cutting region is formed, together with the two-dimensional image on which the two-dimensional cutting region and the virtual camera representing the viewpoint are displayed.
16 . The non-transitory computer readable medium according to claim 15 , wherein
the processing further comprises accepting, as the first operation, an operation of setting a base angle that is a rotation angle about the one point for the first straight line, and, as the second operation, an operation of setting an opening angle that is the angle formed by the first and second straight lines, thereby setting the two-dimensional cutting region.
17 . The non-transitory computer readable medium according to claim 15 , wherein
the processing further comprises causing the display to display, as the two-dimensional image, a latest cross-sectional image corresponding to a current position of the sensor.
18 . The non-transitory computer readable medium according to claim 15 , wherein
the processing further comprises, during an operation by the user for changing the position of the two-dimensional cutting region on the two-dimensional image, displaying the virtual camera on the two-dimensional image at a position corresponding to the position of the two-dimensional cutting region before being changed.